5 tips for data professionals moving to the Data Cloud.
These tips represent what I see in the field working with and coaching teams. Keep in mind the majority of this applies to what I work mostly, that is Snowflake and adopting DataOps style agile patterns.
These aren’t in any order and isn’t intended to be a definitive list, what tips have you got?
1. It’s testing times we live in.
The often-overlooked member of the test family is atomic unit testing. In short, having a distinct test for each assertion made about any logic in the code you’re writing. Understand the basic anatomy of testing, work at it, strive to increase test coverage constantly. Testing is a specialist subject but if you haven’t got access to a specialist tester, you need to be able to write at the very least 1 test for every piece of logic in your code. If you only have 1 test for each piece of logic, you’re not looking hard enough and need to get better!
2. Understand the CI/CD lifecycle and that everything is versioned code.
I hear this far too often to be an anomaly. “That’s a long workflow for a 1-line code change, I could do that straight into production in 2 minutes with no risk”
Any change that isn’t in your main branch means your code base has diverged from your environment, never let this happen. Understand that a production release always goes through a pipeline (yes even a hotfix) and not your keyboard.
Gone are the days of that nervous, sweating and crossing your fingers quarterly deployments. “I don’t think that datatype change will break anything downstream”. Because you’ve got great test coverage from #1 you can ship those features confidently and regularly (like all the time).
3. Your SQL skills are still very relevant.
The core anatomy of SQL is much the same as when I first started learning it in 1991 but you’ll need to slightly re-think how you use and apply that base SQL knowledge, but you’ve got this.
4. Your modelling skills are still very relevant.
This is very important. Listen carefully, the sooner you bring your modelling skills to ‘modern data stack’ the better, be that Dimensional Modelling, Data Vault or some other form of Ensemble Logical Modelling. There is a shortage of professionals operating in this space with good modelling skills, we need you, hurry up already!
5. Develop a cloud mindset.
Understand about walking the line between freshness, performance, and cost.
Cloud compute cost avoidance is a thing, a big, often overlooked thing, for people coming from on-premise data-land. Learn how to fiddle the books. You probably don’t need an automated data pipeline or a potentially costly Materialised View for that monthly report, similarly that operational insight that saves or earns thousands of dollars is probably worth the compute cost of keeping that data fresh.
Thinking about transitioning your years of on-premise skills to the Cloud can be daunting. There are hundreds of things you could learn, but the reality is your skills around coding and modelling transfer. You just need to bring an open mind, be comfortable asking silly questions and a willingness to learn and a desire to adopt new practices.
So what are you waiting for?
Contact us today to see how we can help you