A branching strategy is something a software development team uses when interacting with a version control system for writing and managing code. As the name suggests, the branching strategy focuses on how branches are used in the development process. Factors like type of project, size of team you are working with, the maturity of the team, the version control system, scale or CI/CD can influence your branching strategy selection. Unfortunately there is no one size fits all.

Two common and well known branching strategies include Git Flow and Trunk Based Development. At Data Engineers, we are able to help determine which strategy is best for you so that it:

• provides a clear path for the development process from initial changes to production
• allows users to create workflows that lead to structured releases
• enables parallel development
• optimizes developer workflow without adding any overhead
• enables Peer reviewing of changes
• enables faster release cycles
• efficiently integrates with all DataOps and DevOps practices and tools such as different version control systems

Infrastructure as code is the process of managing and provisioning resources through code rather than physical hardware configuration or interactive configuration tools. When working in a cloud environment, we believe that developers should have the ability to create resource which can then be used to be deployed to different environments. Utilizing IaC allows the treatment of infrastructure as cattle rather than a pet. Providing the capability to go from zero to everything quickly and consistently with the knowledge of what you deployed last time will be the same as what you are deploying this time. At Data Engineers, we are able to help determine which Infrastructure as Code strategy is best for you so that it:

• easily deployable throughout the development process from initial changes to production
• allows users to create additional resources that lead to consistent deployments
• enables faster release cycles
• enables Peer reviewing of changes
• efficiently integrates with DevOps practices and tools such as different version control systems

DataOps and DevOps are complementary approaches to creating a more responsive business — not interchangeable terms. Together, these concepts build and streamline the development and data pipelines to deliver valuable software and insights to the end user faster. At Data Engineers, we are able to help determine which what you need to setup to enable continuous integration and continues delivery based on your tools sets so that it:

• enables continuous integration on code check-in or pull-requests
• enables continuous deployment on merging of code to production branches
• enables faster release cycles
• integration with tools such as different version control systems

dbt, short for data build tool, is an open-source project for managing data transformations in a data warehouse. Once data is loaded into a data platform, dbt enables us to write the models (we’d typically call them views) that transform the data to something consumable for downstream consumption, i.e. our API’s or people with access to the data platform. It also comes with built in testing and documentation generation tools so we can have a high level of confidence in the models we’re generating and analyzing.

At Data Engineers, we have developed our DataOps framework around this tool so that we are able to:

• bring Software Engineering practice to the Data Engineering
• enable improved data governance capabilities
• enable increase in speed to delivery
• enable product like development

schemachange is a simple python based tool to manage all of your Snowflake objects. It follows an Imperative-style approach to Database Change Management (DCM) and was inspired by the Flyway database migration tool. When combined with a version control system and a CI/CD tool, database changes can be approved and deployed through a pipeline using modern software delivery practices. As such, schemachange plays a critical role in enabling Database (or Data) DevOps.

At Data Engineers, we have incorporated this into our DataOps framework around this tool so that it:

• allows easily deployable components throughout the development process from initial changes to production
• allows users to deploy changes which can't be managed by dbt

BimlFlex, a flexible, extensible, and completely customizable data warehouse framework with a collection of templates, metadata definitions, and tooling that enables you to build an end-to-end data solution without ever writing a single line of code. You can pick and choose which of the following capabilities you would like to implement. Just one, a subset, or all of it - you're in control. At Data Engineers, we have a BimlHero who has co-authored the 'The Biml Book' so you couldn't be in better hands.

At Data Engineers, we make extensive use of Terraform as our provider to deploy our cloud based resources and configuration. It lets you define resources and infrastructure in human-readable, declarative configuration files, and manages your infrastructure's lifecycle. Through the use of Terraform Cloud, we are able to build up modular code bases which can be used across your platform of choice (Azure or AWS) so you are able to deploy both resources and its configuration out.