If you’re considering changing careers to data engineering (DE for short), these tips might prove helpful.
I made a career change from project managing into data and analytics engineering, so this advice is based on real-world experience.
If you’d like to ask more about any of the points below, you’re always welcome to drop me a message!
1. Find out if data engineering is for you
You can come into DE from various backgrounds, but I think there are some things that you kind of need to enjoy in order for it to make sense for you. If you get hyped up from the following topics, you’re likely to also enjoy data engineering:
- Math and logic puzzles
- Coding
- Using AI
- Reading and writing documentation
- Obsessing over small details
- Learning new tech (constantly)
- Spending long hours in IDEs and databases
- Being the unsung hero of the company doing a lot of heavy lifting behind the scenes
- Explaining data concepts and outcomes to business folks (not required, but it’ll make you stand out!)
2. Accept that it’ll take ~6-18 months to switch careers
Transitioning into data engineering is not an overnight task. In my experience, you need ~500-600 hours of studying to acquire the minimal set of employable skills as a junior data engineer. So depending on your pace (10-40 hours per week), this can take anywhere between 4-15 months. Then on top of that is the job hunt, which will take another 2-3 months at least. So all in all, we’re talking about 6-18 months in total.
This is not to say that you shouldn’t do it or that it’s impossible. I only want you to be realistic about the time frame. This journey is totally doable, but it will take time and effort.
3. Save up at least 3-6 months of expenses as buffer
Related to above, you will also most probably:
- need to pay for your DE studies
- earn less money (or no money) while you study
- accept a lower income for some time when you’re employed as a junior DE
So I recommend to have at least a financial buffer of 3-6 months of expenses (or more, depending on your case) saved up before starting the career change journey. It’s a lot easier to breathe when you have some financial leeway while doing your studies.
4. Do a data bootcamp
Unless you have ironclad self-discipline, self-studying will be hard and lonely. University degrees tend to get outdated quickly and take years to complete. Bootcamps will get you up-to-speed with a rocket ship pace, will teach you the most modern tools of the trade and, most importantly, have peers doing the same studies you are.
Yes, bootcamps cost money, but in my opinion they are well worth the investment. If you can get your employer to pay for the bootcamp and pivot to a DE role inside your company while working, even better! If you’re in the Nordics, some good bootcamps include Skillio*, Academy and Salt.
The above is not to say that other routes are impossible. You can definitely get into the industry also by studying on your own or through a conventional university degree. Every approach has its pros and cons. Do your own research and see what’s the best fit for you.
*Full disclosure, I studied here.
5. Learn the core skills first
Data engineering has an endless number of tools and frameworks to learn. But you only need to learn the core skills to get hired in your first role. In my opinion, you need a basic understanding of the following:
- SQL
- Python
- Git
- DevOps and CI/CD pipelines
- ELT pipelines and Data Warehousing
- Data Governance and Cybersecurity
- Basics of Data Modelling (Inmon/Kimball)
- 1 Cloud Platform (choose Azure/AWS/GCP, only DE related aspects such as blob storage/S3 and serverless functions)
- 1 Cloud Data Warehouse (choose Snowflake/BigQuery/Redshift/Synapse Analytics)
- 1 Visualization Tool (choose Power BI/Tableau/Looker, this is optional from DE perspective but IMO really useful to understand, also gives you the option to get a job as a data analyst)
6. Start networking on day 1
This one sounds like a cliche, but it’s really essential. The more you go into events and meet people, the more you increase your luck surface area. You’ll never know which chance encounter leads into a discussion about a job, a company to be founded or a freelancing side project. And also, meeting new people is just rewarding in itself.
The data space is in my experience one of the most welcoming and supporting communities there is! And you’ll find out there are also a lot of people who have come into data from other tangential fields. So you’ll definitely not feel alone in data events even as a newcomer.
I warmly recommend attending local meetups, conferences and user group events. I’ve especially enjoyed dbt Meetups, Helsinki Data Week, DataTribe and Snowflake User Group Events.
Networking online is also very easy these days. Some online communities I’ve personally liked are the dbt Community, r/dataengineering and The Data Freelancer.
7. Create a portfolio of 2-3 projects
If you really want to stand out, create a couple of solid projects on your GitHub profile. But don’t overdo this, usually this is not a requirement from any employer, at least outside the tech space. So 2-3 projects is definitely enough. But if you do it, do it well: I think having a lackluster GitHub-profile is worse than having none at all.
Write a nice story with clean documentation in the readme-files. Better yet, write about your projects and share them on LinkedIn or a personal blog. If these projects come up in interviews, be prepared to explain the thought process behind them.
If you need ideas, Simon Späti’s list of Open-Source Data Engineering Projects is a great place to start.
8. Apply for jobs before you feel ready
This is crucial. You’ll never feel 100% ready before starting. Start applying when 50-70% of the job requirements sound familiar and make some sense to you.
The interview process itself is a massive learning opportunity. You’ll discover what skills employers actually prioritize versus what you thought they wanted. Each rejection teaches you something. Plus, imposter syndrome hits everyone in this field, even experienced engineers. Don’t let it stop you from taking the leap.
9. Succeed in your first DE role
It’s not that hard to be successful in your job. Just take care of the basics and go the extra mile when you have the chance.
Show up on time, take responsibility for the tasks you get, get shit done, be active, be eager to learn and ask questions. Show that you really care.
Every once in a while, when you get the opportunity to really show your employer that you can go the extra mile when requested, do it. For example, this could be a task that no one else wants to do or a project that might seem super complex at first. Take ownership of it, do a good job and I’ll promise you, they will notice.
10. Grow as a generalist or find your niche
Getting your first data engineering job is just the beginning.
Afterwards, you can decide whether you want to develop your knowledge about the whole data engineering lifecycle or go deep into one area: data ingestion, transformation, analytics, AI, real-time pipelines, DevOps, infrastructure…
Set aside time regularly for learning new stuff. Get certified for the most important tools you’re using daily. When you’ve mastered the basics listed in point 5, start looking into the tools of the modern data stack: Airbyte, Fivetran, Airflow, Dagster, dbt, Dataform, Fabric, Databricks… and so forth.