\u00a0<\/p>\n
#\u00a0<\/span>SQL + Python Just Isn’t Enough<\/h2>\n\u00a0
For years, the formula seemed simple: learn SQL + learn Python = get a data job. Especially as mid-sized companies started becoming “data-driven.” Hiring managers were happy they could get anyone who could write a half-decent GROUP BY<\/code> and wrangle a pandas DataFrame without breaking something. You know what PostgreSQL is? Get in, you got the job! This worked for some time. Until it didn’t.<\/p>\nIf you haven’t noticed, the data professional’s job market has undergone a structural shift. Yes, SQL and Python are still important<\/strong>; they’re on every job description. But they’ve been demoted from differentiators to prerequisites<\/strong>.<\/p>\nLikely, you’re still optimizing for the interview questions you practiced three years ago. Forget about it. This article is about the gap between what candidates prepare for and what companies actually need right now.<\/p>\n
\u00a0<\/p>\n
#\u00a0<\/span>What the Job Market Is Actually Asking For<\/h2>\n\u00a0
A January 2026 breakdown<\/a> by Future Proof Data Science of over 700 data scientist job postings found that Python and SQL are still among the top three skills, but machine learning and AI skills are second and fourth<\/strong>.<\/p>\n\u00a0<\/p>\n
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-3.png\"\/)
Image Source: Future Proof Data Science<\/a><\/span><\/center>
\n\u00a0<\/p>\nNot all AI-related postings require hands-on AI expertise, but 1 in 3 does. The most required specific AI skills<\/strong> are:<\/p>\n\n- Large language models (LLMs)\n<\/li>\n
- Retrieval-augmented generation (RAG)\n<\/li>\n
- Prompt engineering\n<\/li>\n
- Vector databases\n<\/li>\n<\/ul>\n
This speaks to an increasing demand for data professionals who can build and deploy AI systems<\/strong>.<\/p>\nKeep in mind that the direction and<\/em> the velocity of this change matter. This reminds me of how machine learning went from a niche requirement in 2012 to a near-universal one by 2020.<\/p>\nThe second story<\/strong> is less visible but arguably more immediate for most candidates<\/strong>: the foundational engineering bar has risen sharply<\/strong>. Data engineering skills \u2014 pipelines, orchestration, cloud platforms, data quality checks \u2014 and machine learning in production \u2014 model monitoring, drift detection, evaluation design \u2014 are now core expectations<\/strong> rather than bonuses in data science job postings.<\/p>\nA glance at any major job board confirms it: along with AI skills, roles titled “Data Scientist” routinely list Snowflake<\/a><\/strong>, dbt<\/a><\/strong>, Airflow<\/a><\/strong>, and ETL pipeline ownership as requirements, not nice-to-haves.<\/p>\nThere are four skills that you are probably missing. These are the new differentiators in the current job market.<\/p>\n
\u00a0
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-4.png\"\/)
\u00a0<\/p>\n
#\u00a0<\/span>Skill #1: Data Modeling<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nData modeling<\/strong> is the ability to design how data should be structured, related, and stored<\/strong>. Think of it as deciding what tables to create, what they represent, and how they relate to each other.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nTooling improvements changed the landscape. Snowflake<\/a><\/strong>, dbt<\/a><\/strong>, and BigQuery<\/a><\/strong> all made it relatively easy for data scientists to own the data transformation layer<\/strong>. In other words, modeling decisions that used to belong to data engineers are now being handed over to data scientists.<\/p>\nGet a data schema wrong, and you’re in dangerous waters. Typically, these errors are not obvious immediately. Once they become obvious, it’s too late. Your machine learning work has already been impacted by feature engineering built on data of the wrong granularity \u2014 a direct consequence of a badly modeled foundation.<\/p>\n
\u00a0
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-5.png\"\/)
\u00a0<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nTake a real dataset you work with and redesign its schema from scratch. Ask yourself these questions:<\/p>\n
\n- What are the entities?\n<\/li>\n
- What do they relate to?\n<\/li>\n
- What grain makes sense?\n<\/li>\n
- What queries will run most frequently?\n<\/li>\n<\/ul>\n
After that, read about dimensional modeling. Kimball’s approach<\/a><\/strong>, detailed in his book The Data Warehouse Toolkit<\/a>, remains a useful reference point.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>Skill #2: Performance Optimization<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nPerformance optimization is understanding why a query runs the way it does and how to make it run faster, cheaper, or at greater scale<\/strong>. You can optimize SQL queries<\/strong>, but also Python pipelines<\/strong> and data workflows<\/strong> in general \u2014 data scientists increasingly own them end-to-end.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nFirst, data volumes have grown<\/strong> to the point where a correct but inefficient query can cost hundreds of dollars and time out in production.<\/p>\nSecond, as mentioned earlier, data scientists now have to own much more of the pipeline<\/strong> than they did before. Your code has to be production-ready, not just runnable in Jupyter notebooks.<\/p>\n\u00a0
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-6.png\"\/)
<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nPick several complex SQL queries you’ve written, run EXPLAIN ANALYZE<\/code> on them, and read what the query planner actually did. Then use that to optimize the query<\/strong>. You’ll likely find at least one index, restructuring, or rewrite that improves each query.<\/p>\nFor a slow Python pipeline<\/strong>, profile it<\/strong>. There are two main tools for time<\/strong>:<\/p>\n\n- cProfile<\/a><\/strong>: Run it with
python -m cProfile -s cumulative your_script.py<\/code> and look at the top of the output to see the functions consuming the most cumulative time.\n<\/li>\n- line_profiler<\/a><\/strong>: Goes deeper by showing execution time line by line within a specific function. Use it once cProfile has told you which<\/em> function is slow and you need to know why<\/em>.\n<\/li>\n<\/ul>\n
For memory<\/strong>, use memory_profiler<\/a><\/strong>.<\/p>\nFind the bottleneck<\/strong> \u2014 is it slow because a Python loop should be vectorized? Is data loaded into memory all at once instead of in chunks? \u2014 fix it, and measure the difference.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>Skill #3: Infrastructure Awareness<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nThis skill means you understand the systems data lives in and moves through<\/strong>. These systems include cloud platforms, distributed compute, data pipelines, storage formats, and cost models<\/strong>.<\/p>\nYou should know enough about the infrastructure to design systems that are deployable into it.<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nAgain, because a good chunk of a data engineer’s job has fallen into a data scientist’s lap. If you’re dependent on data engineers for every infrastructure decision, you’re effectively creating a bottleneck \u2014 and that’s not something hiring managers are looking for.<\/p>\n
Infrastructure awareness includes these main interconnected areas.<\/p>\n
\u00a0
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-7.png\"\/)
\u00a0<\/p>\n
You’ll most likely have to familiarize yourself with these tools.<\/p>\n
\u00a0
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-8.png\"\/)
<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nArrange a session with your data engineering team. Sit with them and ask them to walk you through a pipeline end-to-end<\/strong>. Understand where data lives, how it’s partitioned, and what happens when something breaks<\/strong>.<\/p>\nThen step up by building a small pipeline<\/strong> yourself: use a free cloud tier, understand the cost and execution metrics, then deliberately break the pipeline to understand how it fails.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>Skill #4: Designing RAG Systems, Evaluating LLM Outputs, and Running AI Experiments<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nThis cluster of skills relates to practical AI work<\/strong>. You have to know how to design retrieval-augmented generation (RAG) systems (connecting LLMs to real data sources), build evaluation frameworks (measuring whether an LLM-powered feature is actually working), and run experiments on AI features.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nAI tools are the reason. They made it possible to build a RAG pipeline without extensive research knowledge. Frameworks like LangChain<\/a><\/strong> and LlamaIndex<\/a><\/strong>, combined with cloud-native vector databases, lowered the barrier significantly.<\/p>\nSo the question is no longer whether it can be built \u2014 yes, it can be. But can it be built well, evaluated, and trusted in production? Answering that question is what you must be able to do: define metrics, design experiments, and measure outcomes.<\/p>\n
\u00a0
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-9.png\"\/)
\u00a0<\/p>\n
In applying these skills, you will use these tools.<\/p>\n
\u00a0
![\"Hidden](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Rosidi_Hidden-Skill-Gap-10.png\"\/)
<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nFind some interview questions<\/strong> to help you refine your AI thinking. Here are some examples from AI Product & GenAI interview questions<\/a><\/strong> on StrataScratch.<\/p>\nExample #1: Measuring AI Feature Rollout in Retail Stores<\/a><\/p>\n\n\nHow would you measure the impact of an AI-powered inventory recommendation system being rolled out to a sample of retail stores? How would you design the experiment and account for store-level variation?\n<\/p>\n<\/blockquote>\n
\u00a0<\/p>\n
Example #2: RAG System Architecture<\/a><\/p>\n\n\nDescribe how you would architect a RAG system from scratch. What components are needed, and how would you optimize retrieval quality?\n<\/p>\n<\/blockquote>\n
\u00a0<\/p>\n
After you’ve made your thinking clear, build a small RAG application<\/strong>: choose a domain, embed a document corpus, wire up retrieval, and evaluate the outputs using a structured metric.<\/p>\nAlso, design an experiment: write out a hypothesis, define the metrics, and think through a valid test to evaluate it.<\/p>\n
\u00a0<\/p>\n
#\u00a0<\/span>Conclusion<\/h2>\n\u00a0
The four skills \u2014 data modeling, performance optimization, infrastructure awareness, and practical AI skills \u2014 are what comprise the gap between you and the job market. Hopefully you won’t fall into it. To ensure you don’t, this article has included practical advice on how to acquire each one.
\u00a0
\u00a0<\/p>\n
Nate Rosidi<\/a><\/strong><\/b><\/a> is a data scientist and in product strategy. He’s also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Nate writes on the latest trends in the career market, gives interview advice, shares data science projects, and covers everything SQL.<\/p>\n<\/p><\/div>\n
For years, the formula seemed simple: learn SQL + learn Python = get a data job. Especially as mid-sized companies started becoming “data-driven.” Hiring managers were happy they could get anyone who could write a half-decent
GROUP BY<\/code> and wrangle a pandas DataFrame without breaking something. You know what PostgreSQL is? Get in, you got the job! This worked for some time. Until it didn’t.<\/p>\nIf you haven’t noticed, the data professional’s job market has undergone a structural shift. Yes, SQL and Python are still important<\/strong>; they’re on every job description. But they’ve been demoted from differentiators to prerequisites<\/strong>.<\/p>\nLikely, you’re still optimizing for the interview questions you practiced three years ago. Forget about it. This article is about the gap between what candidates prepare for and what companies actually need right now.<\/p>\n
\u00a0<\/p>\n
#\u00a0<\/span>What the Job Market Is Actually Asking For<\/h2>\n\u00a0
A January 2026 breakdown<\/a> by Future Proof Data Science of over 700 data scientist job postings found that Python and SQL are still among the top three skills, but machine learning and AI skills are second and fourth<\/strong>.<\/p>\n\u00a0<\/p>\n
Image Source: Future Proof Data Science<\/a><\/span><\/center>
\n\u00a0<\/p>\nNot all AI-related postings require hands-on AI expertise, but 1 in 3 does. The most required specific AI skills<\/strong> are:<\/p>\n\n- Large language models (LLMs)\n<\/li>\n
- Retrieval-augmented generation (RAG)\n<\/li>\n
- Prompt engineering\n<\/li>\n
- Vector databases\n<\/li>\n<\/ul>\n
This speaks to an increasing demand for data professionals who can build and deploy AI systems<\/strong>.<\/p>\nKeep in mind that the direction and<\/em> the velocity of this change matter. This reminds me of how machine learning went from a niche requirement in 2012 to a near-universal one by 2020.<\/p>\nThe second story<\/strong> is less visible but arguably more immediate for most candidates<\/strong>: the foundational engineering bar has risen sharply<\/strong>. Data engineering skills \u2014 pipelines, orchestration, cloud platforms, data quality checks \u2014 and machine learning in production \u2014 model monitoring, drift detection, evaluation design \u2014 are now core expectations<\/strong> rather than bonuses in data science job postings.<\/p>\nA glance at any major job board confirms it: along with AI skills, roles titled “Data Scientist” routinely list Snowflake<\/a><\/strong>, dbt<\/a><\/strong>, Airflow<\/a><\/strong>, and ETL pipeline ownership as requirements, not nice-to-haves.<\/p>\nThere are four skills that you are probably missing. These are the new differentiators in the current job market.<\/p>\n
\u00a0
\u00a0<\/p>\n
#\u00a0<\/span>Skill #1: Data Modeling<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nData modeling<\/strong> is the ability to design how data should be structured, related, and stored<\/strong>. Think of it as deciding what tables to create, what they represent, and how they relate to each other.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nTooling improvements changed the landscape. Snowflake<\/a><\/strong>, dbt<\/a><\/strong>, and BigQuery<\/a><\/strong> all made it relatively easy for data scientists to own the data transformation layer<\/strong>. In other words, modeling decisions that used to belong to data engineers are now being handed over to data scientists.<\/p>\nGet a data schema wrong, and you’re in dangerous waters. Typically, these errors are not obvious immediately. Once they become obvious, it’s too late. Your machine learning work has already been impacted by feature engineering built on data of the wrong granularity \u2014 a direct consequence of a badly modeled foundation.<\/p>\n
\u00a0
\u00a0<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nTake a real dataset you work with and redesign its schema from scratch. Ask yourself these questions:<\/p>\n
\n- What are the entities?\n<\/li>\n
- What do they relate to?\n<\/li>\n
- What grain makes sense?\n<\/li>\n
- What queries will run most frequently?\n<\/li>\n<\/ul>\n
After that, read about dimensional modeling. Kimball’s approach<\/a><\/strong>, detailed in his book The Data Warehouse Toolkit<\/a>, remains a useful reference point.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>Skill #2: Performance Optimization<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nPerformance optimization is understanding why a query runs the way it does and how to make it run faster, cheaper, or at greater scale<\/strong>. You can optimize SQL queries<\/strong>, but also Python pipelines<\/strong> and data workflows<\/strong> in general \u2014 data scientists increasingly own them end-to-end.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nFirst, data volumes have grown<\/strong> to the point where a correct but inefficient query can cost hundreds of dollars and time out in production.<\/p>\nSecond, as mentioned earlier, data scientists now have to own much more of the pipeline<\/strong> than they did before. Your code has to be production-ready, not just runnable in Jupyter notebooks.<\/p>\n\u00a0
<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nPick several complex SQL queries you’ve written, run EXPLAIN ANALYZE<\/code> on them, and read what the query planner actually did. Then use that to optimize the query<\/strong>. You’ll likely find at least one index, restructuring, or rewrite that improves each query.<\/p>\nFor a slow Python pipeline<\/strong>, profile it<\/strong>. There are two main tools for time<\/strong>:<\/p>\n\n- cProfile<\/a><\/strong>: Run it with
python -m cProfile -s cumulative your_script.py<\/code> and look at the top of the output to see the functions consuming the most cumulative time.\n<\/li>\n- line_profiler<\/a><\/strong>: Goes deeper by showing execution time line by line within a specific function. Use it once cProfile has told you which<\/em> function is slow and you need to know why<\/em>.\n<\/li>\n<\/ul>\n
For memory<\/strong>, use memory_profiler<\/a><\/strong>.<\/p>\nFind the bottleneck<\/strong> \u2014 is it slow because a Python loop should be vectorized? Is data loaded into memory all at once instead of in chunks? \u2014 fix it, and measure the difference.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>Skill #3: Infrastructure Awareness<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nThis skill means you understand the systems data lives in and moves through<\/strong>. These systems include cloud platforms, distributed compute, data pipelines, storage formats, and cost models<\/strong>.<\/p>\nYou should know enough about the infrastructure to design systems that are deployable into it.<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nAgain, because a good chunk of a data engineer’s job has fallen into a data scientist’s lap. If you’re dependent on data engineers for every infrastructure decision, you’re effectively creating a bottleneck \u2014 and that’s not something hiring managers are looking for.<\/p>\n
Infrastructure awareness includes these main interconnected areas.<\/p>\n
\u00a0
\u00a0<\/p>\n
You’ll most likely have to familiarize yourself with these tools.<\/p>\n
\u00a0
<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nArrange a session with your data engineering team. Sit with them and ask them to walk you through a pipeline end-to-end<\/strong>. Understand where data lives, how it’s partitioned, and what happens when something breaks<\/strong>.<\/p>\nThen step up by building a small pipeline<\/strong> yourself: use a free cloud tier, understand the cost and execution metrics, then deliberately break the pipeline to understand how it fails.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>Skill #4: Designing RAG Systems, Evaluating LLM Outputs, and Running AI Experiments<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>What It Is<\/h4>\nThis cluster of skills relates to practical AI work<\/strong>. You have to know how to design retrieval-augmented generation (RAG) systems (connecting LLMs to real data sources), build evaluation frameworks (measuring whether an LLM-powered feature is actually working), and run experiments on AI features.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Why It Became a Differentiator<\/h4>\nAI tools are the reason. They made it possible to build a RAG pipeline without extensive research knowledge. Frameworks like LangChain<\/a><\/strong> and LlamaIndex<\/a><\/strong>, combined with cloud-native vector databases, lowered the barrier significantly.<\/p>\nSo the question is no longer whether it can be built \u2014 yes, it can be. But can it be built well, evaluated, and trusted in production? Answering that question is what you must be able to do: define metrics, design experiments, and measure outcomes.<\/p>\n
\u00a0
\u00a0<\/p>\n
In applying these skills, you will use these tools.<\/p>\n
\u00a0
<\/p>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>How to Acquire It<\/h4>\nFind some interview questions<\/strong> to help you refine your AI thinking. Here are some examples from AI Product & GenAI interview questions<\/a><\/strong> on StrataScratch.<\/p>\nExample #1: Measuring AI Feature Rollout in Retail Stores<\/a><\/p>\n\n\nHow would you measure the impact of an AI-powered inventory recommendation system being rolled out to a sample of retail stores? How would you design the experiment and account for store-level variation?\n<\/p>\n<\/blockquote>\n
\u00a0<\/p>\n
Example #2: RAG System Architecture<\/a><\/p>\n\n\nDescribe how you would architect a RAG system from scratch. What components are needed, and how would you optimize retrieval quality?\n<\/p>\n<\/blockquote>\n
\u00a0<\/p>\n
After you’ve made your thinking clear, build a small RAG application<\/strong>: choose a domain, embed a document corpus, wire up retrieval, and evaluate the outputs using a structured metric.<\/p>\nAlso, design an experiment: write out a hypothesis, define the metrics, and think through a valid test to evaluate it.<\/p>\n
\u00a0<\/p>\n
#\u00a0<\/span>Conclusion<\/h2>\n\u00a0
The four skills \u2014 data modeling, performance optimization, infrastructure awareness, and practical AI skills \u2014 are what comprise the gap between you and the job market. Hopefully you won’t fall into it. To ensure you don’t, this article has included practical advice on how to acquire each one.
\u00a0
\u00a0<\/p>\n
Nate Rosidi<\/a><\/strong><\/b><\/a> is a data scientist and in product strategy. He’s also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Nate writes on the latest trends in the career market, gives interview advice, shares data science projects, and covers everything SQL.<\/p>\n<\/p><\/div>\n