Essential Python questions asked in data engineering interviews at top companies. From core language concepts to data processing patterns.
Python is the lingua franca of data engineering. Interviewers test everything from core language mechanics to practical ETL scripting. These questions cover data structures, list comprehensions, generators, decorators, context managers, Pandas transformations, PySpark UDFs, error handling patterns, and real-world coding challenges you'll face in data pipeline development.
This collection contains 40 curated questions: 30 easy, 4 medium, and 6 hard. There's a strong foundation of fundamentals-focused questions — ideal for building confidence before tackling advanced topics.
The most frequently tested areas in this set are python (24), join (6), spark (5), optimization (4), partition (4), and etl (1). Focusing on these topics will give you the highest return on your preparation time.
Start with the easy questions to warm up and solidify fundamentals. Medium-difficulty questions form the bulk of real interviews — spend the most time here and practice explaining your reasoning out loud. Hard questions often appear in senior and staff-level rounds; attempt them after you're comfortable with the basics. For each question, try answering before revealing the solution. Use our AI Mock Interview to simulate real interview conditions and get instant feedback on your responses.
What are traits in Scala, and how are they different from classes?
Explain the difference between args and kwargs in Python.
Explain the difference between shallow copy and deep copy in Python.
What are decorators in Python, and how do they work?
What is the difference between a list and a tuple in Python?
Write a Python function to check if a string is a palindrome.
Write a Python function to find the first non-repeating character in a string.
Explain the difference between a list and a tuple in Python.
Explain the difference between args and kwargs in Python.
How do you handle exceptions in Python? Provide an example.
How do you handle memory management in Python?
What are decorators in Python, and how do they work?
What is the difference between a generator and a list in Python?
What is the difference between a set and a list in Python?
What is the difference between shallow copy and deep copy in Python?
Write a Python function to check if a string is a palindrome.
Write a Python function to find the first non-repeating character in a string.
Write a Python function to find the first non-repeating character in a string.
Write a Python function to find the maximum value in a list without using the built-in max() function.
Can you give an example of processing nested JSON data using these functions?
Coin Change Problem - minimum number of coins required to make change
Concatenating lists within a range using list comprehensions
Convert a Binary Search Tree (BST) into a skewed tree in either increasing or decreasing order
Convert a sorted array into a Binary Search Tree
Convert the list [1, [2, 3], 4, 5, 6, [7, 8, 9]] to a single list [1, 2, 3, 4, 5, 6, 7, 8, 9].
Count occurrences of elements in a list of tuples using Spark RDDs
Count of Alphabets in String
Create a dictionary with list elements as keys and their occurrences as values.
Create a function to detect anomalies in sales trends using Pandas and NumPy.
Create a Python program to demonstrate the use of set operations (union, intersection).
Create a script to parse and transform a JSON file into a structured CSV.
Describe script implementation and deployment.
Describe Spark's memory management model. How do you handle heap memory overhead issues?
Design a solution to generate unique device names from a list of IoT devices.
Design an algorithm to merge k sorted lists of video streaming data.
Detect a loop in a singly linked list
Develop a Python script to clean data by removing duplicates and handling missing values.
Difference between Stack vs Queue
Differences between Stack, Queue, and Linked List
Discuss the tech stacks and responsibilities at Morgan Stanley
Core data structures and their complexity (dict/set lookups, list operations), generators and iterators for memory-efficient processing, file and JSON/CSV parsing, Pandas transformations, PySpark UDFs, decorators and context managers, error handling and retries, and writing clean ETL scripts. Interviewers care less about clever algorithms and more about whether you can write readable, correct data-processing code.
A list is ordered and mutable; a tuple is ordered and immutable (so it can be a dict key and is slightly faster); a set is an unordered collection of unique elements with O(1) membership tests. In data engineering you use sets to deduplicate and to do fast 'is this key seen?' checks, tuples for fixed records, and lists for ordered, changeable sequences.
A generator produces values lazily one at a time using yield, instead of building the whole list in memory. This lets you stream through a multi-gigabyte file or API result with near-constant memory. For example, reading a file line by line with a generator processes billions of rows without ever loading them all — the standard pattern for memory-safe ETL in Python.
A decorator wraps a function to add behavior without changing its body. In pipelines they're used for retry-with-backoff on flaky network calls, timing and logging task duration, caching expensive lookups, and registering tasks (Airflow's @task, dbt-style hooks). Knowing how to write one with functools.wraps shows you understand Python's first-class functions.
Stream it instead of loading it whole: iterate line by line or read in chunks (e.g., pandas.read_csv with chunksize, or a generator). Process and write each chunk, keeping only aggregates in memory. For columnar data, use formats like Parquet that support predicate pushdown and column pruning, or push the work to a distributed engine like Spark when a single machine can't keep up.
The Data Engineering Interview Answer Vault bundles 750+ reviewed answers into 7 focused PDF volumes — SQL, Spark, Python, System Design, Cloud, Behavioral, and Data Modeling. Study on any device, no subscription required.
800+ hands-on courses — Grokking System Design, Coding Patterns, and AI mock interviews for your DE loop.
Turn any topic or your own notes into an interactive, personalized course in 60 seconds.
The book that gets data engineers through system-design rounds. Essential reading.
Some links below are affiliate links. If you buy through them we may earn a small commission at no extra cost to you — it helps keep DataEngPrep free.
Reading answers is step one. Get instant AI feedback on your answers, run mock interviews, and track readiness — built specifically for data engineering interviews.