Real questions on the AWS data stack — S3, Glue, Redshift, EMR, Kinesis, Athena, and Lambda. Sorted by how often they appear in interviews.
AWS powers a huge share of production data platforms, and data engineer interviews test it heavily. These questions cover S3-based data lakes, Glue ETL and the Data Catalog, Redshift warehousing, EMR for Spark at scale, Kinesis streaming, Athena serverless SQL, and Lambda event processing. Each question includes a detailed answer and the companies that have asked it.
This collection contains 26 curated questions: 7 easy, 5 medium, and 14 hard. The distribution skews toward harder problems, reflecting the depth expected in senior-level interviews.
The most frequently tested areas in this set are spark (14), partition (12), optimization (7), join (6), etl (6), and bigquery (3). Focusing on these topics will give you the highest return on your preparation time.
Start with the easy questions to warm up and solidify fundamentals. Medium-difficulty questions form the bulk of real interviews — spend the most time here and practice explaining your reasoning out loud. Hard questions often appear in senior and staff-level rounds; attempt them after you're comfortable with the basics. For each question, try answering before revealing the solution. Use our AI Mock Interview to simulate real interview conditions and get instant feedback on your responses.
CDC During Migration - explain approaches for real-time Change Data Capture
Architecturally, how would you justify or challenge Hadoop vs. a cloud-native data lake (S3 + EMR/Databricks) for a greenfield enterprise data platform? Discuss scalability ceilings, cost model trade-offs, and operational complexity.
Design a fault-tolerant Spark Streaming checkpoint strategy: what to persist, recovery semantics, and cost/scalability trade-offs with checkpoint frequency.
How would you read data from a web API using PySpark?
How would you read data from a web API? What steps would you follow after reading the data?
What are the key components of AWS Glue, and how do they work together?
What is the difference between Managed and External Tables in Databricks?
What is the difference between S3 and HDFS?
What is the role of AWS Lambda in a data engineering pipeline?
What is the small-file problem in Spark, and how do you solve it?
Articulate the architectural decisions, scalability trade-offs, and cost implications of designing an AWS data platform. How would you justify glue vs. EMR, Redshift vs. Athena, and when would each choice become cost-prohibitive at scale?
Briefly introduce yourself and walk us through your journey as a Data Engineer so far.
Can Presto work with Near Real-Time Data (Streaming Data Source)?
Compare Glue partition discovery with Hive MSCK/ADD PARTITION. Explain the operational and cost implications of crawler-based vs. partition-projection approaches. When does partition projection become necessary, and what are its limitations?
Compare Redshift, BigQuery, and Snowflake in terms of cost, performance, and scalability.
Connecting BigQuery with Linux
Describe a scenario where AWS Data Pipeline is preferred over Glue. Why?
Describe a time when you had to work with a team to solve a complex problem.
Describe a time you had to make a difficult decision with limited information.
Describe an AWS EC2 instance and how IAM roles/policies enhance security.
Describe handling schema evolution in AWS Redshift without downtime.
Describe your role in a team project.
Design a data pipeline from end to end - describe how data would be ingested, processed, stored, and queried.
Design a data pipeline to ingest and process data from multiple sources (e.g., S3, Kinesis) to Redshift using Spark.
Design an end-to-end data pipeline using Glue, Lambda, EC2, S3, Redshift, and Athena.
Design: Migrate data from multiple sources (Hadoop, S3, Oracle DB) into a final S3 bucket
The core set is S3 (data lake storage), AWS Glue (serverless ETL + Data Catalog), Redshift (data warehouse), EMR (managed Spark/Hadoop), Kinesis (streaming), Athena (serverless SQL over S3), and Lambda (event-driven compute). Interviewers probe when to use each, how they fit into a lakehouse, and the cost/performance trade-offs between them.
Redshift is a provisioned (or serverless) columnar data warehouse for fast, repeated analytical queries on structured data you load into it. Athena is serverless and queries data in place on S3 using Presto/Trino, billed per data scanned — ideal for ad-hoc queries on a data lake with no cluster to manage. Use Redshift for high-concurrency BI dashboards; use Athena for exploratory or infrequent queries on raw S3 data.
AWS Glue is a serverless ETL service built on Spark, plus a Data Catalog (a Hive-compatible metastore) and crawlers that infer schemas from S3. Use it to transform and catalog data without managing servers — Glue jobs read from S3/JDBC sources, run PySpark transformations, and write to S3, Redshift, or other targets. The Data Catalog then makes that data queryable from Athena, Redshift Spectrum, and EMR.
S3 is the durable, cheap storage foundation of an AWS data lake. A typical design uses layered buckets/prefixes — raw (landing), staging (cleaned), and curated (analytics-ready) — with data stored in columnar formats like Parquet, partitioned by date. Glue catalogs it, Athena/Redshift Spectrum query it, and EMR/Glue transform it. S3's separation of storage from compute is what makes the lakehouse model cost-effective.
Kinesis Data Streams is for high-throughput, ordered, replayable event streams consumed by multiple readers (analytics, real-time dashboards). SQS is a simple queue for decoupling services with at-least-once delivery and no replay — good for task/work queues, not analytics streams. Amazon MSK is managed Apache Kafka — pick it when you need Kafka's ecosystem, exactly-once semantics, or portability. For most AWS-native streaming pipelines, Kinesis → Lambda/Firehose → S3/Redshift is the default.
The Data Engineering Interview Answer Vault bundles 750+ reviewed answers into 7 focused PDF volumes — SQL, Spark, Python, System Design, Cloud, Behavioral, and Data Modeling. Study on any device, no subscription required.
800+ hands-on courses — Grokking System Design, Coding Patterns, and AI mock interviews for your DE loop.
Turn any topic or your own notes into an interactive, personalized course in 60 seconds.
The book that gets data engineers through system-design rounds. Essential reading.
Some links below are affiliate links. If you buy through them we may earn a small commission at no extra cost to you — it helps keep DataEngPrep free.
Reading answers is step one. Get instant AI feedback on your answers, run mock interviews, and track readiness — built specifically for data engineering interviews.