Real questions on the AWS data stack — S3, Glue, Redshift, EMR, Kinesis, Athena, and Lambda. Sorted by how often they appear in interviews.
AWS powers a huge share of production data platforms, and data engineer interviews test it heavily. These questions cover S3-based data lakes, Glue ETL and the Data Catalog, Redshift warehousing, EMR for Spark at scale, Kinesis streaming, Athena serverless SQL, and Lambda event processing. Each question includes a detailed answer and the companies that have asked it.
This collection contains 50 curated questions: 23 easy, 7 medium, and 20 hard. There's a strong foundation of fundamentals-focused questions — ideal for building confidence before tackling advanced topics.
The most frequently tested areas in this set are spark (26), partition (16), etl (12), optimization (7), join (5), and bigquery (5). Focusing on these topics will give you the highest return on your preparation time.
Start with the easy questions to warm up and solidify fundamentals. Medium-difficulty questions form the bulk of real interviews — spend the most time here and practice explaining your reasoning out loud. Hard questions often appear in senior and staff-level rounds; attempt them after you're comfortable with the basics. For each question, try answering before revealing the solution. Use our AI Mock Interview to simulate real interview conditions and get instant feedback on your responses.
CDC During Migration - explain approaches for real-time Change Data Capture
Architecturally, how would you justify or challenge Hadoop vs. a cloud-native data lake (S3 + EMR/Databricks) for a greenfield enterprise data platform? Discuss scalability ceilings, cost model trade-offs, and operational complexity.
Design a fault-tolerant Spark Streaming checkpoint strategy: what to persist, recovery semantics, and cost/scalability trade-offs with checkpoint frequency.
How would you read data from a web API using PySpark?
How would you read data from a web API? What steps would you follow after reading the data?
What are the key components of AWS Glue, and how do they work together?
What is the difference between Managed and External Tables in Databricks?
What is the difference between S3 and HDFS?
What is the role of AWS Lambda in a data engineering pipeline?
What is the small-file problem in Spark, and how do you solve it?
Amazon Deequ usage and what sort of quality checks are done using it?
Articulate the architectural decisions, scalability trade-offs, and cost implications of designing an AWS data platform. How would you justify glue vs. EMR, Redshift vs. Athena, and when would each choice become cost-prohibitive at scale?
Basic Spark commands – Create RDD, Load data, Filter
Briefly introduce yourself and walk us through your journey as a Data Engineer so far.
Building ETL pipelines to capture changes when new records are inserted into source tables?
Can Presto work with Near Real-Time Data (Streaming Data Source)?
Can you elaborate on your Big Data project experience?
Can you explain how streams and tasks handle data freshness in near real-time?
Can you explain your experience with Jenkins in your project?
Closure Function - explain
Compare Glue partition discovery with Hive MSCK/ADD PARTITION. Explain the operational and cost implications of crawler-based vs. partition-projection approaches. When does partition projection become necessary, and what are its limitations?
Compare ORC and Parquet
Compare Redshift, BigQuery, and Snowflake in terms of cost, performance, and scalability.
Connecting BigQuery with Linux
Core services of AWS used in data engineering?
Count occurrences of a specific word in a file
Daily Data Volume - quantify
Database vs Data Warehouse vs Data Mart vs Data Lake
Databricks notebooks vs. Fabric notebooks - differences
Databricks vs. PySpark?
Describe a custom EMR cluster configuration for Spark-based ETL with minimal cost.
Describe a real-world use case for using Step Functions with Lambda in a data workflow.
Describe a recent project where you used AWS services extensively. What was your role, and what challenges did you face?
Describe a scenario where AWS Data Pipeline is preferred over Glue. Why?
Describe a time when you had to work with a team to solve a complex problem.
Describe a time you had to make a difficult decision with limited information.
Describe Amazon Athena and how it interacts with S3.
Describe an AWS EC2 instance and how IAM roles/policies enhance security.
Describe AWS Glue components and their functions.
Describe handling schema evolution in AWS Redshift without downtime.
Describe how Adidas could use S3 and Athena to analyze clickstream data.
Describe how you would use AWS Glue to schedule and manage Spark jobs.
Describe step scaling policies vs. target tracking policies in AWS Auto Scaling.
Describe the process for migrating data from an on-premises SQL database to AWS. What services and strategies would you use?
Describe using Step Functions to handle retries and error notifications.
Describe your experience with cloud platforms like AWS, Azure, or GCP
Describe your role in a team project.
Design a data pipeline from end to end - describe how data would be ingested, processed, stored, and queried.
Design a data pipeline to ingest and process data from multiple sources (e.g., S3, Kinesis) to Redshift using Spark.
Design an end-to-end data pipeline using Glue, Lambda, EC2, S3, Redshift, and Athena.
The core set is S3 (data lake storage), AWS Glue (serverless ETL + Data Catalog), Redshift (data warehouse), EMR (managed Spark/Hadoop), Kinesis (streaming), Athena (serverless SQL over S3), and Lambda (event-driven compute). Interviewers probe when to use each, how they fit into a lakehouse, and the cost/performance trade-offs between them.
Redshift is a provisioned (or serverless) columnar data warehouse for fast, repeated analytical queries on structured data you load into it. Athena is serverless and queries data in place on S3 using Presto/Trino, billed per data scanned — ideal for ad-hoc queries on a data lake with no cluster to manage. Use Redshift for high-concurrency BI dashboards; use Athena for exploratory or infrequent queries on raw S3 data.
AWS Glue is a serverless ETL service built on Spark, plus a Data Catalog (a Hive-compatible metastore) and crawlers that infer schemas from S3. Use it to transform and catalog data without managing servers — Glue jobs read from S3/JDBC sources, run PySpark transformations, and write to S3, Redshift, or other targets. The Data Catalog then makes that data queryable from Athena, Redshift Spectrum, and EMR.
S3 is the durable, cheap storage foundation of an AWS data lake. A typical design uses layered buckets/prefixes — raw (landing), staging (cleaned), and curated (analytics-ready) — with data stored in columnar formats like Parquet, partitioned by date. Glue catalogs it, Athena/Redshift Spectrum query it, and EMR/Glue transform it. S3's separation of storage from compute is what makes the lakehouse model cost-effective.
Kinesis Data Streams is for high-throughput, ordered, replayable event streams consumed by multiple readers (analytics, real-time dashboards). SQS is a simple queue for decoupling services with at-least-once delivery and no replay — good for task/work queues, not analytics streams. Amazon MSK is managed Apache Kafka — pick it when you need Kafka's ecosystem, exactly-once semantics, or portability. For most AWS-native streaming pipelines, Kinesis → Lambda/Firehose → S3/Redshift is the default.
Hands-on SQL, Python, and pipeline courses — build the skills interviewers test.
A structured, resume-boosting certificate covering the full DE stack.
The book that gets data engineers through system-design rounds. Essential reading.
Some links below are affiliate links. If you buy through them we may earn a small commission at no extra cost to you — it helps keep DataEngPrep free.
Reading answers is step one. Get instant AI feedback on your answers, run mock interviews, and track readiness — built specifically for data engineering interviews.