Q: Garbage Collector in Python - explain

**Python GC:** Ref counting + generational for cycles. **Module:** gc.collect(), get_count(). **CPython:** Refcount primary. **Cycles:** Generations. **Why:** Rare tuning. **Production:** Avoid circular refs for large objects.

Question 1

Explain the differences between a Data Lake and a Data Warehouse.

Accepted Answer

**Data Lake**: Low-cost object storage (S3, ADLS) for raw, semi-structured, unstructured data. Schema-on-read; used for exploratory analytics, ML, archival. **Data Warehouse**: Structured, curated storage optimized for SQL; schema-on-write; used for BI and reporting. **Why both exist**: Lakes offer flexibility and cost at scale; warehouses offer query performance and concurrency....

Question 2

Docker - purpose and handling dependencies

Accepted Answer

Architectural logic: Docker containerizes app + dependencies for portability and reproducibility. Why: 'Runs on my machine' → runs everywhere; isolate from host. Dependencies: Baked into image via Dockerfile (FROM, RUN pip install, COPY). Scalability: Same image scales horizontally; registry for distribution. Cost: Smaller images = faster pull, less storage....

Question 3

Libraries for Data Wrangling

Accepted Answer

Python: Pandas (tabular), Polars (faster, lazy), PySpark (distributed). R: dplyr, data.table. ETL: Pandas for <10GB; Polars for single-node performance; PySpark for big data. WHY: Different scale and latency requirements drive choice. SCALABILITY: Pandas hits memory limits; Polars uses Apache Arrow and multithreading; Spark scales horizontally. COST: Polars is free and often 5–10x faster than Pandas—reduces compute time; Spark clusters have fixed cost....

Question 4

Garbage Collector in Python - explain

Accepted Answer

**Python GC:** Ref counting + generational for cycles. **Module:** gc.collect(), get_count(). **CPython:** Refcount primary. **Cycles:** Generations. **Why:** Rare tuning. **Production:** Avoid circular refs for large objects.

Question 5

GeoPandas - definition and features

Accepted Answer

**GeoPandas:** Pandas + geospatial. GeoDataFrame; geometry column. Shapefiles, GeoJSON. Spatial joins, overlays, buffering. Shapely, pyproj. **Use:** Maps; spatial analytics; routing. **Why:** Unified tabular+spatial....

Question 6

Multiprocessing in Python - explain with example

Accepted Answer

**Why Multiprocessing vs Threading:** GIL limits threading to single-core CPU for CPU-bound work. Multiprocessing spawns separate processes—true parallelism, each with own GIL. Use for: CPU-bound (numeric, parsing), NOT for I/O-bound (use asyncio/threading).

**Scalability Trade-offs:** (1) Process spawn overhead: ~50–100ms each—amortize over chunk size. (2) IPC: Queue/Pipe have serialization cost; shared memory (multiprocessing.Value) for large arrays....

Question 7

Stuff Function for XML Usages

Accepted Answer

**Why STUFF + FOR XML PATH:** SQL Server pattern for string aggregation pre-2017. Builds comma-separated lists, XML fragments. Replaced by STRING_AGG in modern SQL.

**Syntax:** STUFF(string, start, length, replace)—replaces chars. Combined with FOR XML PATH(''): concatenates rows into string; STUFF removes leading comma.

**Scalability:** FOR XML PATH scans all rows—O(n). For large aggregations (100K+ rows per group), consider CLR aggregate or application-layer concatenation....

Question 8

Indexing: When to Use and Avoid

Accepted Answer

Use indexes when: high selectivity (many distinct values), frequent filters/joins, read-heavy workload. Avoid when: low cardinality (e.g., gender), small tables (full scan is cheap), write-heavy (index maintenance cost), columns rarely in WHERE. Over-indexing causes slow writes and storage bloat. Rule of thumb: index columns in WHERE, JOIN ON, ORDER BY. Monitor with pg_stat_user_indexes (Postgres) or sys.dm_db_index_usage_stats (SQL Server). Drop unused indexes....

NAB Data Engineer Interview Questions

Reading isn't practice. Get AI feedback on your answers.

Other Companies

NAB Data Engineer Interview Questions

Reading isn't practice. Get AI feedback on your answers.

Other Companies