Question 1

Design: Migrate data from multiple sources (Hadoop, S3, Oracle DB) into a final S3 bucket

Accepted Answer

**Section 1 — The Context (The 'Why')**
The primary challenge for this design in Cloud/Tools is balancing scale, cost, and reliability. At scale, naive approaches fail: single points of failure cause cascades, schema evolution breaks consumers, and over-provisioning explodes cost. Failure modes include silent data loss from non-idempotent writes, cascading job failures from tight coupling, and operational burden from manual intervention....

Question 2

Why specific cloud services (AWS Glue, EMR) were chosen for scalability and cost-effectiveness

Accepted Answer

**Glue**: Serverless Spark; no cluster management; Data Catalog integration; job bookmarks. Suits standard batch ETL, moderate Spark. Cost-effective for variable/infrequent jobs. 15-min DPU minimum—inefficient for very short jobs. **EMR**: Full Spark control; custom configs; MLlib, graph. Use when Glue doesn't fit—long-running, custom libs. **Scalability**: Both auto-scale; Glue per-job; EMR per-cluster. **Cost**: Glue for sporadic; EMR for steady or custom....

Question 3

Rainwater Trapping Problem - solve with two-pointer technique

Accepted Answer

Water at i = min(left_max, right_max) - height[i]. Two pointers: maintain left_max, right_max; move from side with smaller max. Python: l,r=0,len(h)-1; lm=rm=w=0; while l<r: if h[l]<h[r]: lm=max(lm,h[l]); w+=lm-h[l]; l+=1; else: rm=max(rm,h[r]); w+=rm-h[r]; r-=1; return w. O(n) time, O(1) space....

Question 4

Resolving service downtimes

Accepted Answer

(1) TRIAGE—Confirm scope, check dashboards, recent deploys. (2) MITIGATE—Rollback, scale up, disable non-critical features. (3) INVESTIGATE—Logs (CloudWatch, ELK), metrics, dependencies. (4) FIX—Patch, config, hotfix. (5) COMMUNICATE—Status page, stakeholders. (6) POST-MORTEM—Timeline, root cause, action items. WHY: Minimize MTTR; learn to prevent....

Question 5

Collaborating with cross-functional teams to resolve data quality issues

Accepted Answer

**Situation:** DQ issues affecting multiple teams. **Task:** Resolve with cross-functional alignment. **Action:** Define SLAs; automated checks (Deequ, Great Expectations); catalog + ownership; blameless postmortems; shared dashboards; embedded pipeline checks. **Result:** Trust via transparency; consistent delivery....

Question 6

Priority Queue Problem - task prioritization and dynamic sorting

Accepted Answer

**Why Priority Queues in Data Pipelines:** Task scheduling (Airflow, Celery), event processing (process by timestamp/priority), and merge of K sorted streams (heapq.merge).

**Implementations:** (1) heapq: min-heap, O(log n) push/pop. (2) queue.PriorityQueue: thread-safe wrapper. (3) For dynamic priority updates: need decrease_key—heapq doesn't support; use lazy deletion (mark stale, ignore when popped) or custom Fibonacci heap.

**Scalability:** Single-process heap = millions of items....

Question 7

Query Optimization Strategies

Accepted Answer

Query optimization: use indexes; avoid SELECT *; push filters early; use JOINs not subqueries; analyze execution plans; partition pruning; materialized views; statistics; limit result sets; avoid functions on indexed columns (e.g., WHERE YEAR(date)=2024 prevents index use—use range instead). **Why it matters**: Design choices compound at scale—wrong approach can cause 100× overhead. **Scalability trade-offs**: Profile before optimizing; validate on sample then full....

Question 8

Given two DataFrames, perform specified data transformations and store the result in a new DataFrame

Accepted Answer

**Why it matters**: At scale, design choices directly impact reliability, latency, and cost. Wrong decisions compound across jobs and teams.

Given df1, df2: common transformations—join (df1.join(df2, 'key')), union (df1.union(df2)), concat (unionByName with allowMissingColumns). Then apply filters, aggregates, select. Example: result = df1.join(df2, 'id').select(df1.id, df1.name, df2.score).filter(col('score')>70)....

PayPal Data Engineer Interview Questions

Reading isn't practice. Get AI feedback on your answers.

Other Companies

PayPal Data Engineer Interview Questions

Reading isn't practice. Get AI feedback on your answers.

Other Companies