Question 1

What are the best practices for logging and monitoring bad data?

Accepted Answer

(1) Structured logging—JSON with record_id, rule_id, severity, sample. (2) Centralized—ELK, Splunk, CloudWatch. (3) Metrics—failed count per rule; alert on spikes. (4) DLQ—store bad records for reprocessing. (5) Dashboards—DQ score, trends. (6) Sampling—log sample, not all. WHY: Root cause without full rerun. SCALABILITY: Sampling and aggregation; avoid log explosion....

Question 2

What are the implications of enabling schema auto-detection?

Accepted Answer

Pros: No manual schema; flexible. Cons: Inferred from sample—can be wrong; type errors; schema drift undetected; scan overhead. Best: Exploration or variable sources; production = explicit schema (Avro, JSON Schema) + validation. BigQuery: auto-detect can misinfer; provide schema for critical....

Question 3

Can Schema Evolution lead to data inconsistencies? If so, how do you manage them?

Accepted Answer

**Section 1 — The Context (The 'Why')**
This system faces scale and failure challenges at production. A naive approach breaks under load, loses data, or violates compliance. The primary challenge varies by domain: notifications need preference respect; banking needs ACID; pipelines need idempotency....

Question 4

Differentiate between Schema Enforcement and Schema Evolution.

Accepted Answer

**Section 1 — The Context (The 'Why')**
This system design addresses "Differentiate between Schema Enforcement and Schema Evolution...." at production scale. The primary challenges are throughput under variable load, fault tolerance across distributed components, and maintaining consistency guarantees that match business requirements. Naive monolithic approaches break when single points of failure emerge, network partitions occur, or backpressure from slow consumers cascades upstream....

PWC System Design Interview Questions

Companies asking these questions

All 4 Questions

More Interview Prep Guides

Get All Answers in PDF Format

PWC System Design Interview Questions

Companies asking these questions

All 4 Questions

More Interview Prep Guides

Get All Answers in PDF Format