Reviewed by Aditya Kumar · Last reviewed 2026-03-25
**Section 1 — The Context (The 'Why')** Real-time stock trade systems must capture every tick with millisecond latency. The primary challenge is write throughput, exactly-once semantics for P&L, and querying raw and aggregated data....
This hard-level System Design/Architecture question appears frequently in data engineering interviews at companies like Goldman Sachs. While less common, it tests deeper understanding that distinguishes strong candidates. Mastering the underlying concepts (join, optimization, partition) will help you answer variations of this question confidently.
This is a senior-level question that tests architectural thinking. Lead with the high-level design, then drill into specifics. Discuss trade-offs explicitly - there is rarely one correct answer. Show awareness of scale, fault tolerance, and operational complexity. The expert answer includes a code example that demonstrates the implementation pattern.
Section 1 — The Context (The 'Why')
Real-time stock trade systems must capture every tick with millisecond latency. The primary challenge is write throughput, exactly-once semantics for P&L, and querying raw and aggregated data. At-least-once without dedup causes regulatory risk.
Section 2 — The Diagram
[Exchange] --> [Kafka] --> [TimescaleDB] --> [Agg View]
| | | |
v v v v
[Schema Reg] [Symbol Part] [Hypertables] [Redis/Position]
Section 3 — Component Logic
Exchange feeds into Kafka partitioned by symbol. TimescaleDB ingests; hypertables partition by time. Append-only trade log; idempotency via exchange_trade_id. Redis for position and P&L. Exactly-once via consumer commit after DB write. In production, monitor consumer lag, checkpoint success rate, and sink write latency as primary SLOs. Partitioning strategies should align with query patterns; bucketing within partitions mitigates join skew. TTL policies on raw and intermediate data control storage cost while preserving replay capability for debugging and backfill. Data skew mitigation via salting or secondary hashing prevents single partitions from becoming bottlenecks. Exactly-once semantics require transactional commits at the sink; at-least-once delivery demands idempotent write logic to avoid duplicates. Fan-out patterns allow one source topic to feed multiple downstream consumers without re-ingestion. Backpressure handling ensures that slow processors do not cause unbounded buffer growth; Kafka consumer lag is a key metric. Schema evolution should follow additive-only rules where possible to avoid breaking consumer compatibility. The CAP trade-off should be documented per component: analytics typically favors AP, while financial reconciliation requires CP. Blast radius from component failure is bounded by replication and checkpointing; design for graceful degradation during partial outages. Cost optimization: use Spot instances for batch workloads and tier cold data to lower storage classes. Dead-letter queues preserve failed records for replay rather than dropping them.
Section 4 — The Trade-offs (The 'Senior' part)
Section 5 — Pro-Tip
Pro-Move: Partition by symbol; append-only with exchange_trade_id. Red Flag: At-least-once without dedup - P&L wrong; regulatory risk.
Some links below are affiliate links. If you buy through them we may earn a small commission at no extra cost to you — it helps keep DataEngPrep free.
According to DataEngPrep.tech, this is one of the most frequently asked System Design/Architecture interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.