Reviewed by Aditya Kumar · Last reviewed 2026-03-24
**Situation**: Near-real-time fraud detection needed sub-30-second latency. **Task**: Build streaming pipeline with exactly-once semantics. **Action**: Architecture: Kafka → Databricks Structured Streaming → Delta Lake; ML model scoring in streaming; alerts to operations....
Situation: Near-real-time fraud detection needed sub-30-second latency. Task: Build streaming pipeline with exactly-once semantics. Action: Architecture: Kafka → Databricks Structured Streaming → Delta Lake; ML model scoring in streaming; alerts to operations. Implemented checkpointing for exactly-once; windowed aggregations for patterns. Challenges: Late data (watermarking), backpressure (auto-scaling, rate limiting). Result: Detection latency under 30 seconds; 99.9% uptime. Scalability: Delta for ACID and time travel; batch fallback for reprocessing. Leadership: Drove architecture choice; documented runbooks for ops.
Red Flag: Calling it 'real-time' when it's batch—be precise (near-real-time, seconds latency). Pro-Move: Mention exactly-once, checkpointing, watermarking; reference SLA and uptime.
According to DataEngPrep.tech, this is one of the most frequently asked SQL interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.