Design a data pipeline to collect, process, and visualize customer feedback from Adidas stores worldwide.

Section 1 — The Context (The 'Why')
Customer feedback pipelines aggregate unstructured data from surveys, reviews, and social media into actionable insights. The primary challenge is handling high variance in volume (product launches spike), schema heterogeneity (different survey formats, languages), and the need for both real-time sentiment dashboards and batch NLP analysis. A naive single-path approach breaks when NLP jobs (hours) block real-time dashboards.

Section 2 — The Diagram

[Web/Mobile] --> [API Gateway] --> [Kinesis] --> [Lambda/Sentiment]
     |                                   |              |
     v                                   v              v
[Survey DB] --> [Glue CDC] --> [S3 Bronze] --> [Comprehend/NLP]
                                          |
                                          v
                               [Redshift/QuickSight Dashboard]

Section 3 — Component Logic
Feedback events flow through API Gateway to Kinesis for real-time ingestion. Lambda or lightweight Flink jobs compute sentiment scores and route to a hot-path store for dashboards. Batch feedback lands in S3 via Glue; CDC syncs from survey databases. The bronze layer stores raw JSON/CSV; silver applies schema validation and language detection. NLP runs on scheduled batches; results merge into gold aggregations. Partitioning by product_id or date enables efficient querying. Backpressure: Kinesis shard scaling and Lambda concurrency limits prevent overload. Fan-out allows one stream to feed sentiment and raw archive. In production, monitor consumer lag, checkpoint success rate, and sink write latency as primary SLOs. Partitioning strategies should align with query patterns; bucketing within partitions mitigates join skew. TTL policies on raw and intermediate data control storage cost while preserving replay capability for debugging and backfill. Data skew mitigation via salting or secondary hashing prevents single partitions from becoming bottlenecks. Exactly-once semantics require transactional commits at the sink; at-least-once delivery demands idempotent write logic to avoid duplicates. Fan-out patterns allow one source topic to feed multiple downstream consumers without re-ingestion. Backpressure handling ensures that slow processors do not cause unbounded buffer growth; Kafka consumer lag is a key metric. Schema evolution should follow additive-only rules where possible to avoid breaking consumer compatibility. The CAP trade-off should be documented per component: analytics typically favors AP, while financial reconciliation requires CP. Blast radius from component failure is bounded by replication and checkpointing; design for graceful degradation during partial outages. Cost optimization: use Spot instances for batch workloads and tier cold data to lower storage classes. Dead-letter queues preserve failed records for replay rather than dropping them.

Section 4 — The Trade-offs (The 'Senior' part)

Section 5 — Pro-Tip