Describe the concept of data sharding and when to use it.

Why Sharding Exists: Single-node storage and throughput limits cap scalability. Sharding horizontally partitions data by a shard key (e.g., user_id, region) across N nodes, enabling linear scale-out for reads/writes.

Architectural Logic: Each shard holds a subset; routing uses hash(key) mod N or consistent hashing. Partition boundaries should align with access patterns—sharding by user_id spreads load; by region enables geo-locality.

Scalability Trade-offs: Pro: linear scale, parallel I/O. Con: cross-shard joins are expensive; rebalancing on shard addition requires data movement. Hotspots occur if key distribution is skewed.

Cost Implications: More nodes = higher infra cost but better throughput. Premature sharding adds operational complexity; start with partitioning, shard when single-node limits are approached. At Goldman Sachs scale: shard keys must avoid regulatory or audit boundary violations.