Reviewed by Aditya Kumar · Last reviewed 2026-03-24
**Architectural Logic**: Flags enable observability, restartability, and audit. **Flags**: STARTED, RUNNING, SUCCESS, FAILED; custom: DATA_QUALITY_FAILED, PARTIAL_SUCCESS. **Segregation**: Extract, Transform, Load as distinct tasks; each has status. **Why**: Monitoring; resume...
Architectural Logic: Flags enable observability, restartability, and audit. Flags: STARTED, RUNNING, SUCCESS, FAILED; custom: DATA_QUALITY_FAILED, PARTIAL_SUCCESS. Segregation: Extract, Transform, Load as distinct tasks; each has status. Why: Monitoring; resume from failed step; audit trail. Example: Log batch_id, step_name, status, rows_processed, timestamp. Orchestrators (Airflow) use task states. Scalability: Idempotent steps; flags enable partial rerun. Cost: Minimal; status tables or metadata store. Best Practice: Separate concerns; use flags for observability; allow resume; implement idempotency.
Red Flag: Monolithic ETL with no step separation—can't resume. Pro-Move: Reference checkpointing; status table; Airflow task states; idempotent steps for safe rerun.
According to DataEngPrep.tech, this is one of the most frequently asked SQL interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.