Question 1

Explain the Medallion Architecture (Bronze, Silver, Gold layers).

Accepted Answer

**Section 1 — The Context (The 'Why')**
Medallion addresses tension between raw preservation (audit) and curated analytics. Bronze retains everything; Silver/Gold add structure.

**Section 2 — The Diagram**
```
[Sources] --> [Bronze] --> [Silver] --> [Gold]
  Append    Dedup/Merge   Marts
```

**Section 3 — Component Logic**
**Bronze** raw append-only, immutable. **Silver** schema, dedup, Delta MERGE. **Gold** star schemas. **TTL policies** move Bronze to Glacier....

Question 2

Explain the difference between Service Principal and Managed Identity in Azure.

Accepted Answer

Architectural difference: Service Principal = app identity; client ID, tenant, secret/certificate; for automation, CI/CD, third-party. Managed Identity = Azure-managed; no secrets; tied to resource (VM, ADF, Function). System-assigned vs user-assigned. When: Managed Identity for Azure resources—simpler, no secret rotation. Service Principal for external, non-Azure, Terraform....

Question 3

Secret Scope usage for managing credentials securely.

Accepted Answer

**Why Secret Scopes**: Credentials in notebooks or config get committed or logged. Secret Scopes provide a namespace for secrets, accessed via dbutils.secrets.get(scope="my-scope", key="api-key"). Never log the return value. **Types**: Databricks-backed (stored in Databricks) and Key Vault-backed (Azure) or AWS Secrets Manager-backed—enterprise secrets stay in your vault. **Scalability**: Many secrets across many clusters—use ACLs to restrict scope access per group....

Question 4

When to use each for secure resource access.

Accepted Answer

**IAM roles:** Service-to-service (Lambda→S3). Prefer over keys. **Service accounts:** Apps; rotate creds. **OAuth2/OpenID:** User-facing; PKCE for public. **API keys:** External partners; narrow scope; rotate. **Secrets managers:** Store keys; never commit. **Why each:** Least privilege; workload identity when available....

Question 5

Importance of each layer in Databricks.

Accepted Answer

**Why it matters**: At scale, design choices directly impact reliability, latency, and cost. Wrong decisions compound across jobs and teams.

Databricks layers: (1) Bronze—raw ingestion, schema on read. (2) Silver—cleaned, deduplicated, validated. (3) Gold—aggregated, business-ready. Medallion architecture enables incremental processing, auditability, and clear ownership....

Question 6

Steps to mount storage in Databricks.

Accepted Answer

**Why Mount vs. URI**: Mounts provide a stable path (`/mnt/data`) and handle auth once; URIs (`s3://bucket/path`) require per-access credentials. For multi-workspace, Unity Catalog external locations replace mounts.

**Steps (Azure ADLS)**: (1) Create service principal; grant Storage Blob Data Contributor. (2) Mount: `dbutils.fs.mount(source="abfss://container@account.dfs.core.windows.net/", mount_point="/mnt/mydata", extra_configs={...})`....

Chubb Data Engineer Interview Questions

Reading isn't practice. Get AI feedback on your answers.

Other Companies

Chubb Data Engineer Interview Questions

Reading isn't practice. Get AI feedback on your answers.

Other Companies