Question 1

Can you describe the role of user groups in setting up these policies?

Accepted Answer

**Groups in RBAC**: Assign permissions to groups; add users to groups. Simplifies management—add user to analyst group vs. granting per-object. **Databricks**: Create group; grant on schema/table. Sync from IdP (Okta, Azure AD)....

Question 2

How do these transformations impact memory usage?

Accepted Answer

**Impact**: Shuffle (groupBy, join)—data redistribution; spill if OOM. collect()/toPandas()—pull to driver; avoid on large. cache()—uses memory/disk. Wide transformations—many output partitions. **Mitigation**: More partitions; broadcast small joins; avoid collect; streaming for large. Monitor Spark UI....

Question 3

How do you ensure version control when migrating notebooks?

Accepted Answer

**Approach**: (1) Export to .py or .ipynb; (2) Git—commit notebooks; use nbdime for diff; (3) Review—notebooks are JSON; (4) CI—papermill, nbval for tests; (5) No secrets—templates, inject at runtime. Pair notebooks with .py modules. Databricks Repos for sync....

Question 4

How do you handle passing parameters between notebooks?

Accepted Answer

**Databricks**: Widgets (dbutils.widgets.text); %run ./notebook $param=value. **Papermill**: papermill input.ipynb output.ipynb -p param value. **Shared state**: DBFS file or table. **Orchestrator**: Airflow passes params to tasks....

Question 5

How do you identify resource bottlenecks in cluster logs?

Accepted Answer

**Sources**: Spark UI (tasks, stages, shuffle); executor logs (memory, GC, CPU); driver logs; iostat, top. **Signals**: Skew (one task 10x slower); memory (spill, OOM); network (shuffle); stragglers. Tools: Spark UI, Prometheus....

Question 6

How does cluster size impact parallelism limits?

Accepted Answer

**Relationship**: More workers = more concurrent tasks. Cores per worker = tasks per worker. Total parallelism ≈ min(data_partitions, total_cores). Diminishing returns—overhead dominates. Right-size; auto-scale; monitor utilization....

Question 7

WAQ for Desired Output (Age Group Count)

Accepted Answer

SELECT CASE WHEN age BETWEEN 0 AND 18 THEN '0-18' WHEN age BETWEEN 19 AND 35 THEN '19-35' WHEN age BETWEEN 36 AND 50 THEN '36-50' ELSE '51+' END AS age_group, COUNT(*) FROM people GROUP BY 1 ORDER BY 1. FLOOR(age/10)*10 for decades. Handle NULL; mutually exclusive buckets....

Question 8

WAQ for Desired Output (Node Parent Relationship)

Accepted Answer

Recursive CTE for path: WITH RECURSIVE cte AS (SELECT id, parent_id, CAST(id AS VARCHAR) path FROM tree WHERE parent_id IS NULL UNION ALL SELECT t.id, t.parent_id, cte.path||'-'||t.id FROM tree t JOIN cte ON t.parent_id=cte.id) SELECT * FROM cte. Flat: SELECT node_id, parent_id ORDER BY parent_id NULLS FIRST....

Question 9

What are the implications of enabling encryption at rest on storage performance?

Accepted Answer

Modern: AES-NI hardware—negligible overhead (<5%). Older/software-only may add latency. KMS key fetch: minimal per I/O. Best: provider-managed (S3 SSE, GCS)—no measurable impact. WHY: Enable by default for security; performance rarely a concern.

Question 10

What are the security considerations for the control plane?

Accepted Answer

Control plane = resource management (create, delete, configure). Auth: MFA; minimal service accounts. Authorization: IAM least privilege; no broad admin. Audit: CloudTrail, immutable logs. Network: Private endpoints, VPC. Secrets: Rotate; secrets manager. Compliance: Encryption, retention. For EMR/Glue: Limit cluster create/delete; permission boundaries....

TCS General Interview Questions

Companies asking these questions

All 12 Questions

More Interview Prep Guides

Get All Answers in PDF Format

TCS General Interview Questions

Companies asking these questions

All 12 Questions

More Interview Prep Guides

Get All Answers in PDF Format