Question 1

How do you handle data security and compliance in a cloud environment?

Accepted Answer

Security is layered: (1) Encryption: At rest (KMS-managed keys, SSE-S3, Azure Storage encryption) and in transit (TLS). Why: Compliance (GDPR, HIPAA) and breach mitigation. Trade-off: Key management adds latency and complexity; managed services reduce operational burden. (2) Access: Least-privilege IAM, role-based access, no long-lived keys in code. Use VPC/VNet for network isolation; private endpoints for data stores....

Question 2

Describe a scenario where AWS Data Pipeline is preferred over Glue. Why?

Accepted Answer

Architectural logic: Data Pipeline = templated, EC2/EMR-managed; Glue = serverless Spark, Catalog, modern ETL. When Data Pipeline wins: (1) Cost—simple daily RDS→S3 copy can be cheaper than Glue DPUs; (2) Legacy—existing Pipeline definitions; (3) EC2/EMR coupling—Data Pipeline natively manages clusters for custom non-Spark code; (4) Simplicity—straight copy needs no Spark. Scalability: Data Pipeline scales via instance count; Glue via DPU....

Question 3

Describe how you would use AWS Glue to schedule and manage Spark jobs.

Accepted Answer

Architectural logic: Glue Jobs = serverless Spark; Triggers = orchestration (schedule, event, on-demand). Flow: Define job (script, DPU, timeout) → add Trigger (cron, e.g., 0 6 * * ? *) → chain jobs if needed. Job bookmarks enable incremental processing. Scalability: DPU auto-scales; Triggers can fan-out. Cost: DPU-hours; over-provisioning wastes money....

Question 4

Discuss the key differences between AWS Glue, Lambda, and Data Pipeline for orchestrating data workflows.

Accepted Answer

Architectural comparison: Glue = serverless Spark, Catalog, complex ETL. Lambda = event-driven, 15-min max, lightweight. Data Pipeline = templated, EC2/EMR-managed. When to use: Glue for data lake ETL; Lambda for triggers, small tasks; Data Pipeline for simple, cost-sensitive batch. Scalability: Glue via DPU; Lambda concurrency; Data Pipeline via instances. Cost: Lambda cheapest for small; Glue for Spark; Data Pipeline for simple batch....

Question 5

Explain how AWS Glue interacts with on-premises SQL databases to extract data efficiently.

Accepted Answer

Architectural logic: Glue runs in AWS; connector/agent in on-prem network proxies connection. Efficient extraction: (1) Pushdown—filter at source. (2) Partition-based extract (date). (3) Incremental via bookmarks/watermark. (4) Parallel reads by splitting on numeric key. Why: Full scans over WAN are slow and expensive....

Question 6

Explain when you would use Glue instead of Lambda for a data ingestion use case.

Accepted Answer

Architectural decision: Glue when—large data, Spark needed, >15 min, JDBC/Spark connectors. Lambda when—small payloads, event-driven, <15 min. Example: 100GB S3 transform → Glue. S3 put → validate → DynamoDB → Lambda....

Question 7

In AWS Data Pipeline, how would you design a process to copy only recently modified files from one S3 bucket to another?

Accepted Answer

**Section 1 — The Context (The 'Why')**
The primary challenge for this design in Cloud/Tools is balancing scale, cost, and reliability. At scale, naive approaches fail: single points of failure cause cascades, schema evolution breaks consumers, and over-provisioning explodes cost. Failure modes include silent data loss from non-idempotent writes, cascading job failures from tight coupling, and operational burden from manual intervention....

EPAM Cloud & Tools Interview Questions

Companies asking these questions

All 7 Questions

More Interview Prep Guides

Get All Answers in PDF Format

EPAM Cloud & Tools Interview Questions

Companies asking these questions

All 7 Questions

More Interview Prep Guides

Get All Answers in PDF Format