#partition

What is the small-file problem in Spark, and how do you solve it?

How do you optimize Spark jobs for better performance? Mention at least 5 techniques.

Tell me about a time when you faced a challenging situation at work and how you handled it.

What challenges did you face, and how did you tackle them?

What would you do if a pipeline failed and you couldn't find the reason?

What is Snowflake's architecture, and why is it unique?

Briefly explain the architecture of Kafka.

Describe the data pipeline architecture you've worked with.

Have you worked on Data Warehousing projects?

How would you read data from a web API? What steps would you follow after reading the data?

Retrieve the most recent sale_timestamp for each product (Latest Transaction).

What is the difference between OLTP and OLAP?

Difference Between Internal and External Tables in BigQuery

Difference between ROW_NUMBER(), RANK(), and DENSE_RANK() with examples.

Explain SQL Window Functions with examples.

Explain the use of the MERGE statement in SQL.

How do you optimize a long-running SQL query?

How would you handle duplicate records in an SQL table?

Implement a query to find the top 5 customers by total sales amount.

SQL query to find the second highest salary from each department.