#spark

When would you architecturally choose Dataset[T] over DataFrame in a Scala Spark pipeline, and what are the scalability and portability trade-offs? Include type-safety benefits vs. operational constraints.

What is the difference between Managed and External Tables in Databricks?

Describe a time when you had to work with a team to solve a complex problem. What was your role, and how did the team approach the problem?

Describe your role in a team project.

What are your strengths and weaknesses?

How do you handle version conflicts for libraries?

Introduce yourself, highlighting key projects and tech stacks

Provide a detailed walkthrough of your career journey