Running multiple notebooks - dbutils.notebook.run()

Why dbutils.notebook.run(): Modular workflows—reusable notebook components, parameterized execution. Syntax: dbutils.notebook.run("/path/to/notebook", timeout_seconds, {"param": "value"}). Returns the last evaluated expression. Difference from %run: %run executes inline in the same context; variables are shared. dbutils.run runs in a separate Spark context; no variable sharing; returns output. Use %run for quick composition; dbutils.run for job workflows, parallel execution, and passing arguments. Scalability: Each run spawns a task/job; many runs = many tasks. For 100 notebooks in sequence, consider a single orchestration notebook. Cost: Each run consumes cluster resources; long timeouts hold resources. Set reasonable timeout_seconds. Best practice: Pass minimal data—paths, IDs, not DataFrames. Avoid large payloads in widget values. Use exit value for status codes (0=success, 1=failure) for downstream logic.