Develop and deploy Lakeflow Jobs on Databricks via DABs, Python SDK, or the CLI. Use when creating data engineering jobs with notebooks, Python wheels, SQL, dbt, or pipelines. Invoke BEFORE starting implementation.

Lakeflow Jobs Development FIRST : Use the parent databricks core skill for CLI basics, authentication, profile selection, and data exploration commands. Lakeflow Jobs orchestrate data workflows with multi task DAGs, flexible triggers, and comprehensive monitoring. Jobs support diverse task types and can be managed via Asset Bundles (DABs), Python SDK, or CLI. Reference Files Use Case Reference File Configure task types (notebook, Python, SQL, dbt, pipeline, JAR, run job, for each) [references/task types.md](references/task types.md) Set up triggers and schedules (cron, periodic, file arrival, table update, continuous) [references/triggers schedules.md](references/triggers schedules.md) Configure notifications, health rules, retries, timeouts, queues [references/notifications monitoring.md](references/notifications monitoring.md) Complete worked examples (ETL, warehouse refresh, event driven, ML training, multi env, streaming, cross job) [references/examples.md](references/examples.md) Scaffolding a New Job Project Use databricks bundle init with a config file to scaffold non interactively. This creates a project in the <project name / directory: project name : letters, numbers, underscores only After scaffolding, create CLAUDE.md and AGENTS.md in the project directory. These files are essential to provide agents with guidance on how to work with the project. Use this content: Project Structure Quick Start Asset Bundles (DABs) — recommended Python SDK CLI Core Concepts Multi Task Workflows Jobs support DAG based task dependencies: run if conditions: ALL SUCCESS (default) — run when all dependencies succeed ALL DONE — run when all dependencies complete (success or failure) AT LEAST ONE SUCCESS — run when at least one dependency succeeds NONE FAILED — run when no dependencies failed ALL FAILED — run when all dependencies failed AT LEAST ONE FAILED — run when at least one dependency failed Task Types Summary Task Type Use Case Reference notebook task Run notebooks [references/task types.md notebook task](references/task types.md notebook task) spark python task Run Python scripts [references/task types.md spark python task](references/task types.md spark python task) python wheel task Run Python wheels [references/task types.md python wheel task](references/task types.md python wheel task) sql task Run SQL queries/files/dashboards/alerts [references/task types.md sql task](references/task types.md sql task) dbt task Run dbt projects [references/task types.md dbt task](references/task types.md dbt task) pipeline task Trigger SDP (formerly DLT) pipelines [references/task types.md pipeline task](references/task types.md pipeline task) spark jar task Run Spark JARs [references/task types.md spark jar task](references/task types.md spark jar task) run job task Trigger other jobs [references/task types.md run job task](references/task types.md run job task) for each task Loop over inputs [references/task types.md for each task](references/task types.md for each task) Trigger Types Summary Trigger Type Use Case Reference schedule Cron based scheduling [references/triggers schedules.md cron schedule](references/triggers schedules.md cron schedule) trigger.periodic Interval based [references/triggers schedules.md periodic trigger](references/triggers schedules.md periodic trigger) trigger.file arrival File arrival events [references/triggers schedules.md file arrival trigger](references/triggers schedules.md file arrival trigger) trigger.table update Unity Catalog table change events [references/triggers schedules.md table update trigger](references/triggers schedules.md table update trigger) continuous Always running jobs [references/triggers schedules.md continuous jobs](references/triggers schedules.md continuous jobs) Compute Configuration Job Clusters (recommended) Define reusable cluster configurations shared across tasks: Autoscaling Clusters Existing Cluster Serverless Compute For notebook and Python tasks, omit cluster configuration to use serverless: Job Parameters Parameters defined at job level are passed to ALL tasks (no need to repeat per task): Access in notebooks: Pass to specific tasks: Common Operations Python SDK CLI Asset Bundle Operations Permissions (DABs) Permission levels: CAN VIEW — view job and run history CAN MANAGE RUN — view, trigger, and cancel runs CAN MANAGE — full control including edit and delete Unit Testing Run unit tests locally: Development Workflow 1. Validate : databricks bundle validate profile <profile 2. Deploy : databricks bundle deploy t dev profile <profile 3. Run : databricks bundle run <job name t dev profile <profile 4. Check run status : databricks jobs get run run id <id profile <profile Common Issues Issue Solution Job cluster startup slow Use job clusters with job cluster key for reuse across tasks Task dependencies not working Verify task key references match exactly in depends on Schedule not triggering Check pause status: UNPAUSED and valid timezone File arrival not detecting Ensure path has proper permissions and uses cloud storage URL Table update trigger missing events Verify Unity Catalog table and proper grants Parameter not accessible Use dbutils.widgets.get() in notebooks admins group error Cannot modify admins permissions on jobs Serverless task fails Ensure task type supports serverless (notebook, Python) Related Skills databricks dabs — DABs configuration patterns shared by jobs and pipelines databricks pipelines — SDP (formerly DLT) pipelines triggered by pipeline task Documentation [Lakeflow Jobs](https://docs.databricks.com/jobs) [Task types](https://docs.databricks.com/jobs/configure task) [Declarative Automation Bundles](https://docs.databricks.com/dev tools/bundles/) [Jobs API Reference](https://docs.databricks.com/api/workspace/jobs) [Bundle Examples Repository](https://github.com/databricks/bundle examples)

databricks-jobs