Run Databricks notebooks, jobs and SQL with Claude Code

Watch Claude Code use the Databricks CLI to create a notebook, turn it into a job and fix a failing run.

Who it's for: Anyone who builds analyses or jobs in Databricks.

The problem: Creating analyses and jobs in Databricks can take hours or even days. The native Claude Desktop connector said it was connected but did not work, and the Databricks Lab MCP has been deprecated.

The video installs the Databricks CLI with Homebrew, authenticates it with OAuth or a personal access token, and gives Claude a CLAUDE.md quick reference for the CLI. In plan mode, Claude Code explores Unity Catalog, creates a Python notebook, turns it into a job, then troubleshoots a job built to fail.

Takeaways:

  • OAuth is more secure; a personal access token is more convenient.
  • Check the results in the Databricks UI instead of taking Claude at its word.
  • The same approach works with other generative AI tools in your command line.

Resources