This feature is in Public Preview.
Databricks pools reduce cluster start and auto-scaling times by maintaining a set of idle, ready-to-use instances. When a cluster attached to a pool needs an instance, it first attempts to allocate one of the pool‘s idle instances. If the pool has no idle instances, the pool expands by allocating a new instance from the instance provider in order to accommodate the cluster’s request. When a cluster releases an instance, it returns to the pool and is free for another cluster to use. Only clusters attached to a pool can use that pool‘s idle instances.
Databricks does not charge DBUs while instances are idle in the pool. Instance provider billing does apply; see pricing.
To use pools, you must add new instance profile and tag
permissions to the IAM role or keys used to create your account. In particular, you must
add the permissions
ec2:DeleteTags. For the complete list
of permissions and instructions on how to update your existing IAM role or keys, see
You can create and manage pools using the UI and by invoking the API. This section focuses on performing pool tasks using the UI. For information on the API, see the Instance Pools API.