Billable usage system table reference

This article provides an overview of the billable usage system table, including the schema and example queries. With system tables, your account’s billable usage data is centralized and routed to all regions, so you can view your account’s global usage from whichever region your workspace is in.

For information on using this table to monitor costs and sample queries, see Monitor costs using system tables.

Table path: This system table is located at system.billing.usage.

Billable usage table schema

The billable usage system table uses the following schema:

Column name	Data type	Description	Example
record_id	string	Unique ID for this usage record	11e22ba4-87b9-4cc2 -9770-d10b894b7118
account_id	string	ID of the account this report was generated for	23e22ba4-87b9-4cc2 -9770-d10b894b7118
workspace_id	string	ID of the Workspace this usage was associated with	1234567890123456
sku_name	string	Name of the SKU	STANDARD_ALL_PURPOSE_COMPUTE
cloud	string	Cloud this usage is relevant for. Possible values are AWS, AZURE, and GCP.	AWS, AZURE, or GCP
usage_start_time	timestamp	The start time relevant to this usage record. Timezone information is recorded at the end of the value with +00:00 representing UTC timezone.	2023-01-09 10:00:00.000+00:00
usage_end_time	timestamp	The end time relevant to this usage record. Timezone information is recorded at the end of the value with +00:00 representing UTC timezone.	2023-01-09 11:00:00.000+00:00
usage_date	date	Date of the usage record, this field can be used for faster aggregation by date	2023-01-01
custom_tags	map	Tags applied to this usage. Includes compute resource tags, jobs tags, and budget policy tags.	{ “env”: “production” }
usage_unit	string	Unit this usage is measured in. Possible values include DBUs.	DBU
usage_quantity	decimal	Number of units consumed for this record.	259.2958
usage_metadata	struct	System-provided metadata about the usage, including IDs for compute resources and jobs (if applicable). See Usage metadata reference.	{cluster_id: null; instance_pool_id: null; notebook_id: null; job_id: null; node_type: null}
identity_metadata	struct	System-provided metadata about the identities involved in the usage. See Identity metadata reference.	{"run_as": example@email.com,"created_by":null}
record_type	string	Whether the record is original, a retraction, or a restatement. The value is ORIGINAL unless the record is related to a correction. See Record type reference.	ORIGINAL
ingestion_date	date	Date the record was ingested into the usage table.	2024-01-01
billing_origin_product	string	The product that originated the usage. Some products can be billed as different SKUs. For possible values, see Billing origin product reference.	JOBS
product_features	struct	Details about the specific product features used.	For possible values, see Product features.
usage_type	string	The type of usage attributed to the product or workload for billing purposes. Possible values are COMPUTE_TIME, STORAGE_SPACE, NETWORK_BYTES, API_OPERATION, TOKEN, or GPU_TIME.	STORAGE_SPACE

Usage metadata reference

The values in usage_metadata tell you about the objects and resources involved in the usage record.

Value	Data type	Description
cluster_id	string	ID of the cluster associated with the usage record
warehouse_id	string	ID of the SQL warehouse associated with the usage record
instance_pool_id	string	ID of the instance pool associated with the usage record
node_type	string	The instance type of the compute resource
job_id	string	ID of the job associated with the usage record. Only returns a value for serverless compute or jobs compute usage, otherwise returns null.
job_run_id	string	ID of the job run associated with the usage record. Only returns a value for serverless compute or jobs compute usage, otherwise returns null.
job_name	string	User-given name of the job associated with the usage record. Only returns a value for jobs run on serverless compute, otherwise returns null.
notebook_id	string	ID of the notebook associated with the usage. Only returns a value for serverless compute for notebook usage, otherwise returns null.
notebook_path	string	Workspace storage path of the notebook associated with the usage. Only returns a value for serverless compute for notebook usage, otherwise returns null.
dlt_pipeline_id	string	ID of the Delta Live Tables pipeline associated with the usage record
dlt_update_id	string	ID of the Delta Live Tables pipeline update associated with the usage record
dlt_maintenance_id	string	ID of the Delta Live Tables pipeline maintenance tasks associated with the usage record
run_name	string	Unique user-facing identifier of the Foundation Model Fine-tuning run associated with the usage record
endpoint_name	string	The name of the model serving endpoint or vector search endpoint associated with the usage record
endpoint_id	string	ID of the model serving endpoint or vector search endpoint associated with the usage record
central_clean_room_id	string	ID of the central clean room associated with the usage record
source_region	string	Region of the workspace associated with the usage. Only returns a value for networking-related costs.
destination_region	string	Region of the resource being accessed. Only returns a value for networking-related costs.
metastore_id	string	ID of the metastore associated with the usage record.
app_id	string	ID of the app associated with the usage record.
app_name	string	User-given name of the app associated with the usage record.

Identity metadata reference

The identity_metadata column can help you identify who is responsible for a serverless billing record. The column includes a run_as value that attributes the usage to an identity.

Additionally, usage attributed to Databricks Apps log a value in the identity_metadata.created_by field. This value is populated with the email of the user who created the app.

The identity recorded in identity_metadata.run_as depends on the product associated with the usage. Reference the following table for the identity_metadata.run_as behavior:

Workload type	Identity of run_as
Jobs compute	The user or service principal defined in the run_as setting. By default, jobs run as the identity of the job owner, but admins can change this to be another user or service principal.
Serverless compute for jobs	The user or service principal defined in the run_as setting. By default, jobs run as the identity of the job owner, but admins can change this to be another user or service principal.
Serverless compute for notebooks	The user who ran the notebook commands (specifically, the user who created the notebook session). For shared notebooks, this includes usage by other users sharing the same notebook session.
Delta Live Tables pipelines	The user whose permissions are used to run the Delta Live Tables pipeline. This can be changed by transferring the pipeline’s ownership.
Foundation Model Fine-tuning	The user or service principal that initiated the fine-tuning training run.
Predictive optimization	The Databricks-owned service principal that runs predictive optimization operations.
Lakehouse monitoring	The user who created the monitor.

note

In workspaces enabled for the FedRamp compliance standard, all non-null values in the identity_metadata column will be replaced with __REDACTED__.

Record type reference

The billing.usage table supports corrections. Corrections occur when any field of the usage record is incorrect and must be fixed.

When a correction happens, Databricks adds two new records to the table. A retraction record negates the original incorrect record, then a restatement record includes the corrected information. Correction records are identified using the record_type field:

• RETRACTION: Used to negate the original incorrect usage. All fields are identical to the ORIGINAL record except usage_quantity, which is a negative value that cancels out the original usage quantity. For example, if the original record’s usage quantity was 259.4356, then the retraction record would have a usage quantity of -259.4356.
• RESTATEMENT: The record that includes the correct fields and usage quantity.

For example, the following query returns the correct hourly usage quantity related to a job_id, even if corrections have been made. By aggregating the usage quantity, the retraction record negates the original record and only the restatement’s values are returned.

SQL

SELECT
  usage_metadata.job_id, usage_start_time, usage_end_time,
  SUM(usage_quantity) as usage_quantity
FROM system.billing.usage
GROUP BY ALL
HAVING usage_quantity != 0

note

For corrections where the original usage record should not have been written, a correction may only add a retraction record and no restatement record.

Billing origin product reference

Some Databricks products are billed under the same shared SKU. To help you differentiate usage, the billing_origin_product and product_features columns provide more insight into the specific product and features associated with the usage.

The billing_origin_product column shows the Databricks product associated with the usage record. The values include:

• JOBS
• DLT
• SQL
• ALL_PURPOSE
• MODEL_SERVING
• INTERACTIVE
• DEFAULT_STORAGE
• VECTOR_SEARCH
• LAKEHOUSE_MONITORING
• PREDICTIVE_OPTIMIZATION
• ONLINE_TABLES
• FOUNDATION_MODEL_TRAINING
• AGENT_EVALUATION
• FINE_GRAIN_ACCESS_CONTROL

• NETWORKING: Costs associated with connecting serverless compute to your workspace resources

• APPS: Costs associated with building and running Databricks Apps

Product features reference

The product_features column is an object containing information about the specific product features used and includes the following key/value pairs:

• jobs_tier: values include LIGHT, CLASSIC, or null
• sql_tier: values include CLASSIC, PRO, or null
• dlt_tier: values include CORE, PRO, ADVANCED, or null
• is_serverless: values include true or false, or null
• is_photon: values include true or false, or null
• serving_type: values include MODEL, GPU_MODEL, FOUNDATION_MODEL, FEATURE, or null

• networking.connectivity_type: values include PUBLIC_IP and PRIVATE_IP