Delta Sharing: Zero-Copy Data Exchange Across Workspaces and Beyond

Data teams don’t operate in isolation. The marketing analytics team needs the customer 360 table from the data engineering group. The external consulting firm needs read access to aggregated metrics. The ML team in a different workspace needs the feature store tables.

The traditional answer is data copies. Export to S3, share credentials, hope the consumer picks up the right version. Or worse, set up a dedicated ETL pipeline just to move data from one workspace to another. Every copy is a liability: stale data, duplicated storage costs, credentials sprawling across systems.

Delta Sharing takes a different approach. The provider publishes a share. The consumer reads from it. No data moves. No credentials are exchanged for the underlying storage. The provider controls what’s shared at the table level, and Unity Catalog enforces it.

This works across Databricks workspaces, and it works outside Databricks entirely. The Delta Sharing protocol is open – any client that speaks it can consume shared tables. That includes pandas, Apache Spark OSS, and PowerBI.

How Delta Sharing Works

#

flowchart TB
    subgraph Provider["Provider Workspace"]
        UC["Unity Catalog"]
        T1["gold.customer_360"]
        T2["gold.revenue_metrics"]
        SH["Share: analytics_share"]
        UC --> T1
        UC --> T2
        T1 --> SH
        T2 --> SH
    end

    subgraph DBX["Consumer Workspace (Databricks)"]
        RC["Recipient Catalog"]
        Q1["SELECT * FROM shared_catalog..."]
    end

    subgraph EXT["External Consumer"]
        PY["Python / pandas"]
        PBI["Power BI"]
    end

    SH -->|"Databricks-to-Databricks
(automatic)"| RC --> Q1
    SH -->|"Open protocol
(activation link)"| PY
    SH -->|"Open protocol"| PBI

    style SH fill:#3498db,color:white
    style RC fill:#2ecc71,color:white
    style PY fill:#e74c3c,color:white
    style PBI fill:#e74c3c,color:white

Key concepts:

• Share – a named collection of tables (and optionally schemas) that you publish for consumption
• Recipient – an identity that can access a share. Can be another Databricks workspace or an external consumer
• Grant – the permission linking a share to a recipient

The provider never exposes storage credentials. The consumer queries through the Delta Sharing protocol, and Unity Catalog authorizes every request.

Step 1: Prepare the Provider Tables

#

First, set up the data you want to share. These tables live in your Unity Catalog:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

CREATE CATALOG IF NOT EXISTS analytics;
CREATE SCHEMA IF NOT EXISTS analytics.gold;

-- Customer 360 table
CREATE TABLE IF NOT EXISTS analytics.gold.customer_360 (
    customer_id STRING,
    email STRING,
    tier STRING,
    region STRING,
    lifetime_value DOUBLE,
    last_order_date DATE,
    updated_at TIMESTAMP
)
USING DELTA
TBLPROPERTIES ('delta.enableChangeDataFeed' = 'true');

-- Revenue metrics
CREATE TABLE IF NOT EXISTS analytics.gold.revenue_metrics (
    date DATE,
    region STRING,
    product_category STRING,
    revenue DOUBLE,
    orders INT,
    avg_order_value DOUBLE
)
USING DELTA
TBLPROPERTIES ('delta.enableChangeDataFeed' = 'true');

-- Insert sample data
INSERT INTO analytics.gold.customer_360 VALUES
    ('C001', '[email protected]', 'gold', 'US-WEST', 45000.00,
     '2026-03-15', current_timestamp()),
    ('C002', '[email protected]', 'silver', 'EU-WEST', 12000.00,
     '2026-04-01', current_timestamp()),
    ('C003', '[email protected]', 'platinum', 'APAC', 89000.00,
     '2026-04-10', current_timestamp());

INSERT INTO analytics.gold.revenue_metrics VALUES
    ('2026-04-01', 'US-WEST', 'electronics', 1250000.00, 3400, 367.65),
    ('2026-04-01', 'EU-WEST', 'electronics', 890000.00, 2100, 423.81),
    ('2026-04-01', 'APAC', 'electronics', 2100000.00, 5800, 362.07);

Step 2: Create a Share and Add Tables

#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

-- Create the share
CREATE SHARE IF NOT EXISTS analytics_share
COMMENT 'Curated analytics tables for cross-team consumption';

-- Add tables to the share
ALTER SHARE analytics_share
ADD TABLE analytics.gold.customer_360
COMMENT 'Customer 360 view - refreshed daily';

ALTER SHARE analytics_share
ADD TABLE analytics.gold.revenue_metrics
COMMENT 'Daily revenue aggregates by region and category';

-- Verify what's in the share
SHOW ALL IN SHARE analytics_share;

You can also share at the schema level:

1
2
3

-- Share an entire schema (all current and future tables)
ALTER SHARE analytics_share
ADD SCHEMA analytics.gold;

Schema-level sharing is convenient but less controlled. For most use cases, explicit table grants are safer.

Step 3: Set Up Recipients

#

Databricks-to-Databricks Recipient

#

For sharing between Databricks workspaces in the same account (or with metastore-level sharing enabled):

1
2
3
4
5
6
7

-- Create a recipient for another workspace
CREATE RECIPIENT IF NOT EXISTS team_marketing
COMMENT 'Marketing analytics workspace'
USING ID 'aws:us-west-2:<metastore-uuid>';

-- Grant access
GRANT SELECT ON SHARE analytics_share TO RECIPIENT team_marketing;

The consumer workspace sees the shared tables as a catalog:

1
2
3
4
5
6
7

-- On the consumer workspace:
CREATE CATALOG IF NOT EXISTS shared_analytics
USING SHARE analytics.analytics_share;

-- Query shared data directly
SELECT * FROM shared_analytics.gold.customer_360
WHERE region = 'US-WEST';

No data was copied. The query goes through the Delta Sharing protocol, the provider’s Unity Catalog authorizes it, and the data is read directly from the provider’s storage.

External Recipient (Non-Databricks)

#

For consumers outside Databricks (pandas, Spark OSS, Power BI):

1
2
3
4
5
6
7
8

-- Create an external recipient
CREATE RECIPIENT IF NOT EXISTS acme_consulting
COMMENT 'External analytics partner';

-- This generates an activation link
-- Run immediately after creating the recipient:
DESCRIBE RECIPIENT acme_consulting;
-- Look for the activation_link column

The activation link is a one-time URL. The recipient opens it, downloads a credentials file (.share profile), and uses it with any Delta Sharing client. The profile file contains a bearer token and the sharing server endpoint – no storage credentials.

1
2

-- Grant access to the external recipient
GRANT SELECT ON SHARE analytics_share TO RECIPIENT acme_consulting;

Step 4: Consume from Outside Databricks

#

Python / pandas

#

Install the Delta Sharing Python connector:

1

pip install delta-sharing

Read shared tables:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import delta_sharing

# Load the profile from the downloaded .share file
profile_file = "config.share"

# List available shares and tables
shares = delta_sharing.SharingClient(profile_file).list_all_tables()
for table in shares:
    print(f"{table.share}.{table.schema}.{table.name}")

# Read a table as a pandas DataFrame
table_url = f"{profile_file}#analytics_share.gold.customer_360"
df = delta_sharing.load_as_pandas(table_url)
print(df.head())

1
2
3
4

  customer_id           email    tier  region  lifetime_value last_order_date
0        C001  [email protected]    gold  US-WEST       45000.0      2026-03-15
1        C002    [email protected]  silver  EU-WEST       12000.0      2026-04-01
2        C003  carol@startup  platinum    APAC       89000.0      2026-04-10

Spark OSS

#

1
2
3
4
5
6

# Read from Spark (non-Databricks)
df = (
    spark.read.format("deltaSharing")
    .load("config.share#analytics_share.gold.customer_360")
)
df.show()

Incremental Reads with CDF

#

If the shared table has Change Data Feed enabled, consumers can read only what changed:

1
2
3
4
5
6
7

# Read changes since version 3
changes = delta_sharing.load_table_changes_as_pandas(
    table_url,
    starting_version=3,
)
print(changes[["customer_id", "_change_type",
               "_commit_version"]])

This is how you build efficient downstream pipelines over shared data. The consumer doesn’t need to diff the full table – they get the exact rows that were inserted, updated, or deleted.

Step 5: Security Controls

#

IP Access Lists

#

Restrict which networks can access your shares. IP access lists for recipients are managed via the Databricks SDK or REST API (there is no SQL syntax for this):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()

w.recipients.update(
    name="acme_consulting",
    ip_access_list={
        "allowed_ip_addresses": ["203.0.113.0/24", "198.51.100.0/24"]
    }
)

Token Rotation

#

Recipient tokens should be rotated periodically. The recipient re-activates with a new link:

1
2
3
4
5

-- Rotate the activation token
ALTER RECIPIENT acme_consulting ROTATE TOKEN;

-- New activation link appears in:
DESCRIBE RECIPIENT acme_consulting;

The old token stops working immediately. Coordinate with the recipient before rotating.

Table-Level vs Schema-Level Grants

#

Schema-level sharing (ADD SCHEMA) includes all current and future tables. If a sensitive table is added to the schema later, it’s automatically shared. For most production use cases, table-level grants give you tighter control:

1
2
3
4
5
6

-- Explicit is better than implicit
ALTER SHARE analytics_share
ADD TABLE analytics.gold.customer_360;

-- NOT:
-- ALTER SHARE analytics_share ADD SCHEMA analytics.gold;

Step 6: Audit Who Accessed What

#

Unity Catalog logs every Delta Sharing query in the system tables:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

-- Who queried shared tables, and when?
SELECT
    request_params.recipient_name,
    request_params.share,
    request_params.schema,
    request_params.name AS table_name,
    user_identity.email AS requester,
    event_time,
    response.status_code
FROM system.access.audit
WHERE action_name = 'deltaSharingQueryTable'
ORDER BY event_time DESC
LIMIT 50

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

-- Which recipients are most active?
SELECT
    request_params.recipient_name,
    COUNT(*) AS query_count,
    MIN(event_time) AS first_query,
    MAX(event_time) AS last_query
FROM system.access.audit
WHERE action_name = 'deltaSharingQueryTable'
  AND event_time >= current_date() - 30
GROUP BY request_params.recipient_name
ORDER BY query_count DESC

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

-- Detect unusual access patterns
SELECT
    request_params.recipient_name,
    DATE(event_time) AS query_date,
    COUNT(*) AS daily_queries
FROM system.access.audit
WHERE action_name = 'deltaSharingQueryTable'
GROUP BY request_params.recipient_name, DATE(event_time)
HAVING COUNT(*) > 100  -- flag recipients hitting tables excessively
ORDER BY daily_queries DESC

This audit trail is how you answer compliance questions: “Who accessed customer data last month? When? How many times?” All without building custom logging infrastructure.

Gotchas

#

You can’t share views. Delta Sharing operates on physical Delta tables. If you need to share a filtered or transformed dataset, materialize it as a table first. This is a common stumbling block for teams that use views for access control – views don’t travel through shares.

Column masking interaction. If a table has column masks (from Unity Catalog fine-grained access control), the mask is applied at the provider side. The recipient sees masked values. This is correct behavior, but it means you need to think about which mask applies – the provider’s mask policy, not the recipient’s.

Sharing across clouds. Delta Sharing works across cloud providers (AWS to Azure, for example), but the latency depends on the data size and network path. For large tables, the first query can be slow because data is read from the provider’s storage. Incremental CDF reads mitigate this for ongoing pipelines.

Recipient token expiry. External recipient tokens don’t expire by default, which is a security concern. Set up a rotation schedule (quarterly is reasonable) and track activation status:

1
2
3
4
5

-- Check recipient status
SHOW RECIPIENTS;
-- Or use the SDK to list recipients and check activation status:
-- w.recipients.list() returns RecipientInfo objects with
-- activated, activation_url, created_at, updated_at fields

Schema evolution. If the provider adds a column to a shared table, the consumer sees it on the next query. If the provider removes a column, consumers referencing it break. Communicate schema changes to recipients before deploying them – there’s no built-in schema compatibility negotiation in the protocol.

Conclusion

#

Delta Sharing replaces file extracts, credential sharing, and one-off ETL pipelines with a governed, auditable protocol. The provider controls what’s shared and can revoke access instantly. The consumer reads live data without managing storage credentials. The audit trail proves who accessed what and when.

The open protocol means this isn’t a Databricks-only solution. External consumers use the same pandas or Spark interface they already know. The .share profile file is the only credential they need, and it contains no storage keys.

For large-scale data exchange – between internal teams, across workspaces, with external partners – this is how you avoid the “export to S3 and pray” pattern. Zero copies, full governance, real-time access to live tables.