Unity Catalog Deep Dive: Fine-Grained Access Control

If you’ve ever had a data engineer ask “can I just give everyone SELECT on everything?” — this article is your ammunition for saying no.

Unity Catalog (UC) isn’t just a metastore replacement. It’s Databricks’ answer to the question every enterprise data platform eventually faces: who can see what, and how do we prove it?

In this deep dive, we’ll go beyond the basics and explore fine-grained access control — column masking, row-level security, and the privilege model that ties it all together. With real SQL you can run today, not just slides from a vendor pitch.

Introduction: Why Access Control Matters More Than You Think

#

Most data breaches don’t happen because someone hacked through a firewall. They happen because someone had access they shouldn’t have had. An analyst querying PII they didn’t need. A service account with ALL PRIVILEGES because “it was easier during development.”

Unity Catalog centralizes governance across all Databricks workspaces. One place for access policies. One audit trail. No more per-workspace ACLs that drift apart like continents.

Prerequisites

#

Before you start, make sure you have:

• A Databricks workspace with Unity Catalog enabled
• Account admin or metastore admin privileges (for initial setup)
• A Unity Catalog metastore attached to your workspace
• Basic familiarity with SQL GRANT/REVOKE statements
• At least one catalog and schema created (we’ll create sample tables below)

The UC Privilege Model: Three-Level Namespace

#

Unity Catalog organizes everything into a three-level namespace:

1

catalog.schema.table

This isn’t just naming convention — it’s the security boundary hierarchy. Privileges cascade downward, and each level acts as a permission checkpoint.

flowchart TD
    subgraph Metastore["Metastore (Account Level)"]
        direction TB
        MS["Metastore Admin"]
    end

    subgraph Catalog["Catalog: production"]
        direction TB
        CO["Catalog Owner"]
    end

    subgraph Schema["Schema: production.finance"]
        direction TB
        SO["Schema Owner"]
    end

    subgraph Objects["Tables / Views / Functions"]
        direction TB
        T1["invoices"]
        T2["payments"]
        T3["customers"]
    end

    MS --> CO
    CO --> SO
    SO --> T1
    SO --> T2
    SO --> T3

    style MS fill:#e74c3c,color:white
    style CO fill:#e67e22,color:white
    style SO fill:#f1c40f,color:black
    style T1 fill:#2ecc71,color:white
    style T2 fill:#2ecc71,color:white
    style T3 fill:#2ecc71,color:white

Securable Objects

#

UC governs access to these objects:

Setting Up: A Realistic Example

#

Let’s build a scenario that mirrors a real enterprise. We have a production catalog with a finance schema containing sensitive customer payment data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

-- Create the catalog and schema
CREATE CATALOG IF NOT EXISTS production;
CREATE SCHEMA IF NOT EXISTS production.finance;

-- Create a customers table with sensitive columns
CREATE TABLE IF NOT EXISTS production.finance.customers (
    customer_id STRING,
    full_name STRING,
    email STRING,
    phone STRING,
    ssn STRING,
    credit_score INT,
    region STRING,
    created_at TIMESTAMP
);

-- Insert sample data
INSERT INTO production.finance.customers VALUES
    ('C001', 'Alice Johnson', '[email protected]', '+1-555-0101', '123-45-6789', 780, 'US-WEST', current_timestamp()),
    ('C002', 'Bob Smith', '[email protected]', '+1-555-0102', '987-65-4321', 650, 'US-EAST', current_timestamp()),
    ('C003', 'Carol Martinez', '[email protected]', '+1-555-0103', '456-78-9012', 720, 'EU-WEST', current_timestamp()),
    ('C004', 'Dave Chen', '[email protected]', '+1-555-0104', '321-54-9876', 800, 'APAC', current_timestamp());

Now let’s lock it down properly.

Catalog & Schema-Level Access

#

The first layer of defense is controlling who can even see that a catalog or schema exists.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

-- Create groups for different teams
-- (In production, sync these from your IdP via SCIM)

-- Grant the analytics team access to the catalog
GRANT USE CATALOG ON CATALOG production TO `analytics-team`;

-- Grant access to the finance schema
GRANT USE SCHEMA ON SCHEMA production.finance TO `analytics-team`;

-- Grant SELECT on specific tables
GRANT SELECT ON TABLE production.finance.customers TO `analytics-team`;

Without USE CATALOG, a user can’t even list schemas. Without USE SCHEMA, they can’t list tables. This is deny-by-default — if you don’t explicitly grant, access doesn’t exist.

1
2

-- Verify grants
SHOW GRANTS ON TABLE production.finance.customers;

The Ownership Model

#

Every securable object has an owner. Owners have implicit ALL PRIVILEGES on their objects. This is powerful but dangerous if you’re not careful:

1
2
3

-- Transfer ownership to a service principal (recommended for production)
ALTER TABLE production.finance.customers
SET OWNER TO `finance-data-platform-sp`;

Column Masking: Hiding Sensitive Data In Place

#

This is where it gets interesting. Column masking lets you define functions that transform sensitive column values based on who’s querying. The data stays intact in storage — the masking happens at query time.

Step 1: Create a Masking Function

#

1
2
3
4
5
6
7
8

-- Mask SSN: show full value to finance-admins, redacted for everyone else
CREATE OR REPLACE FUNCTION production.finance.mask_ssn(ssn STRING)
RETURNS STRING
RETURN
  CASE
    WHEN is_account_group_member('finance-admins') THEN ssn
    ELSE CONCAT('***-**-', RIGHT(ssn, 4))
  END;

Step 2: Apply the Mask to the Column

#

1
2

ALTER TABLE production.finance.customers
ALTER COLUMN ssn SET MASK production.finance.mask_ssn;

What Users See

#

Finance admin queries the table:

1
2
3
4

| customer_id | full_name       | ssn         |
|-------------|-----------------|-------------|
| C001        | Alice Johnson   | 123-45-6789 |
| C002        | Bob Smith       | 987-65-4321 |

Analytics team member queries the same table:

1
2
3
4

| customer_id | full_name       | ssn         |
|-------------|-----------------|-------------|
| C001        | Alice Johnson   | ***-**-6789 |
| C002        | Bob Smith       | ***-**-4321 |

Same table. Same query. Different results based on identity. No separate views to maintain. No ETL pipelines to strip PII. The security boundary lives with the data.

More Masking Patterns

#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

-- Email masking: show domain only
CREATE OR REPLACE FUNCTION production.finance.mask_email(email STRING)
RETURNS STRING
RETURN
  CASE
    WHEN is_account_group_member('finance-admins') THEN email
    ELSE CONCAT('****@', SPLIT(email, '@')[1])
  END;

-- Credit score: show range instead of exact value
CREATE OR REPLACE FUNCTION production.finance.mask_credit_score(score INT)
RETURNS STRING
RETURN
  CASE
    WHEN is_account_group_member('finance-admins') THEN CAST(score AS STRING)
    WHEN score >= 750 THEN 'Excellent'
    WHEN score >= 700 THEN 'Good'
    WHEN score >= 650 THEN 'Fair'
    ELSE 'Poor'
  END;

-- Apply both masks
ALTER TABLE production.finance.customers
ALTER COLUMN email SET MASK production.finance.mask_email;

ALTER TABLE production.finance.customers
ALTER COLUMN credit_score SET MASK production.finance.mask_credit_score;

Row-Level Security: Filtering What You Can’t See

#

Column masking hides values. Row-level security hides entire rows. If a user shouldn’t know a customer exists in a certain region — they simply don’t see that row.

Step 1: Create a Row Filter Function

#

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

-- Regional managers only see customers in their region
CREATE OR REPLACE FUNCTION production.finance.region_filter(region STRING)
RETURNS BOOLEAN
RETURN
  CASE
    WHEN is_account_group_member('finance-admins') THEN TRUE
    WHEN is_account_group_member('us-west-team') AND region = 'US-WEST' THEN TRUE
    WHEN is_account_group_member('us-east-team') AND region = 'US-EAST' THEN TRUE
    WHEN is_account_group_member('eu-team') AND region = 'EU-WEST' THEN TRUE
    WHEN is_account_group_member('apac-team') AND region = 'APAC' THEN TRUE
    ELSE FALSE
  END;

Step 2: Apply the Row Filter

#

1
2

ALTER TABLE production.finance.customers
SET ROW FILTER production.finance.region_filter ON (region);

The Result

#

A us-west-team member runs SELECT * FROM production.finance.customers:

1
2
3

| customer_id | full_name     | region  |
|-------------|---------------|---------|
| C001        | Alice Johnson | US-WEST |

They get one row. Not because they ran a WHERE clause — because the platform enforced it. The other three rows don’t exist in their view of the world.

The Security Architecture: How It All Fits Together

#

flowchart LR
    subgraph Identity["Identity Layer"]
        IDP["IdP (Entra ID / Okta)"]
        SCIM["SCIM Sync"]
        Groups["UC Groups"]
        IDP --> SCIM --> Groups
    end

    subgraph Governance["Unity Catalog Governance"]
        direction TB
        Privs["Privilege Grants\n(GRANT/REVOKE)"]
        ColMask["Column Masks\n(UDF-based)"]
        RowFilter["Row Filters\n(UDF-based)"]
        Audit["Audit Logs\n(System Tables)"]
    end

    subgraph Data["Data Layer"]
        direction TB
        Tables["Delta Tables"]
        ExtLoc["External Locations\n(S3/ADLS/GCS)"]
        StorCred["Storage Credentials"]
    end

    subgraph Compute["Compute Layer"]
        SQL["SQL Warehouse"]
        Cluster["All-Purpose Cluster"]
        Job["Job Cluster"]
        Serverless["Serverless"]
    end

    Groups --> Privs
    Groups --> ColMask
    Groups --> RowFilter
    Privs --> Tables
    ColMask --> Tables
    RowFilter --> Tables
    Tables --> ExtLoc
    ExtLoc --> StorCred
    Compute --> Governance

    style IDP fill:#3498db,color:white
    style Groups fill:#3498db,color:white
    style Privs fill:#e74c3c,color:white
    style ColMask fill:#e74c3c,color:white
    style RowFilter fill:#e74c3c,color:white
    style Audit fill:#e74c3c,color:white

The key insight: every compute type goes through Unity Catalog. SQL Warehouses, interactive clusters, jobs, serverless — they all enforce the same policies. No backdoors.

Auditing: Proving You Did It Right

#

Access control without auditing is like a lock without a camera. Unity Catalog logs every access event to system tables:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

-- Who accessed the customers table in the last 7 days?
SELECT
    event_date,
    user_identity.email AS user_email,
    action_name,
    request_params.full_name_arg AS table_name,
    source_ip_address
FROM system.access.audit
WHERE request_params.full_name_arg = 'production.finance.customers'
  AND event_date >= current_date() - INTERVAL 7 DAYS
ORDER BY event_date DESC;

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

-- Show all GRANT/REVOKE operations for compliance reporting
SELECT
    event_date,
    user_identity.email,
    action_name,
    request_params
FROM system.access.audit
WHERE action_name IN ('grantPermission', 'revokePermission')
  AND event_date >= current_date() - INTERVAL 30 DAYS
ORDER BY event_date DESC;

This is your compliance paper trail. SOC 2, GDPR, HIPAA — auditors love receipts.

Key Takeaways

#

• Deny-by-default: Unity Catalog grants no access unless explicitly given. USE CATALOG and USE SCHEMA are your first gates.
• Column masking transforms sensitive values at query time using SQL functions — no separate views or ETL pipelines needed.
• Row-level security filters rows based on user identity, enforced by the platform regardless of how the query is written.
• All compute types respect UC policies — there’s no way to bypass governance by switching to a different cluster type.
• System tables provide a complete audit trail for compliance and incident response.

Next Steps

#

• Implement attribute-based access control (ABAC): Combine UC groups with workspace tags for dynamic policies.
• Explore Delta Sharing: Extend UC governance to external data sharing across organizations.
• Set up automated compliance reports: Schedule queries against system.access.audit for weekly governance summaries.
• Read the next article in the series: We’ll tackle Delta Lake performance tuning — Z-Order, Liquid Clustering, and Optimize.

Unity Catalog doesn’t just protect your data — it lets you prove it’s protected. And in a world where regulators have better lawyers than your company, that proof is worth its weight in gold.