โWhat were our earnings last quarter?โ If you get different answers, the KPI is not governed.
Metric views define a KPI once in Unity Catalog so humans and agents report the same value.
This post shows how to register a metric view as a Unity Catalog securable, add agent metadata, and query it with MEASURE in @ApacheSpark.
Check it out ๐
unitycatalog.io/blogs/uc-metโฆ
#UnityCatalog #MetricViews #DataGovernance
๐ฃ Join us for the next Unity Catalog Community Sync!
Weโll cover what is in flight for the upcoming Unity Catalog v0.8.0 release, including local S3 support, walk through the new UI preview, and open the floor for questions with maintainers and the community.
๐๏ธ Tuesday, October 20
๐ 8:00 AM PT
๐ Register: luma.com/gy8naenq
#UnityCatalog #OpenLakehouse #OpenSource
Native @ApacheIceberg writes are in Unity Catalog 0.7.0! ๐ง
Until now, Iceberg support in Unity Catalog was read-only and needed Delta UniForm to generate Iceberg metadata. 0.7.0 adds the write side of the Iceberg REST catalog, so you can create, commit to, rename, and drop native Iceberg tables. No UniForm required.
This post walks through it:
๐น Spark and PyIceberg reading and writing the same managed tables
๐ The same USE CATALOG, USE SCHEMA, and SELECT grants across engines
๐ป A local setup with unitycatalog-playground v0.2.0
Learn more: unitycatalog.io/blogs/unity-โฆ
#UnityCatalog #ApacheIceberg
Reminder: The Unity Catalog Community Sync is tomorrow! ๐จ
Join us for a recap of v0.7, a sneak peek at the v0.8 roadmap, and live Q&A with the maintainers.
๐
Tomorrow, Oct 6
๐ 8:00 AM PT / 17:00 CEST
๐ Virtual
RSVP here: luma.com/uc-sync1
#UnityCatalog #AI #DataGovernance
Join us for next weekโs Unity Catalog Community Sync: v0.7 recap, a first look at v0.8, and Q&A with the maintainers!
Weโll walk through what shipped in v0.7, share the v0.8 roadmap, and open the floor for questions and feedback.
๐
Tues, Oct 6
๐ 8:00 AM PT / 17:00 CEST
๐ Virtual
Register ๐ luma.com/uc-sync1
#UnityCatalog #DataGovernance #AI
Control, choice, and cost in an open lakehouse for AI agents. ๐
This conversation explores Unity Catalog as an open metadata center across @ApacheSpark, @DeltaLakeOSS, and @ApacheIceberg.
๐ Fine-grained access for humans and agents
๐ Open standards for swapping engines and formats
๐ฐ Compute, token, and iteration costs with MLflow
๐ฅ Watch: youtube.com/watch?v=BsxGjbNWโฆ
#OpenLakehouse #UnityCatalog #ApacheIceberg
New Open Lakehouse + AI episode now available! Stop shipping 3 versions of annual revenue. ๐
Scott Haines and Lisa Cao on metric views in Spark 4.2.
๐ One YAML KPI. Catalog serves it globally.
๐ค Same number for dashboards, agents, and AI/BI.
๐ Native interop with Unity Catalog.
๐ฅ youtu.be/JJkkOBLPge4
#openlakehouse #unitycatalog #apachespark
Ask three teams for last quarterโs earnings. Get three numbers.
Metric views in Unity Catalog define the KPI once (YAML: source, joins, filters, dimensions, measures). Humans and agents query the same number. Spark uses MEASURE().
Try it on Unity Catalog 0.6 with Apache Spark 4.3.
๐ Learn more: unitycatalog.io/blogs/uc-metโฆ
#UnityCatalog #MetricViews #ApacheSpark #OpenSource
Introducing the UC Delta API in Unity Catalog 0.5 ๐
๐น Versioned, discoverable, atomic REST surface.
๐น Native Delta wire format and intent-based commits.
๐น Run it locally with Spark or DuckDB.
Read the full post: unitycatalog.io/blogs/unity-โฆ
#UnityCatalog #DeltaLake
[Ep 1] Open Lakehouse + AI: The Catalog Layer, Interoperability & AI-Agent Governance
Learn how composable lakehouses center the catalog for metadata, versioning, and commit coordination. ๐
๐ธ Interop: Iceberg REST, Unity Catalog (OSS), Spark, Delta Lake, Delta-RS, Iceberg.
๐ธ Governance: credential vending, row/col filters, column masks, audits, and why โno governance was the easiest governance.โ
๐ธ Agents as data customers (Temporal). Fine-grained access beats blanket credentials on 24/7 workloads.
๐ฅ Full episode: youtube.com/watch?v=dEFAkS7vโฆ
#OpenLakehouse #UnityCatalog @ApacheSpark @ApacheIceberg @DeltaLakeOSS
Simplified Governance with Catalog-Managed Tables ๐
Unity Catalog 0.4.0 introduces support for UC managed tables, enabling data teams to centrally govern, discover, access, and audit their data through Unity Catalog. Instead of relying on scattered storage paths, separate credentials, and manual maintenance, teams can rely on Unity Catalog as the single logical system of record for their data estate.
Leverage UC managed tables to strengthen governance, improve performance, and build on the most modern open catalog for the data and AI era.
๐ Check out the announcement and implementation details on the Unity Catalog blog: lnkd.in/eZWkMBaR
#unitycatalog #governance #opensource #catalogs #deltalake
With UC managed tables, teams unlock:
๐ธ ๐จ๐ป๐ถ๐ณ๐ถ๐ฒ๐ฑ ๐ด๐ผ๐๐ฒ๐ฟ๐ป๐ฎ๐ป๐ฐ๐ฒ:ย Unity Catalog centralizes access control, replacing fragmented storage-level policies. This simplifies how teams ensure all engines access data in a governed, consistent manner.
๐ธ ๐ฆ๐๐ฎ๐ป๐ฑ๐ฎ๐ฟ๐ฑ๐ถ๐๐ฒ๐ฑ ๐ฑ๐ถ๐๐ฐ๐ผ๐๐ฒ๐ฟ๐: Unity Catalog provides stable logical table identifiers, eliminating the need for clients to depend on physical storage paths for discovery.
๐ธ ๐๐ณ๐ณ๐ผ๐ฟ๐๐น๐ฒ๐๐ ๐๐ฎ๐ฏ๐น๐ฒ ๐ผ๐ฝ๐๐ถ๐บ๐ถ๐๐ฎ๐๐ถ๐ผ๐ป๐: By automating storage tuning and credential management, Unity Catalog removes the burden of manual operational maintenance from data teams.
๐ธ ๐๐ผ๐น๐ถ๐๐๐ถ๐ฐ ๐ฎ๐๐ฑ๐ถ๐๐ฎ๐ฏ๐ถ๐น๐ถ๐๐:ย Metadata and permissions are centralized in a single interface, allowing for high-level oversight of ownership and access instead of parsing low-level storage logs.
๐ธ ๐๐ป๐ณ๐ผ๐ฟ๐ฐ๐ฒ๐ฎ๐ฏ๐น๐ฒ ๐ฐ๐ผ๐ป๐๐๐ฟ๐ฎ๐ถ๐ป๐๐: Unity Catalog can authoritatively validate or reject schema and constraint changes, preventing incompatible updates that could compromise data integrity or break downstream workloads.
๐ธ ๐๐ฎ๐๐๐ฒ๐ฟ ๐พ๐๐ฒ๐ฟ๐ ๐ฝ๐น๐ฎ๐ป๐ป๐ถ๐ป๐ด ๐ฎ๐ป๐ฑ ๐ณ๐ฎ๐๐๐ฒ๐ฟ ๐๐ฟ๐ถ๐๐ฒ๐: Unity Catalog delivers table metadata directly to Delta clients, bypassing cloud storage requests to significantly reduce metadata latency and accelerate query planning and writes.
We are excited to announce ๐จ๐ป๐ถ๐๐ ๐๐ฎ๐๐ฎ๐น๐ผ๐ด ๐ฌ.๐ฐ.๐ฌ which includes exciting new features and many bug-fixes and improvements! ๐
Check out some of the highlights ๐
๐จ๐ ๐ฆ๐ฒ๐ฟ๐๐ฒ๐ฟ
โ๏ธ Storage Credentials for AWS
โ๏ธ External Locations for AWS
โ๏ธ Managed storage location for catalogs and schemas
๐จ๐ ๐ฆ๐ฝ๐ฎ๐ฟ๐ธ ๐๐ผ๐ป๐ป๐ฒ๐ฐ๐๐ผ๐ฟ
โก Credential Renewal Enabled by Default
โก Support for Spark 4.1 and Delta 4.1
โก Atomic CTAS for Delta Tables in UCSingleCatalog
๐จ๐ ๐๐
๐ค DSPY Integration with AI Functions
A huge thank you to the awesome community who made this release possible!
๐ Full release notes: github.com/unitycatalog/unitโฆ
#unitycatalog #opensource #oss
Weโre excited to announce the release of Unity Catalog v0.3.1!ย ๐ This release includes exciting new features and many bug-fixes and improvements.
Version 0.3.1 focuses on three major areas:
๐น ๐๐บ๐ฝ๐ฟ๐ผ๐๐ฒ๐ฑ ๐๐ฎ๐๐ฎ ๐ฐ๐น๐ถ๐ฒ๐ป๐ ๐๐ฃ๐ ๐๐ถ๐๐ต ๐ข๐๐๐๐ต ๐๐๐ฝ๐ฝ๐ผ๐ฟ๐: designed for reliability and extensibility in production environments.
๐น ๐๐๐๐ผ๐บ๐ฎ๐๐ถ๐ฐ ๐ฐ๐ฟ๐ฒ๐ฑ๐ฒ๐ป๐๐ถ๐ฎ๐น ๐ฟ๐ฒ๐ป๐ฒ๐๐ฎ๐น: to support long-running workloads cross cloud platforms.
๐น ๐จ๐-๐บ๐ฎ๐ป๐ฎ๐ด๐ฒ๐ฑ ๐๐ฒ๐น๐๐ฎ ๐๐ฎ๐ฏ๐น๐ฒ๐: this enables Unity Catalog to coordinate table storage and commits centrally.
This release is the result of contributions from our growing open-source community. A big thank-you to everyone who reported issues, submitted pull requests, reviewed code, and shared feedback!
๐ Dive into the release notes for the full list of highlights:ย github.com/unitycatalog/unitโฆ
#unitycatalog #opensource #oss #catalog
Managing data pipelines at scale is complicated, often resulting in the data silo problemโwhere valuable assets are spread across systems. This makes it difficult to track, secure access, and scale cleanly.
The solution is a clear structure paired with centralized governance. โ
The Medallion Architecture structures your data into three distinct layers:
๐ฅ ๐๐ฟ๐ผ๐ป๐๐ฒ: Raw, ingested data.
๐ฅ ๐ฆ๐ถ๐น๐๐ฒ๐ฟ: Cleaned, enriched data.
๐ฅ ๐๐ผ๐น๐ฑ: Business-level data, ready for reporting.
Pair this framework with Unity Catalog, and you get a unified system to manage, govern, and organize your entire data flow.
๐ Walk through how this works: unitycatalog.io/blogs/buildiโฆ
#UnityCatalog #MedallionArchitecture #DataGovernance #OpenSource
โก๏ธ From vector DB to multimodal lakehouse at petabyte scale. Join us in Mountain View on Nov 13 for โScaling Multimodal AI Lakehouse with Lance & LanceDBโ with Chang She (@lancedb) at the Open Lakehouse + AI Mini Summit!
Learn:
๐น Lowโlatency random access + search APIs (vectors, text, binaries)
๐น Schema primitives across blobs + metadata for feature engineering
๐น Hybrid search (vector + fullโtext) for training/fineโtuning at scale
Two exciting tracks, one epic afternoon. ๐
#OpenLakehouse #AI #LanceDB #VectorDB #DataEngineering
Unity Catalog retweeted
Join us at Open Lakehouse + AI Paris on November 24, 6:30โ10PM โ co-located with the Forward Data Conference! ๐ซ๐ท
Secure your spot now โฌ๏ธ luma.com/OLM-1124
Weโre bringing together data innovators and open source contributors for an evening packed with insight and inspiration. Hear talks from:
โ
Alexandre BERGERE (@DataGalaxy / Datalex) on Building a Scalable Usage Insights Platform with Delta Sharing
โ
Bartosz Konieczny (@waitingforcode) on Design Patterns for the Open Lakehouse
โ
Youssef Mrini & El Ghali Benchekroun (@Databricks) on The Future of Open Table Formats & @unitycatalog_io
Food, drinks, networking โ and plenty of new ideas (& swag) to take home. ๐
#opensource #deltalake #oss #apacheiceberg #unitycatalog #lakehouse #openlakehouse #ai
Missed it liveโSee how an observabilityโfirst Telemetry Lake correlates data movement with system behavior to detect issues, diagnose root causes, and adapt in real timeโwhy correlation > coverage and why the first ~150 characters of signal matter.
Watch: youtube.com/watch?v=r04iCvm6โฆ
Whatโs inside: @OpenLineage + @opentelemetry + an LLM reasoning layer that turns noisy lineage and traces into prioritized actions to cut TTD/TTR and reduce blast radius.
#openlakehouse #datalineage #observability #AI
๐ค @wslulciuc (oleander), hosted by Lisa N. Cao (@Databricks)
#openlakehouse #datalineage #observability #oleander #AI #opensource

















