A data lakehouse makes database-style operations possible on object storage. Most lakehouse evaluations spend their time on the query engine. The storage layer gets a throughput number and a price per terabyte. That order is backwards. The storage layer is the foundational layer of the lakehouse.
Written for data leaders and platform architects choosing the storage layer under a lakehouse, this guide covers how the layers divide, which decisions belong to storage, six questions to ask before committing, and the mistakes that surface after the contract is signed. It is not a table-format comparison. For that, see What Is an Open Table Format, and for one of the formats itself, What Is Apache Iceberg.
A plain data lake is a set of files in buckets. A pile of files. Engines list directories, infer schemas, and coordinate nothing. A lakehouse adds a metadata layer that turns those files into tables with transactions, schema evolution, and time travel. The lakehouse stack has three main components:
For many lakehouse deployments there is a fourth component: the catalog. Apache Iceberg in particular has a catalog as a required, central part of its architecture. The catalog's job is to store a pointer to the current table metadata file, and every reader or writer must go through the catalog to find that pointer. Where the catalog is deployed, inside the storage platform or as a standalone component, is a deployment decision.
For a lakehouse on object storage, compute stays separate from storage by definition. Query Engines scale independently of storage capacity allowing several engines read the same tables and for a team can replace an engine without migrating data
The question buyers are somewhat nuanced different. A lakehouse still depends on services that run next to the data rather than inside the query engine:
None of that is query compute, and all of it has to run somewhere. In many deployments each item is a separate scheduled job, owned by a data engineering team, running on the same cluster as production queries. Some newer catalogs and managed table services now run parts of this maintenance automatically, which proves the point: the work moved, it did not disappear.
Decoupled compute also does not mean remote for every read. Engines cache hot data on local disk or in memory. The storage layer still has to serve the cold path and the metadata path at speed, because caches do not help the first scan or the planning step.
As mentioned, every Iceberg table has a current metadata file, and the catalog stores the pointer to it. When an engine commits, it writes a new metadata file and asks the catalog to swap the pointer atomically. If another writer committed first, the swap fails and the engine retries against the new state. That single operation gives a lakehouse database-grade concurrency on top of immutable files. The catalog also holds namespaces, exposes views, and is the natural point to enforce who can see which table.
Modern Iceberg catalogs, embedded or standalone, converge on the same interface: the Iceberg REST Catalog API. Engines connect the same way regardless of what runs behind it. REST is the constant, not the differentiator. The differentiator is where the service runs and who operates it.
A standalone catalog is the right choice in three situations. Tables live across several object stores or clouds and need one namespace. The organization has already standardized governance on an existing catalog and wants every table registered there. Or a business catalog for lineage, ownership, and glossary terms needs a platform catalog underneath it to federate the technical catalogs. Open-source options such as Apache Polaris, Project Nessie, Lakekeeper, and Apache Gravitino fill that role, as do managed catalogs from cloud and platform vendors. In each case the embedded catalog still serves as the technical catalog for its platform, federated upward.
Everywhere else, the embedded model removes a component, a failure domain, and a second security model from the architecture.
AIStor Tables as a worked example. MinIO AIStor Tables supports the Iceberg REST Catalog API. Iceberg Warehouses, namespaces, and tables are storage primitives with ACID commits, schema evolution, time travel, and atomic commits across multiple tables, and namespaces carry their own access policies. Any engine that implements the Iceberg REST Catalog API connects without modification.
1. Which formats and engines can read and write our data, and what does leaving cost?
Openness is the reason to build a lakehouse at all. Check that the platform exposes standard interfaces, the S3 API for objects and the Iceberg REST Catalog API for tables, with no proprietary client library or wrapper in the path. Check that table metadata stays in the open format's own files rather than in a private database. Then ask what leaving looks like. If the answer is "point another engine at the same buckets," the data is yours. If the answer involves an export job, the platform owns it.
2. Does performance hold on the metadata path, not only on bulk throughput?
Before an engine reads a byte of table data, it reads the table metadata file, the manifest list, and every manifest that matches its filter. Those are many small reads, so planning time depends on small-object latency and concurrency, not streaming throughput. Frequent commits also concentrate load on the catalog. AIStors analysis of the Iceberg catalog API makes the point directly: for most operations, including queries, ingest, and compaction, performance is primarily determined by the object storage underneath. Test planning time on a table with hundreds of snapshots and thousands of manifests, small-object latency at concurrency, and commit latency with several writers active. A large-sequential-read benchmark answers none of those questions.
3. Does the platform provide the consistency and atomic operations table formats depend on?
An Iceberg commit is safe only if the catalog swap is atomic and the store returns exactly what was written, immediately. On an eventually consistent store, an engine can be told by committed metadata that a file exists and then be served "not found," or read a stale copy of an overwritten metadata file. A non-atomic commit path lets two writers both believe they won. Test with two engines writing to one table at the same time, then confirm the snapshot history is one linear chain with no orphaned data files, and confirm that reads from a different client immediately reflect each write.
4. How are the data and the catalog state protected?
For objects, the checklist is familiar: erasure coding, bit-rot detection, versioning, and, for regulated data, object immutability with retention and legal hold. The part evaluations miss is the catalog's state. A table is recoverable only if both its files and the pointer to its current metadata survive. A standalone catalog's database needs its own backup, and it must be restored to a point consistent with the data or tables come back pointing at snapshots that no longer exist. Run a restore drill that brings back tables, not just buckets, and confirm time travel still works afterward.
5. How granular is access control, and how does it fit our governance?
Bucket-level permissions are not table-level permissions. Look for access control at the warehouse, namespace, and table level, audit logs on catalog operations, and short-lived, scoped credentials handed to engines instead of shared keys. Then look at how the technical catalog surfaces to the business catalog the organization already uses for ownership and lineage. Test by granting one identity read access to one namespace, confirming that both the catalog and the object path enforce it, and confirming the access appears in the audit log.
6. What does our team have to run, where can it run, and what drives the cost?
List the components: the object store, the catalog service and its database, the maintenance jobs, replication tooling, and any metastore left over from an earlier architecture. Each one is a version to patch, a failure domain to monitor, and a role to staff. Then confirm where the platform runs: on-premises, private cloud, sovereign environments, hybrid, and on hardware the organization chooses. Finally, separate the cost drivers: capacity pricing versus per-operation and egress fees, duplicate copies held across systems, component count, and staff time. Ask for the derivation behind any savings figure a vendor presents. A number that cannot be traced to its inputs does not belong in a business case.
Two access patterns are often described with one phrase, and they work differently.
Inside the organization, the goal is one copy of the data reached by any engine that implements the Iceberg REST Catalog API: Spark, Trino, Flink, Dremio, Starburst, PyIceberg, and others. A shared catalog is what makes it work. Every engine plans against the same current snapshot, and a commit from one engine is visible to the rest on their next read.
Across organizational boundaries, the pattern is the open Delta Sharing protocol. A provider publishes a share, a recipient authenticates with a token, and the sharing server returns either short-lived pre-signed URLs to the data files or, in the protocol's newer directory-based mode, the table's storage location with temporary scoped credentials. In both modes the recipient never holds the provider's standing storage credentials, and access is read-only. MinIO AIStor Table Sharing is a native implementation of the protocol. It shares Delta tables directly and Iceberg tables through Delta Universal Format, and it returns pre-signed URLs signed with a short-lived credential scoped to the one table being queried. Consumers query shared tables; they do not write to them, and sharing does not replace the catalog for internal access.
Common Evaluation Mistakes to Avoid
Does a data lakehouse need compute built into the storage layer, or can it be decoupled?
Query compute is decoupled from storage in the mainstream lakehouse architectures, and it should stay that way. The catalog service and the table maintenance a lakehouse depends on still have to run somewhere, and deciding where is the real architectural choice. An embedded catalog reduces components and failure domains; a standalone catalog fits tables spread across several stores or clouds.
What questions should a CDO ask before choosing a lakehouse platform?
Which formats and engines can read and write the data and what leaving costs; whether performance holds on the metadata path; whether the platform provides strong consistency and atomic commits; how both the data and the catalog state are protected and recovered; how granular access control is and how it fits existing governance; and what the team has to run, where it can run, and what drives the cost; who owns table maintenance.
Is Apache Iceberg required for a lakehouse?
No. Delta Lake, Apache Hudi, and Apache Paimon are open table formats as well. Iceberg is supported by the major query engines and defines a standard catalog API, which is why it anchors most multi-engine designs.
Can you switch table formats later?
A qualified yes. Metadata translation tools such as Apache XTable (Incubating), along with Delta Lake's Universal Format, expose the same Parquet files under a second format's metadata without rewriting them. Feature parity is not guaranteed, particularly for row-level deletes and newer types, translation typically runs one way per table, and active streaming writers complicate the cutover. Plan a switch as a metadata project, not a data migration.
Decoupling compute from storage is settled. The decisions that still matter sit in the storage layer: where the catalog and the maintenance services live, and whether the platform holds the table format's guarantees under the metadata read pattern, the commit concurrency, and the recovery scenarios a lakehouse produces at scale. The six questions above put those decisions on the table before a contract does.
Request a free trial of MinIO AIStor to evaluate AIStor Tables against these questions in your own environment.
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
Block quote
Ordered list
Unordered list
Bold text
Emphasis
Superscript
Subscript
.svg)