Keep the Data Where It Lives: Three Ways to Run ClickHouse Private on AIStor

ClickHouse is one of the fastest analytical databases available, and ClickHouse Private brings that speed on premises (ClickHouse docs). For organizations with strict data sovereignty rules, very large data volumes, or a security team that wants everything inside its own perimeter, that's a big deal.

AIStor is a data foundation for enterprise AI and analytics. It's software that runs on industry-standard servers in your own data center, giving AI training and inference, lakehouse analytics, and databases one place to keep and serve their data, from petabytes up to exabytes. Apache Iceberg tables are built in through AIStor Tables, so structured tables and unstructured files live side by side. And because AIStor implements the complete S3 API, the common language most modern data tools already use, those tools connect to it without custom code.

There's one design choice that makes it work: ClickHouse Private keeps its data in MinIO AIStor and the ClickHouse servers themselves hold no permanent copy. Separating storage from compute this way is the gold standard for modern analytical databases, because with this design choice each side can scale on its own. Performance is a key element for both sides of the equation, and it is for this reason (among others) that AIStor is the on-premises data platform ClickHouse itself supports and recommends for ClickHouse Private. This is the first time ClickHouse has certified an on-premises data foundation, so let's explore this new architecture.

This post walks through three ways to run ClickHouse Private on AIStor. In all three, the data stays in AIStor, on hardware the customer owns, and ClickHouse brings the analytics.

Why Underlying Data Platform Matters

With ClickHouse Private's SharedMergeTree engine, all table data and metadata live in object storage, coordinated through ClickHouse Keeper. That means storage decides how fast queries run, how quickly new data lands, and how far a deployment can grow. ClickHouse's own deployment guide makes the point directly: if the object store can't keep up with the load, ClickHouse can't either.

On-premises data platforms tend to fall short in two places. Many implement only part of the S3 API, and the missing pieces often stay hidden during testing and then surface in production. Others perform well at a few hundred terabytes and slow down as data grows into the petabytes.

AIStor implements all 400+ S3 API operations and runs at exabyte scale on customer-owned hardware. It's listed as the on-premises storage in ClickHouse Private's supported configurations, and ClickHouse's bare-metal guide names it as the recommended on-premises option.

Pattern 1: ClickHouse Tables Stored in AIStor

This is the default for ClickHouse Private. When data arrives in ClickHouse, through inserts or a streaming tool like Kafka, ClickHouse writes it to AIStor in SharedMergeTree format. That's the only copy, and it's in ClickHouse's own format, so only ClickHouse reads it.

Each ClickHouse server keeps a local cache on fast NVMe drives in front of AIStor. Recently read data is served from the cache, and anything else is read from AIStor when a query needs it. Because every server reads the same data in AIStor, a new server can start answering queries right away, with no data to move or rebalance.

This is what cloud-native means here: a design that leverages cloud-like practices to scale quickly and easily by adding resources. It has nothing to do with public cloud. Teams add ClickHouse servers when queries grow and AIStor capacity when data grows, each on its own schedule, all on their own hardware.

Pattern 2: Open Formats Queried in Place

Many organizations already keep Parquet files or Apache Iceberg tables in AIStor for other analytics and AI tools, such as Spark or Trino. ClickHouse can query that data directly, without loading it first.

For Iceberg, ClickHouse's DataLakeCatalog engine connects to AIStor Tables, the Iceberg catalog built into AIStor. It signs in with OAuth, and every Iceberg table in the catalog shows up as a table in a ClickHouse database. From there, ClickHouse can read, create, and write to Iceberg tables. This works on ClickHouse 26.9 and later.

Here's what connecting looks like, adapted from MinIO's test environment:

SET allow_database_iceberg = 1;

CREATE DATABASE aistor_tables
ENGINE = DataLakeCatalog('https://aistor.example.com/_iceberg/v1',
                         '<access_key>', '<secret_key>')
SETTINGS catalog_type       = 'rest',
         warehouse          = 'analytics',
         catalog_credential = '<client_id>:<client_secret>',
         oauth_server_uri   = 'https://aistor.example.com/_iceberg/v1/oauth/tokens',
         auth_scope         = 'catalog',
         storage_endpoint   = 'https://aistor.example.com';

SELECT count() FROM aistor_tables.`demo.trips`;

The benefit is one open copy of the data. Spark, Trino, AI pipelines, and ClickHouse all read the same Iceberg tables in AIStor.

Pattern 3: Two Copies, Both in AIStor

Querying Iceberg in place is flexible, but it has a speed cost. ClickHouse's own tutorial shows a filtered query on an external Iceberg table scanning the whole dataset in several seconds, then running much faster once the same data sits in a ClickHouse table.

Pattern 3 gets both. AIStor holds two copies of the hot data: an open Iceberg version for every tool, and a ClickHouse-only SharedMergeTree version tuned for speed. ClickHouse creates the SharedMergeTree copy with one SQL statement, with no separate ETL tool:

CREATE TABLE trips_hot (id Int64, day Date, fare Float64)
ENGINE = MergeTree ORDER BY (day, id);‍INSERT INTO trips_hotSELECT id, day, fare FROM aistor_tables.`demo.trips`;

On ClickHouse Private, that MergeTree table is created as SharedMergeTree, so it lives in AIStor right alongside the Iceberg source. The ORDER BY clause sets how the data is sorted, which is what lets ClickHouse skip data it doesn't need. (MinIO tested this copy step with open source ClickHouse 26.9 and standard MergeTree tables. On ClickHouse Private, the same statements create SharedMergeTree tables.)

To keep the copy current, teams rerun the copy on a schedule or use a refreshable materialized view, which reruns it automatically at a set interval. A TTL (time-to-live) rule can also keep only recent data in the fast copy, say the last 90 days. Older rows drop out of the ClickHouse table, but the full history stays in Iceberg, where ClickHouse can still query it through Pattern 2.

The difference from Pattern 1 is where the data starts. Pattern 1 suits data that only ClickHouse uses. Pattern 3 suits data that other tools already wrote to AIStor and that also needs ClickHouse speed for dashboards.

Choosing a Pattern

Most organizations end up running all three: open formats for data shared across tools, and ClickHouse tables for the busiest dashboards.

Security With Temporary Credentials

Long-lived storage keys are a common weak spot in analytics stacks. AIStor lets ClickHouse sign in through the organization's own identity provider, such as Keycloak or Okta, instead.

For Parquet data, ClickHouse gets a token from the identity provider and hands it to AIStor's Security Token Service. AIStor checks the token and returns short-lived credentials limited to what that client is allowed to do. In MinIO's testing, read-only clients could read but not write, tampered tokens were refused, and keys used without their session token were denied.

For Iceberg tables, ClickHouse signs in to the AIStor Tables catalog with OAuth, and each access policy can be limited to a single warehouse. In the tested setup, ClickHouse still uses fixed storage keys to read the Iceberg data files themselves, so those keys belong in a secrets manager.

Getting Started

Setting up ClickHouse Private on AIStor takes five steps:

1. Deploy AIStor on-premises, on servers with high network throughput and fast drives, sized for the data plus growth. Teams already running AIStor can use their existing deployment.
2. Deploy ClickHouse Private on self-hosted Kubernetes and give it an AIStor bucket and credentials. SharedMergeTree tables now write to AIStor automatically.
3. Connect AIStor to your identity provider so ClickHouse receives temporary credentials, and turn on OAuth for the AIStor Tables catalog.
4. Optionally, point ClickHouse at existing Parquet files in AIStor, or attach the AIStor Tables catalog so every Iceberg table appears in ClickHouse.
5. Run standard ClickHouse SQL across all of it, and copy the busiest slices into SharedMergeTree tables when they need more speed.

Two configuration tips from ClickHouse: use strict read-after-write consistency on AIStor, and turn off object versioning on ClickHouse buckets, since versioning can leave data behind after ClickHouse deletes it.

To try these patterns hands-on, MinIO's ClickHouse + AIStor OAuth demo runs on a single machine with Docker. It covers Parquet access through temporary credentials, Iceberg reads and writes through AIStor Tables, and copying Iceberg data into a ClickHouse table. For the full architecture, read the MinIO AIStor + ClickHouse solution brief.