मुख्य विषयवस्तु में जाएं

External S3 (BYO)

विहंगावलोकन​

इलम does not require a bundled object storage sub-chart. Any S3-compatible backend reachable from the cluster can serve as the data plane, including AWS S3, Wasabi, Backblaze B2, on-prem मिनियो, Google Cloud Storage via the S3 interoperability layer, or any self-hosted S3 implementation.

This page explains the configuration shape. The integration is operator-driven: इलम does not provision the buckets, manage the credentials, or expose a console for an external provider.

Propertyमूल्य
लाइसेंसProvider-dependent
Status in IlumSupported
Bundled chartNone
Bundled imageNone
Default in helm_aioNot active by default; requires setting both rustfs.enabled और minio.enabled तक गलत (the chart enables मीनो by default)
Distributed modeProvider-dependent
Console UXProvider-hosted; not embedded in the इलम UI
OIDC integrationProvider-dependent

When to use an external S3 backend​

  • Production deployments where object storage operations should be delegated to a managed service.
  • Multi-cluster setups where multiple इलम releases share a single backing bucket layout.
  • Air-gapped or compliance-bound environments that already operate a central S3-compatible service.
  • Cost or scale profiles that exceed what the bundled providers offer in their current default configurations.

Configure Ilum against an external S3 backend​

Disable both bundled providers and supply the endpoint plus credentials through the chart's existing values:

helm upgrade ilum ilum/helm_aio \
  --अस्त हो rustfs.enabled=गलत \
  --अस्त हो minio.enabled=गलत \
  --अस्त हो objectStorage.endpoint=https://s3.us-east-1.amazonaws.com \
  --अस्त हो objectStorage.credentials.accessKey=<aws-access-key-id> \
  --अस्त हो objectStorage.credentials.secretKey=<aws-secret-access-key>

The shared ilum-objectstorage-credentials Secret carries the credentials; every bundled consumer reads from it via the env keys documented in Rotate Object Storage Credentials.

When both rustfs.enabled और minio.enabled are गलत, the chart does not render the ilum-objectstorage Service alias. Consumers resolve the configured endpoint hostname directly.

Region-specific examples​

AWS S3​

objectStorage:
  समापन बिंदु: एचटीटीपीएस://s3.us-पूर्व-1.amazonaws.com
  region: us-पूर्व-1
  pathStyle: गलत
  प्रत्यय-पत्र:
    existingSecret: इलम-objectstorage-प्रत्यय-पत्र

Wasabi (EU)​

objectStorage:
  समापन बिंदु: एचटीटीपीएस://s3.eu-केंद्रीय-1.wasabisys.com
  region: eu-केंद्रीय-1
  pathStyle: सच्चा
  प्रत्यय-पत्र:
    existingSecret: इलम-objectstorage-प्रत्यय-पत्र

Backblaze B2​

objectStorage:
  समापन बिंदु: एचटीटीपीएस://s3.us-west-002.backblazeb2.com
  region: us-west-002
  pathStyle: गलत
  प्रत्यय-पत्र:
    existingSecret: इलम-objectstorage-प्रत्यय-पत्र

Google Cloud Storage (S3 interop)​

objectStorage:
  समापन बिंदु: एचटीटीपीएस://storage.googleapis.com
  region: auto
  pathStyle: गलत
  प्रत्यय-पत्र:
    existingSecret: इलम-objectstorage-प्रत्यय-पत्र

The S3 interop credentials are HMAC keys. Provision them with gcloud against a service account that has the Storage HMAC Key Admin role:

# Grant the role on the project (skip if the SA already has it).
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
  --member="serviceAccount:$SA_EMAIL" \
  --role="roles/storage.hmacKeyAdmin"

# Create the HMAC key. Capture access_id and secret from the output.
gcloud storage hmac create "$SA_EMAIL"

Both the accessId and the रहस्य from the output map onto the access-key और secret-key entries in ilum-objectstorage-credentials. The S3-compatible endpoint for GCS is https://storage.googleapis.com regardless of bucket location.

Bucket bootstrap​

The bundled init नौकरीs (init-rustfs-buckets, init-minio-policies) do not run against an external backend. The operator must create the default buckets manually before installing इलम:

इलम फ़ाइलें
ilum-data
इलम टेबल्स
ilum-mlflow
ilum-kestra
ilum-ducklake
ilum-langfuse

The bucket list is configurable via objectStorage.defaultBuckets. Bundled consumers expect bucket names from that list.

Known limitations​

  • No embedded console. The Object Storage view in the इलम UI does not render for external backends. Operators access the provider's own console through the provider's domain.
  • No bucket lifecycle automation. The bundled migration नौकरी (migration.minioToRustfs) runs against the in-cluster providers only. Migrations involving an external backend should be performed manually with mc or the provider's own tooling. See Migrate Between Providers for the procedure.
  • OIDC and IAM-mode credentials. The chart sources S3 credentials from the shared Secret. AWS IAM-role-based authentication (IRSA, instance profiles) requires a sub-chart override that bypasses the Secret-based credential flow.

Reference​

Last updated on