मुख्य विषयवस्तु में जाएं

Migrate Between Object Storage Providers

विहंगावलोकन​

इलम supports several S3-compatible object storage backends as the data plane: the bundled मिनियो (default in the 6.7.x line), the bundled RustFS (opt-in; planned default in 6.8.0), and any external S3-compatible service such as AWS S3, Wasabi, or Backblaze B2. Switching between them is performed entirely through Helm values. The procedure below is the canonical migration playbook. The most-traveled path, मिनियो तक RustFS, is documented in detail; the Other migration recipes section at the end covers the reverse path and migrations involving external providers.

A pre-upgrade hook protects existing installs from accidental data loss, so the safe path is to make a deliberate choice rather than to upgrade with no overrides.

See first

How migration works​

Migration in इलम is parameterized: the same playbook covers any source-and-target pair. Three properties of the chart make this work:

  • Provider-agnostic alias. A stable Service named ilum-objectstorage is provisioned as a label-selector alias that routes to whichever provider is currently active. Bundled consumers target this alias rather than provider-specific Service names. Switching providers does not require consumer reconfiguration.
  • Shared credentials Secret. A single Secret, ilum-objectstorage-credentials, carries the S3 root credentials used by every consumer. The Secret exposes six aliased keys (access-key, secret-key, root-user, root-password, RUSTFS_ACCESS_KEY, RUSTFS_SECRET_KEY) so each consumer can read the same value under its native env-var name. For rotation, refer to Rotate Object Storage Credentials.
  • Pre-upgrade safety hook. एक pre-upgrade Helm hook detects an existing MinIO PersistentVolumeClaim and refuses to proceed when disabling MinIO would orphan the PVC without an acknowledged cutover. The hook is gated behind preUpgradeChecks.enabled (default सच्चा).

For the release-by-release record of when each property landed, refer to the नोट्स अपग्रेड करें. For the upstream chart status and known caveats of each provider, refer to the per-provider reference pages linked from the Object Storage Overview.

The cutover model​

The migration playbook is parameterized by three Helm values. Operators running 6.7.2-RC2 or later should prefer the new names; the legacy flag from earlier releases continues to work as a back-compat alias.

Helm valueप्रकारPurpose
objectStorage.activeProviderतारExplicit override. Set to a provider name (rustfs, मीनो, ...) to pin the alias to that provider. Defaults to auto, which defers to the resolution rules.
objectStorage.previousProviderतारNames the data-bearing side during a cutover when two providers are enabled and activeProvider=auto. Defaults to मीनो.
objectStorage.cutoverAcknowledgedboolFlips the alias from previousProvider to the other enabled provider once the operator has finished migrating data. Defaults to गलत.
rustfs.migrationAcknowledgedboolLegacy name from 6.7.2-RC2 onward; honored as an alias for objectStorage.cutoverAcknowledged. Still accepted; deprecated in favour of the new name in a future major release.

Both objectStorage.cutoverAcknowledged=true and the legacy rustfs.migrationAcknowledged=true produce the same result. The examples below use the legacy flag where it matches the existing मिनियो तक RustFS migration path. Subsequent migrations between other providers should prefer the generalized objectStorage.cutoverAcknowledged name.

Choose a path​

Three operator paths
  • Path A — Keep MinIO. Set two Helm flags. No data movement. No reconfiguration of consumers.
  • Path B — Migrate to RustFS. Run both providers side by side, mirror data, then disable MinIO. Either automated or manual.
  • Path C — Net-new install. Nothing to do. MinIO is the default in the 6.7.x line; the alias Service is created automatically. To start on RustFS instead, see Path B or the RustFS provider reference.

Path A: Keep MinIO​

Run a single हेल्म अपग्रेड with two explicit flags. The pre-upgrade hook accepts the upgrade because MinIO remains enabled.

helm upgrade ilum ilum/helm_aio \
  --अस्त हो minio.enabled=सच्चा \
  --अस्त हो rustfs.enabled=गलत

वही ilum-objectstorage Service alias is rendered with a selector matching MinIO pods, so consumer traffic continues to reach MinIO without any further reconfiguration.

Path B: Migrate to RustFS​

The migration runs both providers concurrently, mirrors bucket contents, and then disables MinIO. Either an automated migration Job or the manual mc mirror procedure can be used.

The bundled migration Job is gated on migration.minioToRustfs.enabled. It refuses to render unless both minio.enabled और rustfs.enabled are सच्चा.

Step 1. Run both providers and acknowledge the cutover.

helm upgrade ilum ilum/helm_aio \
  --अस्त हो minio.enabled=सच्चा \
  --अस्त हो rustfs.enabled=सच्चा \
  --अस्त हो rustfs.migrationAcknowledged=सच्चा

Step 2. Dry-run the migration. The Job invokes mc mirror --fake and reports what would be copied without writing.

helm upgrade ilum ilum/helm_aio \
  --अस्त हो minio.enabled=सच्चा \
  --अस्त हो rustfs.enabled=सच्चा \
  --अस्त हो rustfs.migrationAcknowledged=सच्चा \
  --अस्त हो migration.minioToRustfs.enabled=सच्चा \
  --अस्त हो migration.minioToRustfs.dryRun=सच्चा

Step 3. Inspect the Job logs.

कुबेक्टल -n ilum logs -l app.kubernetes.io/component=migration --tail=-1

Step 4. Run the migration for real.

helm upgrade ilum ilum/helm_aio \
  --अस्त हो minio.enabled=सच्चा \
  --अस्त हो rustfs.enabled=सच्चा \
  --अस्त हो rustfs.migrationAcknowledged=सच्चा \
  --अस्त हो migration.minioToRustfs.enabled=सच्चा

कुबेक्टल -n इलम wait job -l app.kubernetes.io/component=migration \
  --for=condition=complete --timeout=600s

Step 5. Verify (see the next section), then disable MinIO.

helm upgrade ilum ilum/helm_aio \
  --अस्त हो rustfs.migrationAcknowledged=सच्चा

Skipping the migration step​

Acknowledging the cutover without mirroring data loses the default job scaffold

Flipping minio.enabled=false, rustfs.enabled=trueऔर rustfs.migrationAcknowledged=true (या objectStorage.cutoverAcknowledged=true) without running the migration नौकरी or the manual mc mirror recipe leaves the new RustFS इलम फ़ाइलें bucket empty. Subsequent Spark job submissions fail with FileNotFoundException: No such file or directory: s3a://ilum-files/ilum/default/.py in the driver pod.

When the migration step has already been skipped, restart Ilum core to re-upload the default job scaffold to the active object-storage provider:

kubectl rollout restart deploy/ilum-core -n इलम

After the pod becomes Ready, resubmit the failing Spark job.

Verify the cutover​

CheckCommandExpected
Object Storage view loads in UInavigate to /object-storageiframe renders the RustFS console
Bucket paritymc diff src/ dst/empty output for every default bucket
Spark writesubmit a job that writes to s3a://ilum-files/probe/object visible in the RustFS pod
Default buckets presentmc ls dst/seven default buckets
MinIO PVC retainedkubectl get pvc -n ilum | grep ilum-minioPVC still exists until manually deleted

Migration-window operational FAQ​

Can consumers continue writing while mc mirror is running?​

Yes. Before objectStorage.cutoverAcknowledged करने के लिए सेट किया गया है सच्चावही ilum-objectstorage Service alias still targets the source provider, so every consumer continues to write there. Objects written to the source after the first mc mirror pass started may be missed by that pass. The recommended pattern is therefore to run the mirror twice: a first pass during normal operation, then a brief read-only window (quiesce schedulers and Spark jobs, scale long-running consumers to zero) during which a second mc mirror run finalizes the copy.

What happens if the migration Job fails mid-run?​

The bundled migration नौकरी is configured with backoffLimit: 0. A failure exits the Job and leaves whichever buckets had completed in their post-mirror state on the destination. Re-running the Job is safe: mc mirror --preserve is idempotent and resumes from the existing destination state. Delete the failed Job before the next हेल्म अपग्रेड if the Job's name would otherwise conflict:

कुबेक्टल -n ilum delete job -l app.kubernetes.io/component=migration

How do I verify the mirror was complete?​

mc diff src/ dst/ returns no output when source and destination are byte-identical. Run it for every bucket in objectStorage.defaultBuckets:

के लिए bucket में ilum-files ilum-data ilum-tables ilum-mlflow ilum-kestra ilum-ducklake ilum-langfuse; करना
  echo "=== $bucket ==="
  mc diff src/$bucket dst/$bucket
समाज-सम्मत

Empty output for every bucket means the cutover is safe. Any objects listed by mc diff should be reconciled (typically by running mc mirror again) before acknowledging the cutover.

Rollback​

Helm preserves PVCs across हेल्म अपग्रेड और helm rollback by default, so the original MinIO data is recoverable as long as its PVC has not been manually deleted. Re-enable MinIO and disable RustFS:

helm upgrade ilum ilum/helm_aio \
  --अस्त हो minio.enabled=सच्चा \
  --अस्त हो rustfs.enabled=गलत \
  --अस्त हो rustfs.migrationAcknowledged=सच्चा

Consumer traffic returns to MinIO automatically because the ilum-objectstorage Service alias re-targets MinIO pods.

Other migration recipes​

वही मिनियो तक RustFS path above is the canonical example. The same playbook applies to any source-and-target pair. The recipes below cover the three other paths that operators commonly take.

RustFS to MinIO (downgrade or rollback)​

The rollback procedure from the Rollback section above is the supported way to revert from RustFS back to मिनियो, provided the original मिनियो PersistentVolumeClaim still exists.

When the original मिनियो PVC has been deleted, the procedure becomes symmetric to the मिनियो तक RustFS playbook:

  1. Enable both providers: --set minio.enabled=true --set rustfs.enabled=true --set objectStorage.previousProvider=rustfs.
  2. Run mc mirror से rustfs/ तक minio/ for every default bucket.
  3. Set objectStorage.cutoverAcknowledged=true to flip the alias to मिनियो.
  4. Once मिनियो is verified, disable RustFS के साथ --set rustfs.enabled=false.

Migrate to an external S3 provider (AWS S3, Wasabi, Backblaze B2)​

Migrating to a managed external backend has no in-cluster Helm install for the target. Only the alias target and the data copy change.

  1. Provision the external bucket(s) and an IAM credential pair with read and write access. Make sure the bucket names match objectStorage.defaultBuckets.

  2. Update ilum-objectstorage-credentials to carry the external credential pair, or create a new Secret and point objectStorage.credentials.existingSecret at it.

  3. Run mc mirror from the in-cluster source to the external destination using the external provider's S3 endpoint as the destination alias.

  4. Disable the in-cluster provider and point objectStorage.endpoint at the external service:

    helm upgrade ilum ilum/helm_aio \
      --अस्त हो minio.enabled=गलत \
      --अस्त हो rustfs.enabled=गलत \
      --अस्त हो objectStorage.endpoint=https://s3.us-east-1.amazonaws.com

For provider-specific endpoint examples, refer to Provider Reference: External S3.

Migrate to a non-bundled provider added via the registry​

When the target is a provider that इलम does not yet ship as a sub-chart (for example, Garage नहीं तो समुद्री शैवाल एफएस), the migration follows the same shape as the मिनियो तक RustFS path but the bundled migration नौकरी does not apply: it is hard-coded for मिनियो तक RustFS. The procedure is:

  1. Provision the target. Stand up the new provider in the release namespace, with pods carrying labels app.kubernetes.io/name: और app.kubernetes.io/instance: . Follow Add a New Provider for the registry entry and the per-provider Service shape.
  2. Mirror the buckets manually. Use the mc mirror recipe from the Manual mc mirror tab, substituting the destination alias for the new provider's in-cluster Service name.
  3. Flip the alias. Set objectStorage.activeProvider= to pin the alias to the new backend explicitly. Skip objectStorage.cutoverAcknowledged (it applies only to the auto-mode resolution between two enabled providers).
  4. Verify के साथ mc diff for every default bucket, then disable the source provider once the verification is clean.

Migrate from external S3 back to a bundled provider​

To move from a managed external backend to a bundled provider (for example, repatriating storage to keep everything in-cluster), reverse the steps above:

  1. Enable the target bundled provider (--set rustfs.enabled=true नहीं तो --सेट minio.enabled=true). The ilum-objectstorage Service alias renders automatically.
  2. Run mc mirror from the external source to the bundled destination (http://ilum-rustfs-svc:9000 नहीं तो http://ilum-minio:9000).
  3. Update objectStorage.endpoint to point at the bundled provider's in-cluster Service, or remove the override so the chart resolves the alias automatically.

If the cutover goes wrong​

If the इलम UI returns 502 Bad Gateway after a cutover, the ilum-objectstorage Service alias most likely lost its endpoints. The fastest recovery is a helm rollback to the previous release revision. For the full diagnosis and recovery procedure, refer to Troubleshoot Object Storage: 502 from the Object Storage view.

Known limitations​

  • RustFS distributed mode is "under testing" upstream. The bundled defaults configure standalone mode with a single PVC.
  • Auto-wiring of Hydra OIDC into RustFS is not yet shipped. Operators that depend on OIDC against the object-storage console should override rustfs.extraEnv directly until the integration stabilizes.
  • MinIO root credentials are recorded by the MinIO server on first install. Rotating the shared Secret after MinIO has been bootstrapped does not change the live MinIO root user. RustFS reads its root credentials from the env on every Pod start, so RustFS supports live rotation.

References​

For the upgrade summary and the Helm chart changelog refer to the नोट्स अपग्रेड करें, which mirrors the in-tree helm/CHANGELOG.md.

For the upstream RustFS chart refer to the Artifact Hub listing.

For the mc client reference refer to the MinIO Client documentation.

Last updated on