Kvindo™ Cloud Docs
Copy indices from another OpenSearch cluster into a Kvindo Cloud cluster, prove the copy is complete, then point your application at the new one. The script is re-runnable and checkpoints per index, so an interrupted migration resumes instead of starting over.
A mapping on its own is not enough. The most common failure in this migration is an index whose strings use a custom analyzer: the analyzer is declared in settings.index.analysis, and the mapping only names it. Create the target index with the mapping alone and OpenSearch rejects it outright:
So read both, and carry across the settings that define the index's shape — analysis, number_of_shards (immutable once created), number_of_replicas.
Treat resource_already_exists_exception as success. A migration script gets re-run after an interruption, and a create step that refuses to run twice turns a five-minute resume into a manual cleanup.
Scroll the source and write with _bulk, keeping the source _id on every document. Set index.refresh_interval to -1 for the duration and restore the value you recorded afterwards — at the default 1s the verify step below reads a partially refreshed index and reports a short count that looks exactly like a failed copy.
Three failures actually happen, and each needs its own handling:
• Rate limiting. Opening many scrolls at once answers 429; an idle scroll answers search_context_missing_exception. Both invalidate the scroll — the retry must recreate it, with backoff and jitter, not re-issue the dead id.
• Partial bulk failures. _bulk answers HTTP 200 even when documents were rejected; the result is in the body's errors flag. A mapper_parsing_exception there means a field's type drifted between source and target — report that document, do not skip it.
• Re-running an index. Safe with the index operation, which overwrites the same _id. Using create instead returns version_conflict_engine_exception for every document already present.
Progress is checkpointed per index, never by document offset: scroll order is not stable, so an index is either finished or redone from the start — which is safe precisely because every write carries the source _id.
A scroll over a live index gives no point-in-time consistency, so stop your writers before the final pass and prove they stopped: watch the source's docs.count and its newest @timestamp stay flat for the interval you agreed. A "newest timestamp matches" check alone passes happily if the last write was forty minutes ago.
Then _refresh and compare GET <index>/_count and the newest @timestamp on both sides. For an index with no @timestamp field, the count alone is the check.
Use index-level endpoints, not _cat/indices: a tenant OpenSearch User has no cluster permissions by design, so cluster-level reads are refused with no permissions for [cluster:monitor/state]. _count, _mapping and _search all work.
If your application queries an alias rather than a concrete index, recreate that alias on the target before cutover, or the switchover breaks queries that were fine a minute earlier. Adding an alias that already exists is a no-op, so this step is safe to re-run.
Point the application at the target cluster's REST endpoint(s) from status.restEndpoints, pin the target's CA (status.caCertificate), and swap the credential for the OpenSearch User's. Keep the alias name unchanged so nothing else has to move.
The copy only ever read from the source, so any failure before this point is recoverable: point the application back at the old cluster, and resume the copy the next day. Nothing has to be undone.
Reads the source settings, creates the target index, loads with retry and a per-index checkpoint, restores the refresh interval, and refuses to mark an index done unless the target's own count matches what it wrote.
Measured on a real migration — source: a managed OpenSearch 2.19.6 cluster with 9,810 indices, 913 M documents, 346 GB; target: a Kvindo Cloud cluster with 2 data nodes and 3 cluster managers running 3.9.0.
| Phase | Measured |
|---|---|
| Build the work list (one aggregation) | ~9 s for all 9,810 indices |
| Per-index cost, end to end | ~580 ms at 8 workers (620 ms on a 40-index slice; 1,027 ms at 4 workers) |
| Delta pass: 1,729 changed indices, 1.7 M documents | 1,006 s |
Read that table for what it is: the cost is per index, not per document. These figures describe a delta pass over many small indices. A full initial load of a large corpus is dominated by bulk throughput instead, and was not measured here — do not scale these numbers by document count.
• The custom analyzer lives in settings, not the mapping — see step 1
• Per-index overhead dominates a delta; copying more documents into the same index is cheap, touching more indices is not
• The source rate-limits concurrent scrolls — run in waves rather than opening hundreds at once
• On the cluster itself network.host is the node's private IP, so 127.0.0.1:9200 refuses; use the address from status.restEndpoints
• Open file descriptors. Every shard segment a node holds costs descriptors, and a freshly loaded index has many segments before its merges run — so a bulk load is the moment a node runs out. Watch each node's open-descriptor count against its limit, and stop the wave if any node passes 80%. Exhausting them does not fail the load cleanly: shards come back ALLOCATION_FAILED — Too many open files and the cluster goes red
• Deletes are not propagated. A document deleted on the source after your copy stays in the target. If your data is not append-only, reconcile deletions separately.
• Index templates and lifecycle policies are not copied. Indices created after cutover inherit whatever the target defines, not the source's templates.
• It assumes an append-only source. Verify that before relying on it — step 4 is where you prove it, not where you assume it.
If a shard is stuck ALLOCATION_FAILED, POST _cluster/reroute?retry_failed=true usually clears it once the load has stopped. The stronger escape — allocate_empty_primary with an explicit node and accept_data_loss: true — discards the shard's data. It is acceptable for a system index such as an audit log; it is destructive on a tenant index, and the request body looks the same either way, so check which index you are naming before you run it.
Do not copy the source's system indices (.opendistro-*, .plugins-*, .security, .kibana*). The target's security configuration and system indices are managed by the platform; writing the source's over them would overwrite the cluster's own authorization configuration.
Resource docs