Managing petabytes of search information means making powerful decisions: preserve all the things quick and costly, or make it reasonably priced however read-only. UltraWarm is a confirmed, cost-effective resolution for read-heavy historic information. Nevertheless, some workloads sometimes must replace historic information, reminiscent of late-arriving information or compliance corrections. With UltraWarm, you need to migrate these indices again to sizzling, carry out the replace, and migrate again. What in the event you may write on to your cost-effective heat storage as an alternative?
On this put up, I present you ways writable heat storage removes the expensive migration cycle. You possibly can cut back your infrastructure prices by as much as 48 % and replace historic information in seconds as an alternative of hours. I stroll by a real-world value comparability and efficiency benchmarks, and assist you resolve when to make use of writable heat versus UltraWarm.
The problem with tiered storage
Amazon OpenSearch Service handles data-intensive search and analytics workloads, from real-time log analytics and software monitoring to safety occasion detection. As your information volumes develop from terabytes to petabytes, you face a basic query: how do you retain latest information quick whereas making earlier information reasonably priced?
OpenSearch Service addresses this with a tiered storage structure:
- Scorching – Highest efficiency for lively indexing and search utilizing instance-attached storage.
- UltraWarm – Value-effective, read-only tier backed by Amazon Easy Storage Service (Amazon S3) with native caching for much less incessantly queried information.
- Chilly – Totally indifferent from the cluster, with the bottom value for hardly ever accessed information. Chilly indices should be migrated again to UltraWarm or sizzling earlier than any reads or writes could be carried out.
For immutable log information, this mannequin works effectively. Nevertheless, a selected class of workloads hits its limitations once they sometimes want to put in writing to earlier information, and read-only turns into a bottleneck.
Conditions
To make use of writable heat storage, you want the next:
- An Amazon OpenSearch Service area working model 3.3 or later.
- OpenSearch Optimized (OI2) occasion household help in your AWS Area.
- Workloads with a minimal 5-second refresh interval.
- Knowledge nodes utilizing the OpenSearch Optimized occasion household (OR2 for warm, OI2 for heat).
Notice: Writable heat doesn’t at the moment help the chilly storage tier.
The UltraWarm bottleneck
With UltraWarm, updating even a single doc requires migrating the index again to sizzling, performing the write, and migrating it again. This spherical journey entails a pressure merge (consolidating index segments), snapshot creation, and shard relocation. These operations eat important CPU, reminiscence, and disk area in your sizzling nodes, and so they take roughly 130 minutes per 100 GB index. This time was measured on a site with 3 × r6g.2xlarge sizzling nodes, 3 × ultrawarm1.massive heat nodes, and three devoted chief nodes (US East, N. Virginia), utilizing a single-shard index with one reproduction. Precise occasions range primarily based on area configuration, shard depend, section depend, sizzling node utilization, and migration queue depth. The result’s that you simply over-provision sizzling nodes, construct advanced pipelines, or preserve information in sizzling longer than needed, which will increase value and complexity.
Introducing writable heat storage
OpenSearch Service now presents writable heat nodes that use OpenSearch Optimized (OI2) cases, the identical occasion household that powers sturdy, Amazon S3-backed storage on sizzling nodes. As a result of information is already continued on Amazon S3, tier transitions grow to be a light-weight shard relocation reasonably than a resource-intensive migration. The Lucene engine, which is OpenSearch’s underlying search library, operates identically on each tiers. Because of this, writable heat nodes help lively writes, background merges, and periodic refreshes, similar to sizzling nodes.
Late-arriving information, compliance backfills, and corrections that beforehand required a warm-to-hot-to-warm spherical journey now resolve with a direct write in seconds. There isn’t a pressure merge, no snapshot, no shard relocation, and no sizzling node useful resource consumption.
UltraWarm (legacy) information stream: Knowledge is ingested into the recent tier (SSD, learn and write). Index State Administration (ISM) insurance policies migrate indices to UltraWarm (Amazon S3-backed, read-only). Any replace requires migrating the index again to sizzling (dashed arrow), writing, then migrating again.
Writable heat (new) information stream: Identical ingestion path by sizzling, with ISM transitioning indices to writable heat. The important thing distinction is that writable heat helps each reads and writes. Late-arriving updates go on to heat, with no migration again to sizzling. As a result of each tiers use Amazon S3 as sturdy storage by OpenSearch Optimized cases, transitions are light-weight shard relocations, not resource-intensive migrations.
The advantages: value, operations, and suppleness
Writable heat delivers benefits in three areas: value, operational simplicity, and suppleness.
Value
Not like UltraWarm, which solely presents on-demand pricing, OI2 cases help Reserved Occasion (RI) pricing, a commitment-based low cost mannequin. By committing to a 1-year or 3-year Reserved Occasion, it can save you 31–52 % in comparison with UltraWarm nodes. This makes writable heat considerably cheaper for predictable, long-running workloads. The newly launched Database financial savings plan for OpenSearch Service supplies financial savings of round 22 % over UltraWarm cases. Each tiers use Amazon S3 for sturdy storage, so node failure means solely short-term unavailability, not information loss. For cost-sensitive workloads that may tolerate transient downtime throughout node restoration, you’ll be able to configure zero replicas on heat indices to scale back prices additional.
Actual-world value comparability
Think about a workload ingesting 2 TB/day with 210 days whole retention, the place updates can arrive at any level. With UltraWarm’s read-only constraint, you need to preserve information in sizzling for 30 days earlier than migrating to heat. With writable heat, updates occur instantly on heat, so sizzling retention drops to solely 7 days.
At small scale, the recent tier discount profit is modest. Writable heat continues to be cost-effective in the event you want write functionality on heat information, can decide to RI pricing, or worth the operational simplicity of eliminating migration pipelines. For purely immutable information with quick retention, UltraWarm on-demand would possibly nonetheless be cheaper. Use the AWS Pricing Calculator to mannequin your particular situation.
The next desk exhibits estimated month-to-month prices utilizing on-demand and All Upfront Reserved Occasion (AURI) pricing within the US East (N. Virginia) Area as of March 2026. For the most recent pricing, see Amazon OpenSearch Service pricing on the AWS web site.
| Part | Scorching + UltraWarm (30d sizzling / 180d heat) | Scorching + writable heat (7d sizzling / 203d heat) |
| Scorching information nodes | $12,264 (21 × or2.2xlarge) | $12,264 (21 × or2.2xlarge) |
| Scorching EBS value | $10,212.84 (21 * 3986 GB) | $2,636 |
| Scorching distant storage | $2,008.28 | $518 |
| Heat information nodes | $39,128 (20× ultrawarm1.massive) | $50,409 (15× oi2.8xlarge) |
| Amazon S3 storage | $9,504 | $1,070 |
| Chief nodes | $1,307 (3 × m8g.2xlarge) | $1,307 (3 × m8g.2xlarge) |
| On-demand whole | $74,427 | $69,297 |
| 1-year AURI | $69,674 | $43,918 (~36% much less) |
| 3-year AURI | $67,367 | $34,939 (~48% much less) |
| Database financial savings plan | $71,708 | $55,406 (~22%) |
Operations
Reclaim sizzling node capability. Writable heat removes two widespread causes of sizzling node over-provisioning: reserving 35 % of disk area for pressure merge operations, and sustaining further capability to quickly transfer information again to sizzling for writes. You possibly can run your sizzling tier at greater utilization, which reduces the variety of sizzling nodes you want.
Less complicated migrations. UltraWarm migrations are multi-step operations (pressure merge, snapshot, and shard relocation) that want cautious scheduling throughout low-traffic home windows, and they’re restricted to 10 queued at a time. Writable heat simplifies this to a light-weight shard relocation, with extra easy ISM insurance policies and no scheduling constraints.
Flexibility
UltraWarm presents solely two occasion sizes: ultrawarm1.medium (1.5 TiB) and ultrawarm1.massive (20 TiB). Writable heat with OI2 cases presents a full vary from oi2.massive to oi2.16xlarge. Every measurement addresses as much as 5× its native cache measurement, so you’ll be able to right-size heat capability exactly to your workload.
Search efficiency
We benchmarked search latency utilizing the NYC Taxis workload, evaluating writable heat (oi2.massive) towards UltraWarm nodes. All measurements are P90 latencies.
On the NYC_TAXIS benchmark, writable heat matched or beat UltraWarm on 6 of seven question varieties at P90, together with light-weight filters, ranges, types, and time-histogram aggregations. For many real-world search patterns, writable heat delivers comparable or higher efficiency than UltraWarm, plus the power to put in writing on to the tier.
Search efficiency: writable heat in comparison with UltraWarm
| Job | Writable heat node latency in ms | UltraWarm latency in ms | UltraWarm vs. writable heat diff % |
| NYC_TAXIS workload sort | ** ** | ** ** | ** ** |
| default (P90) | 21.287 | 23.857 | 12.07223 |
| vary (P90) | 21.23 | 21.016 | -1.00718 |
| distance_amount_agg (P90) | 5,069 | 3929.23 | -22.48406 |
| autohisto_agg (P90) | 21.076 | 22.002 | 4.39348 |
| date_histogram_agg (P90) | 21.363 | 21.792 | 2.01031 |
| desc_sort_tip_amount (P90) | 23.224 | 23.797 | 2.46636 |
| asc_sort_tip_amount (P90) | 22.483 | 22.482 | -0.00445 |
When to decide on what
Do you have to swap from UltraWarm to writable heat? It depends upon your workload.
| Requirement | Writable Heat | UltraWarm |
| Write enabled | ✓ | Learn-only |
| Reserved Occasion pricing | ✓ | ✗ |
| Occasion measurement flexibility | Big selection (massive–8xlarge) | 2 choices solely |
| Chilly tier help | ✗ | ✓ |
| Want for OpenSearch Optimized occasion households | ✗ | ✓ |
| Concurrent tier transitions | ✓ | ✗ (sequential) |
| Scorching node affect throughout migration | Minimal | Excessive (CPU/reminiscence) |
Clear up assets
For those who created a check area to judge writable heat storage, delete it to keep away from ongoing expenses. Within the OpenSearch Service console, choose your area and select Delete. This removes all nodes and stops Amazon S3 storage expenses for that area.
Abstract
On this put up, I confirmed you ways writable heat storage eliminates the expensive migration cycle that UltraWarm’s read-only limitation creates. You stand up to 36 % value financial savings with 1-year Reserved Situations, sooner search efficiency, and a less complicated operational mannequin. Writable heat additionally removes information transitions between tiers, and Reserved Occasion pricing turns into accessible for heat storage for the primary time.
Writable heat requires OpenSearch Service model 3.3 or later with OI2 cases. For domains needing chilly tier help, earlier OpenSearch Service variations, or non-optimized occasion households, UltraWarm stays the appropriate selection.
Subsequent steps: Begin by analyzing your present sizzling and heat cut up. What number of days of information do you retain in sizzling solely to accommodate occasional updates? Use the AWS Pricing Calculator to mannequin your potential financial savings, and allow writable heat on a check area in minutes. On the time of this put up, writable heat is supported on OpenSearch Service model 3.3. For step-by-step directions, see Migrating to writable heat storage within the OpenSearch Service documentation.
Have you ever tried writable heat storage? I’d love to listen to about your expertise and any questions you’ve gotten within the feedback.
In regards to the creator
