This can be a visitor put up by Satoru Ishikawa, Options Architect at Classmethod in partnership with AWS.
In April 2025, AWS introduced the deprecation of Amazon Redshift DC2 situations, guiding customers emigrate to both Redshift RA3 situations or Redshift Serverless. Redshift RA3 situations and Serverless undertake a design that separates storage and compute, presents new options reminiscent of information sharing, concurrency scaling for writes, zero-ETL , and cluster relocation.
On this put up, we share insights from one in all our prospects’ migration from DC2 to RA3 situations. The client, a big enterprise within the retail business, operated a 16-node dc2.8xlarge cluster for enterprise intelligence (BI) and ETL workloads. Going through rising information volumes and disk capability limitations, they efficiently migrated to RA3 situations utilizing a Blue-Inexperienced deployment method, reaching improved ETL question efficiency and expanded storage capability whereas sustaining price effectivity.
Amazon Redshift structure varieties
Amazon Redshift presents two deployment choices: Provisioned mode, the place you select the occasion sort and variety of nodes and handle resizing as wanted, and Redshift Serverless, which routinely provisions information warehouse capability and intelligently scales the underlying assets. The next diagram compares these two structure varieties.
Provisioned clusters require you to find out cluster dimension upfront, however you possibly can optimize prices by buying Reserved Situations (RI) or scheduling pause and resume actions. Serverless routinely provisions assets as wanted, with a pay-per-use mannequin the place you solely pay for compute assets consumed. Each companies help migration between one another and provide the identical options together with SQL, zero-ETL, and Federated Question capabilities. For particular pricing particulars, see Amazon Redshift pricing.
Provisioned clusters are appropriate for large-scale, predictable workloads and provide automated scaling primarily based on queuing. Serverless gives management-free automated scaling for variable workloads with AI-driven optimization that scales primarily based on workload complexity and information volumes. For extra particulars, consult with Evaluating Amazon Redshift Serverless to an Amazon Redshift provisioned information warehouse.
Buyer case research: Migration from DC2 situations
This part describes the client’s migration from Amazon Redshift DC2 to RA3 occasion varieties. The migration used a Blue-Inexperienced deployment method that minimized downtime whereas reaching each price optimization and efficiency enchancment.
The client’s workload had the next traits:
Use instances
The client had the next key use instances for his or her Amazon Redshift deployment:
- Question by way of BI device throughout enterprise hours
- Excessive quantity of learn queries
- Peak entry throughout Mondays and starting of months
- Information processing in early morning
- Concentrated write queries for information loading and transformation
- Regular-state workload traits
- Run queries greater than 16 hours each day
Necessities
The client had the next key necessities for his or her Amazon Redshift migration:
- Efficiency
- Use auto-scaling (reminiscent of concurrency scaling) throughout peak entry intervals
- Information dimension
- Disk capability growth wanted
- Price Administration
- Simple finances prediction and administration
- Make the most of low cost companies for long-term utilization
- Compatibility
- Keep compatibility with present functions and BI instruments
- Keep away from endpoint modifications
- Availability
- Most downtime of 8 hours acceptable throughout migration
- Community
- Don’t modify the prevailing 2-Availability Zone (AZ) subnet configuration
- When emigrate
- To be carried out throughout low-load days and hours
- Deliberate downtime doable inside 8 hours
Key concerns in system design, implementation, and operation included prolonged operation hours, ease of finances prediction and administration, price optimization by means of Reserved Situations (RI), and sustaining compatibility with present methods (avoiding endpoint modifications). The client evaluated Amazon Redshift Serverless, which provided enticing options reminiscent of a pay-per-use mannequin, automated scaling capabilities, and the potential for higher worth efficiency for variable workloads. Whereas each Redshift Serverless and provisioned clusters might successfully help their workload patterns, the client selected the provisioned mannequin with RA3 nodes, leveraging their years of operational expertise with provisioned environments, present RI technique, and established capability planning method.
Options of RA3 occasion sort
Constructed on the AWS Nitro System, RA3 situations with managed storage undertake an structure that separates computing and storage, permitting unbiased scaling and separate billing for every element. These situations use high-performance SSDs for warm information and Amazon S3 for chilly information, offering ease of use, cost-effective storage, and quick question efficiency. For extra particulars, consult with Amazon Redshift RA3 situations with managed storage.
Migration conditions
The client had the next migration conditions in place:
- The client used a Redshift cluster with 16 nodes of dc2.8xlarge configuration.
- The client selected a Blue-Inexperienced deployment method for migration, the place they might restore from a snapshot to RA3 occasion sort, enabling fast rollback if vital.
- The client carried out cluster switching and rollback by means of endpoint switching utilizing cluster identifier rotation.
- Moreover, to enhance efficiency with excessive concurrency, they transitioned the transaction isolation stage from SERIALIZABLE ISOLATION to SNAPSHOT ISOLATION.
Cluster migration strategies
There have been two migration choices out there: Elastic Resize and Traditional Resize.
Amazon Redshift’s Traditional Resize performance had been enhanced, for resizing to RA3 occasion varieties, considerably lowering the write-unavailable interval. Primarily based on PoC testing, after initiating the resize, the cluster’s standing was modifying for 16 minutes earlier than it grew to become out there. Primarily based on these outcomes, the client proceeded with the Traditional Resize method.
Cluster sizing
Sizing concerned figuring out the occasion sort and variety of nodes for the migration goal. Sizing factors thought of workload traits reminiscent of CPU-intensive (queries utilizing excessive CPU), I/O-intensive (queries with excessive information learn/write), or each.When migrating from DC2 occasion varieties, extra nodes may be required relying on workload necessities. Nodes have been added or eliminated primarily based on the computing necessities for vital question efficiency.
Evaluating configurations with comparable cluster prices when it comes to occasion dimension and rely, for a dc2.8xlarge 16-node cluster, the beneficial configuration was 8 nodes of ra3.16xlarge. The next was the associated fee comparability within the Tokyo Area:
- Really helpful: dc2.8xlarge 16-node cluster => ra3.16xlarge * 8-node cluster
- $97.52/h (6.095/h * 16 nodes) => $122.776/h (15.347/h * 8 nodes)
- Price-focused: dc2.8xlarge 16-node cluster => ra3.16xlarge * 6-node cluster
- $97.52/h (6.095/h * 16 nodes) => $92.082/h (15.347/h * 6 nodes)
For this migration, the client proceeded with a cost-efficient 6-node ra3.16xlarge cluster to remain inside present finances constraints. Nevertheless, since this node rely might face throughput limitations throughout sure instances, they enabled concurrent scaling for the RA3 occasion sort to deal with spike entry.
Concurrency scaling gives as much as 1 hour of free credit per day for every energetic cluster, accumulating as much as 30 hours. On-demand utilization charges apply when exceeding this free tier.Whereas the client selected to implement concurrency scaling, Elastic Resize to quickly enhance nodes throughout peak hundreds was additionally thought of however rejected as a consequence of on-demand prices for added nodes and the temporary disconnection interval throughout switching.
Managed storage price
RA3 situations use Redshift Managed Storage (RMS), which is charged at a set GB-month charge. The client’s roughly 2 TB of knowledge required together with storage prices within the estimates. For pricing particulars, see Amazon Redshift pricing.
Migration step from DC2 to RA3
After creating an RA3 cluster from the DC2 cluster’s snapshot, the client swapped the cluster identifiers. The next diagram reveals this course of.
- Take a snapshot of the present DC2 cluster.
- Restore RA3 cluster from the snapshot with a unique cluster identifier (Traditional Resize)
- Swap the cluster identifiers between the present DC2 cluster and the brand new RA3 cluster.
If any points come up after the cluster swap, you possibly can rapidly roll again by returning the unique DC2 cluster to its unique cluster identifier.
Word: Restore from a snapshot
Working the restore operation utilizing CLI instructions is beneficial to reduce operational errors and guarantee reproducibility. The next is a pattern command.
Manufacturing migration length
The time required for the restore and basic resize steps can fluctuate considerably relying on information quantity and goal cluster specs. The client carried out a rehearsal beforehand to measure the precise required time.
Check outcomes
Earlier than the manufacturing migration, the client created a take a look at cluster by restoring a snapshot to the RA3 occasion sort. Whereas Redshift Check Drive is usually helpful for workload testing, this buyer confronted distinctive constraints: enabling audit logging of their manufacturing cluster would require configuration modifications, cluster restarts, and complicated approval processes underneath their strict change administration insurance policies. To deal with this, they developed a customized load testing device that captured workload patterns utilizing Amazon Redshift system views (SYS_QUERY_HISTORY and SYS_QUERY_TEXT), which keep 7 days of question historical past. The device replayed 55,755 historic queries with 50-way parallelism in opposition to each DC2 and RA3 clusters, evaluating metrics together with question execution time, CPU utilization, and disk I/O. Question outcome caching was disabled throughout testing to make sure correct comparisons.
BI question efficiency
BI queries have been examined utilizing the customized load testing device. The outcomes characterize the typical execution time from 15 take a look at runs of 55,755 queries executed with 50-way parallelism. With out concurrency scaling, the dc2.8xlarge 16-node cluster averaged 45.82 seconds per question, whereas the ra3.16xlarge 6-node cluster averaged 91.30 seconds. This indicated that RA3 situations confirmed longer execution instances for brief and medium queries in a direct migration with out optimizations. Nevertheless, enabling concurrency scaling improved RA3 efficiency progressively. With concurrency scaling enabled at most 2 clusters, the ra3.16xlarge 6-node cluster achieved a mean of 72.48 seconds per question, a 21% enchancment over the non-scaled configuration.
| Node Sort / Variety of nodes | Common Question Time |
| ra3.16xlarge 6-node cluster | 72.48 seconds |
ETL question efficiency comparability
For long-running ETL queries (execution time better than 10 minutes), the RA3 cluster demonstrated higher efficiency than DC2. These outcomes represented a direct migration of the client’s workload with no optimizations utilized.
- For the Giant-scale information load workload 1, the ra3.16xlarge cluster accomplished the question 28% sooner than the dc2.8xlarge cluster (41 minutes vs. 57 minutes).
- For the Advanced transformation workload 1, the ra3.16xlarge cluster was 23% sooner (1 hour 1 minute vs. 1 hour 20 minutes).
These outcomes indicated that the RA3 node sort was extra performant for time-intensive information loading and transformation duties. The upper CPU utilization values for RA3 recommended more practical compute useful resource utilization.
| Node Sort / Variety of nodes | Common Question Time | MAXCPU% |
| ra3.16xlarge 6-node cluster | 41 minutes 09 seconds | 11:45 |
| dc2.8xlarge 16-node cluster | 57 minutes 07 seconds | 10:85 |
| Node Sort / Variety of nodes | Common Question Time | MAXCPU% |
| ra3.16xlarge 6-node cluster | 1 hour 01 minutes 33 seconds | 74:23 |
| dc2.8xlarge 16-node cluster | 1 hour 20 minutes 36 seconds | 53:58 |
Efficiency tuning
Primarily based on the take a look at outcomes, the client recognized that RA3 confirmed longer execution instances for brief and medium BI queries however sooner efficiency for long-running ETL queries in comparison with DC2. To optimize total efficiency, they centered on figuring out gradual queries and continuously referenced tables, prioritizing optimizations with the best influence.
Efficiency tuning technique
The client thought of a number of optimization methods to leverage RA3’s architectural benefits. One key technique concerned pre-processing ad-hoc quick and medium question workloads throughout low-load intervals, creating pre-processed tables or materialized views for queries that repeatedly carried out joins, aggregations, filters, and projections. RA3’s separated compute and storage structure, with cost-effective large-scale storage, supported this method.
Changing common views to materialized views
Evaluation of gradual queries revealed using joins in views, and continuously referenced tables have been being accessed a number of instances by means of these views. As a countermeasure, the client changed continuously used common views with materialized views, eradicating pointless information ranges and redundant columns.
Amazon Redshift helps incremental updates of materialized view contents by way of the REFRESH MATERIALIZED VIEW command, enabling environment friendly information updates.
Materialized views and question rewrite
By changing common views to materialized views, present queries could also be routinely optimized by means of the “question rewrite” characteristic supplied by the question planner. For extra particulars, consult with “Automated question rewriting to make use of materialized views“.
Automated tuning with AutoMV
On the DC2 cluster, disk utilization persistently exceeded 80%, which disabled the AutoMV characteristic as a consequence of inadequate disk house. With RA3’s expanded storage, automated tuning by means of AutoMV grew to become doable, resulting in additional efficiency enhancements. For extra particulars about AutoMV, consult with Automated materialized views.
Efficiency tuning outcomes
After making use of these optimizations, the client achieved the next outcomes:
- Maintained present efficiency whereas controlling price will increase
- Achieved greater CPU utilization whereas sustaining throughput
- Enhanced dynamic throughput throughout peak load intervals utilizing concurrency scaling’s automated scaling
Conclusion
On this put up, you realized how a big retail enterprise efficiently migrated from Amazon Redshift DC2 to RA3 situations. The Blue-Inexperienced deployment method enabled a secure migration with fast rollback functionality, whereas the separated compute and storage structure of RA3 supplied flexibility to deal with rising information volumes. Though RA3 confirmed completely different efficiency traits for brief BI queries in comparison with DC2, the client achieved important enhancements in long-running ETL question efficiency (as much as 28% sooner for information hundreds and 23% sooner for advanced transformations). By leveraging RA3-specific options reminiscent of materialized views and AutoMV, they optimized total question efficiency whereas sustaining price effectivity by means of Reserved Situations and concurrency scaling.
To proceed your RA3 migration journey, see Greatest practices for upgrading from Amazon Redshift DC2 to RA3 and Amazon Redshift Serverless and Resize Amazon Redshift from DC2 to RA3 with minimal or no downtime for added steerage and greatest practices.
Concerning the authors
