Introducing Amazon EBS Quantity Clones: Create instantaneous copies of your EBS volumes

As somebody that used to work at Solar Microsystems, the place ZFS was invented, I’ve all the time cherished working with storage programs that supply instantaneous quantity copies for my improvement and testing wants.

Right now, I’m excited to share that AWS is bringing related capabilities to Amazon Elastic Block Retailer (Amazon EBS) with the launch of Amazon EBS Quantity Clones, a brand new functionality that allows you to create instantaneous point-in-time copies of your EBS volumes inside the identical Availability Zone.

Many shoppers must create copies of their manufacturing information to assist improvement and testing actions in a separate nonproduction surroundings. Till now, this course of required taking an EBS snapshot (saved in Amazon Easy Storage Service (Amazon S3)) after which creating a brand new quantity from that snapshot. Though this strategy works, the method creates operational overhead attributable to a number of steps.

With Amazon EBS Quantity Clones, now you can create copies of your EBS volumes with a single API name or console click on. The copied volumes can be found inside seconds and supply instant entry to your information with single-digit millisecond latency. This makes Quantity Clones notably helpful for rapidly organising check environments with manufacturing information or creating short-term copies of databases for improvement functions.

Let me present you the way Quantity Clones works
For this submit, I created a small Amazon Elastic Compute Cloud (Amazon EC2) occasion, with an connected quantity. I created a file on the foundation file system with the command echo "Whats up CopyVolumes" > hey.txt.

To provoke the copy, I open a browser on the AWS Administration Console and I navigate to EC2, Elastic Block Retailer, Volumes. I choose the quantity I wish to copy.

Be aware that, on the time of publication of this submit, solely encrypted volumes will be copied.

On the Actions menu, I select the Copy Quantity possibility.

Subsequent, I select the small print of the goal quantity. I can change the Quantity kind and regulate the Measurement, IOPS, and Throughput parameters. I select Copy quantity to start out the Quantity Clone operation.

The copied quantity enters the Creating state and turns into out there inside seconds. I can then connect it to an EC2 occasion and begin utilizing it instantly.

Information blocks are copied from the supply quantity and written to the quantity copy within the background. The quantity stays within the Initializing state till the method is full. I can monitor its progress with the describe-volume-status API. The initializing operation doesn’t have an effect on the efficiency of the supply quantity. I can proceed utilizing it usually in the course of the copy course of.

I really like that the copied quantity is out there instantly. I don’t want to attend for its initialization to finish. Throughout the initialization part, my copied quantity delivers efficiency based mostly on the bottom of: a baseline of three,000 IOPS and 125 MiB/s, the supply quantity’s provisioned efficiency, or the copied quantity’s provisioned efficiency.

After initialization is accomplished, the copied quantity turns into absolutely impartial of the supply quantity and delivers its full provisioned efficiency.

Alternatively, I can use the AWS Command Line Interface (AWS CLI) to provoke the copy:

aws ec2 copy-volumes                          
     --source-volume-id vol-1234567890abcdef0 
     --size 500                               
     --volume-type gp3

After the quantity copy is created, I connect it to my EC2 occasion and mount it. I can examine the file I created at begin is current.

First, I connect the quantity from my laptop computer, utilizing the attach-volume command:

aws ec2 attach-volume 
         --volume-id 'vol-09b700e3a23a9b4ad' 
         --instance-id 'i-079e6504ad25b029e'   
         --device '/dev/sdb'

Then, I hook up with the occasion, and I kind these instructions:

$ sudo lsblk -f
NAME          FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
nvme0n1                                                                              
├─nvme0n1p1   xfs          /     49e26d9d-0a9d-4667-b93e-a23d1de8eacd    6.2G    22% /
└─nvme0n1p128 vfat   FAT16       3105-2F44                               8.6M    14% /boot/efi
nvme1n1                                                                              
├─nvme1n1p1   xfs          /     49e26d9d-0a9d-4667-b93e-a23d1de8eacd                
└─nvme1n1p128 vfat   FAT16       3105-2F44     

$ sudo mount -t xfs /dev/nvme1n1p1 /information

$ df -h
Filesystem        Measurement  Used Avail Use% Mounted on
devtmpfs          4.0M     0  4.0M   0% /dev
tmpfs             924M     0  924M   0% /dev/shm
tmpfs             370M  476K  369M   1% /run
/dev/nvme0n1p1    8.0G  1.8G  6.2G  22% /
tmpfs             924M     0  924M   0% /tmp
/dev/nvme0n1p128   10M  1.4M  8.7M  14% /boot/efi
tmpfs             185M     0  185M   0% /run/consumer/1000
/dev/nvme1n1p1    8.0G  1.8G  6.2G  22% /information

$ cat /information/residence/ec2-user/hey.txt 
Whats up CopyVolumes

Issues to know
Quantity Clones creates copies inside the identical Availability Zone as your supply quantity. You’ll be able to create copies from encrypted volumes solely, and the scale of your copy should be equal to or higher than the supply quantity.

Quantity Clones creates crash-consistent copies of your volumes, precisely like snapshots. For software consistency, it’s worthwhile to pause software I/O operations earlier than creating the copy. For instance, with PostgreSQL databases, you should utilize the pg_start_backup() and pg_stop_backup() features to pause writes and create a constant copy. On the working system stage on Linux with XFS, you should utilize the xfs_freeze command to briefly droop and resume entry to the file system and guarantee all cached updates are written to disk.

Though Quantity Clones creates point-in-time copies, it enhances reasonably than replaces EBS snapshots for backup functions. EBS snapshots stay the advisable resolution for information backup and safety towards AZ-level and quantity failures. Snapshots present incremental backups to Amazon S3 with 11 nines of sturdiness, in comparison with Quantity Clones which maintains EBS quantity sturdiness (99.999% for io2, 99.9% for different quantity varieties). Think about using Quantity Clones particularly for check and improvement surroundings eventualities the place you want instantaneous entry to quantity copies.

Copied volumes exist independently of their supply volumes and proceed to incur normal EBS quantity expenses till you delete them. To handle prices successfully, implement governance guidelines to establish and take away copied volumes which are not wanted on your improvement or testing actions.

Pricing and availability
Quantity Clones helps all EBS quantity varieties and works with volumes in the identical AWS account and Availability Zone. This new functionality is out there in all AWS business Areas, chosen Native Zones, and within the AWS GovCloud (US).

For pricing, you’re charged a one-time charge per GiB of information on the supply quantity at initiation and normal EBS pricing for the brand new quantity.

I discover Quantity Clones notably invaluable for database workloads and steady integration (CI) eventualities. For example, you may rapidly create a duplicate of your manufacturing database for testing new options or troubleshooting points with out impacting your manufacturing surroundings or ready for information to hydrate from Amazon S3.

To get began with Amazon EBS Quantity Clones, go to the Amazon EBS part on the console or take a look at the EBS documentation. I sit up for listening to how you utilize this functionality to enhance your improvement workflows.

— seb