In the previous article, [Step-by-Step Guide] How to Export Your EC2 Instances to Your Openstack Cloud, we described the steps required to export an instance out of AWS and import it into your OpenStack cloud.
The process required a significant number of manual steps – making it error prone and difficult to use for more than a one-off export – which is not very useful in your cloud management strategy. You are entitled to have multiple options and to this end, you need to have a way to perform this export procedure on a regular basis – with little or no manual intervention.
In this article, we provide an example of a script that automates the export of an instance from AWS to OpenStack.
You will need the following elements in order to execute this script:
- A Linux Instance that you can access through SSH. This instance shall be named: Worker.
- The Worker instance must:
- Be able to access both your OpenStack cloud API endpoints and the AWS endpoints. (If your OpenStack cloud is not publicly accessible, then it makes sense that this instance resides on the corporate network with access to both.)
- Have the following software installed (with all their prerequisites)
- Have enough disk space available to download the exported image from AWS to a local disk.
- Credentials to AWS – already sourced and loaded into the Environment variables
- Credentials to OpenStack – already sourced and loaded into the Environment variables.
- The ID of the instance that you want to export from AWS ( in the format i-09ec9d3f6f91eead5)
- An S3 bucket that already exists in the same region your instance is located.
- The instance that is being exported must also have the AWS CLI installed. (This is usually the case when the instance is an AWS AMI – if not – then the AWS CLI needs to be installed on the instance.)
The script should be run from the Worker instance:
## This script automates the process of exporting a single AWS instance and
## importing it as an Glance image into OpenStack
##Check for parameters
if [ $# -ne 5 ]
echo “Not all arguments supplied were supplied. Please provide all the arguments to the script.”
echo “Usage: $progname <instance-id> <bucket-name> <aws_access_key_id> <aws_secret_access_key> <default_region>”
echo “For example: $progname i-09ec9d3gd731eead5 my-bucket AKIXVSDESIIDIH5OBXZA s4Zss3tF0fFF9FOOISIISwHrWVs3VG us-west-1”
echo “Script now Exiting…..”
AWS_DEFAULT_REGION=$5The scripts requires five parameters, and will fail to launch if any of the five are not present:
## Get Volume and PublicIP information from instance
PUBLIC_IP=$(aws ec2 describe–instances —instance–id $INSTANCE_ID | grep PublicIpAddress | sed –e ‘s/^[ \t]*//’ –e ‘s/[ \t]*$//’ –e ‘s/\”//g’ –e ‘s/PublicIpAddress: //g’ –e ‘s/,//g’)
— The AWS CLI query returns a JSON object with all the information on the instance, using SED to zoom in on the Volume ID and the Public IP Address of instance.
VOL_SIZE=$(aws ec2 describe–volumes —volume–id $VOL_ID | grep Size | sed –e ‘s/^[ \t]*//’ –e ‘s/[ \t]*$//’ –e ‘s/\”//g’ –e ‘s/Size: //g’)
VOL_AZ=$(aws ec2 describe–volumes —volume–id $VOL_ID | grep AvailabilityZone| sed –e ‘s/^[ \t]*//’ –e ‘s/[ \t]*$//’ –e ‘s/\”//g’ –e ‘s/AvailabilityZone: //g’ –e ‘s/,//g’)
— The commands above use the AWS CLI to retrieve specific information about the volume that is attached to instance: its type, size and Availability Zone. These are all required for the creation of another volume with the same characteristics.
## Create new Volume
## Attach Volume to instance
— The commands above use the AWS CLI to attach the volume to the instance, partition the disk, and copy an image of the original disk to an image file.
— In order to copy the newly created image to S3, the instance needs to have the correct credentials to push a file to the S3 bucket. This is injected into the instance above and then the file is copied.
— Next we clean up –
there’s no reason to keep this additional volume around and keep on paying. This was only a temporary measure and the volume can now be removed.
— Once the image is copied to the Worker, it can be imported into your OpenStack environment with the glance command.
You can build on the script above to automate the export of one or all of your AWS instances to your private OpenStack cloud to create a hybrid environment. The script can be expanded in many ways to add additional functionality and error handling, and integrated into the workflow that you use in your environment to even run an export of all your AWS instances into your cloud on a regular basis.
This is not a comprehensive solution to ensure seamlessly migration of your AWS environment, or even part of it, into your OpenStack private cloud. When you decide to migrate will vary according to your specific requirements – be they performance, security, or costs.
Many additional services that people enjoy AWS do not have a decent counterpart in OpenStack. This script will not cover the network topology and security that you have already implemented in AWS for your instances (Network ACLs and Security groups, for example). The process can be quite time consuming as you are copying large amounts of data (even though a good part of that data is probably blank blocks as instances are not usually maximised to full capacity). However, it does provide some degree of freedom and allows you to choose to move some of your resources back to your on-premises OpenStack environment.
To create a real hybrid cloud environment, you should consider an out-of-the-box solution such as Symphony, which offers a quick-to-deploy OpenStack private cloud for the enterprise’s data center. The platform ensures optimal utilization of your data center hardware and enables users to use all services via APIs. In addition, it has a native plugin to your AWS cloud environment. This inherent compatibility with AWS simply makes your private cloud your on-premises AWS region, providing your enterprise users a unified experience.
Learn more: OpenStack APIs versus AWS APIs