In today’s market businesses are likely familiar with the enterprise services provided in Azure and have already started planning a cloud transition. As Azure expands its core offering and more business transition to a cloud platform, the question becomes, “how does an organization move large amounts of data to the cloud, specifically to Azure? Introducing the Microsoft Azure Import/Export for Azure Resource Manager (ARM) (currently in Beta) tool.
In a recent project, West Monroe Partners designed, implemented, and migrated a large complex banking “software as a service” (SaaS) application out of a “Big-3” credit bureau’s on-premises environment to Microsoft Azure. The effort involved designing the core infrastructure systems from the ground up to support the financial institution’s SaaS application in the Azure Cloud. One of the challenges confronted during the project was the effort to transfer large amounts of data to Azure. There were two other options that we considered: 1) send the data over the Internet circuit or 2) setup an ExpressRoute. While both options were technically possible, transferring data over the Internet would have (literally) taken months and the setup time for ExpressRoute would have forced the team to miss tight project deadlines. The Import/Export Tool was a great “middle-ground” option.
In the case of this project, we utilized Microsoft’s Import/Export Tool in ARM to accomplish the data transfer (importing) from existing servers to an Azure storage account. The Import functionality of the tool allows a company to securely copy data to standard SATA hard drives. The hard drives are then shipped to a Microsoft data center, where an automated process transfers the data to a blob storage account in an Azure subscription. The Export feature of the tool can also be used to transfer data from an Azure Storage Account to an On-Premises environment. This article references the tool as “Import/Export”, however the focus will be on the tool’s Import features and functionalities.
While the Import/Export tool has been around in Azure Classic mode for some time, the service had not yet been released to the public and was not documented. Fortunately, the partnership between West Monroe and Microsoft allowed exclusive access to the Import/Export tool in ARM.
The steps below will assist in the planning and identifying the prerequisites required to relocate data to an Azure Storage Account using the Import/Export Tool in ARM. In addition, the guidelines below will help determine the total storage capacity required and the number of hard drives to purchase.
Prerequisites & Requirements:
- An Azure Subscription is required. At this point, the assumption is that the core infrastructure is developed and operational in ARM
- An Azure Storage Account in ARM is required to stage the migrated data, it must be available and configured. Proceed with creating the blob containers that will store the migrated data
- Ensure that your storage account(s) reside in a supported Azure Data center. The supported US data centers are:
- East US
- West US
- East US 2
- Central US
- North Central US
- South Central US
- Only 2.5” inch SSD or 2.5” or 3.5” SATA II or III internal hard drives are supported. SSD and/or SATA III is highly recommended, and up to 10 TB max capacity is supported. The storage requirements will drive the decision on how many hard drives are required
- A USB Adapter is required. We had a good experience using the “Anker USB 3.0 to SATA 3ft Converter”; it’s around $20.00 on Amazon. Purchase a couple for the sake of redundancy
- Collect business storage requirements. For example, calculate the total amount of data that will be migrated to Azure and then estimate the time to copy said data over the LAN and/or USB to a hard drive to ensure any time frame requirements can be met.
- Tip: Use an online calculator to estimate the time to copy data to the hard drives. The “File Transfer Time Calculator” on http://thecloudcalculator.com/ is a great online tool
Now that the prerequisites are complete, the next step is to purchase the necessary hardware, configure the dedicated blob storage account in Azure, and begin preparing the data for transit. In Part 2 of “More Data, More Problems” we will discuss the techniques, lessons learned, and issues associated with the Import/Export tool in ARM.