Migrate a Server 2008 R2 VM from atlantic.net to AWS

Question

I have a number of servers on the Atlantic Net Cloud put up many years ago, and now we have most of our stuff on AWS. The nice thing about AWS is it allows me to shut down (and not pay for) a rarely used machine. I would like to take advantage of that, but migrating it will be difficult, especially the one Windows server that's a pain in the youknowwhat to set up. No big issue with Unix.

Is there a way to just download the VM image? What if I just pulled the binary disk image (with cygwin dd or so), send it to AWS, write it binary on a EBS disk (dd) and then try booting that. Would this have any chance of success?

I have done that with physical laptop drives many times: just pull out of a NUC and into a ThinkPad and then when that one falls apart, transfer it on another ThinkPad, and back to the NUC, whatever, it has always worked. But I've never done that with VMs and AWS is particularly critical as you don't get console access during boot.

Answer 1

You should use the AWS Application Migration Service. Yes, you use something called "application migration service" to migrate servers, go figure. Once you go into that area of the console it will guide you through the process, but you basically install an agent on your source server, it replicates into AWS, you stop the old server, you start the new server.

If your OS is too old and unsupported by AWS you should copy your data over and reinstall your applications, rather than doing unsupported workarounds.

Answer 2

I was actually able to do this and I feel extremely accomplished now.

One thing most people don't realize is how easy it is to move Windows from one hardware to another. The approach is simply to take a raw copy of the disk, plug it into the new machine, and boot.

For those who think setting up a Windows system from scratch is easier, it's not, especially not if you rely on over 10 year old software and you don't have the installation disks: it is definitely easier to just swap a system disk into a new hardware!

With migrating to AWS it's a little more tricky without a console. But it can be done.

Overview

1. Install AWS drivers on the old, non AWS machine, and turn on boot logging.
2. Get cygwin if you haven't already, this is to have dd that can access the raw disk, gzip and ssh was also used.
3. Shrink your system volume (C: drive) as much as you can.
4. If you have enough space on the disk after shrinking, create a file system partition on the other disk or prepare to clone the system partition raw into the new one, if your can shut down the system and have access to the disk, even easier. But on the Atlantic cloud you can't do that.
5. Take a raw dd copy of the system disk to the other space, from the the start of the disk until just after the system partition, and the last few blocks of the physical disk, writing the big copy through gzip saves you some 40% space, this is handy for the next step.
6. Copy those two raw data files to some temporary space ideally connected to a UNIX (FreeBSD, Linux, whatever) server on AWS. I used scp or a pipe through ssh. This is the most time consuming step, took me 10 hours at 1.6 MB/s transfer speed. This is why it's so important to shrink that system volume.
7. Get an AWS EC2 EBS disk of the size of the original (in my case 160 GB and it ended up being precisely the same number of blocks as my original, very nice.)
8. Attach that new disk to the running UNIX system.
9. Now zcat | dd the main raw data file over to that new disk, and write those last blocks also.
10. Detach that new disk from the UNIX system.
11. Launch an EC2 Windows instance, start, and connect. I used t2.micro (free tier) which is the working architecture (t3.micro did not work).
12. Attach that new disk to the new Windows system.
13. In the partitioning tool you might see the new disk as "Offline", right-click and bring Online
14. You might see that spare partition in the shrunk space, that content was not copied, just delete that partition. On the system disk clone, click Properties and do a Scan / check of the file system (that was shut down dirty in the cloning).
15. Do a clean shut down the Windows system
16. On the EC2 console, once system is Stopped, detach both disks, and attach the cloned disk as /dev/sda1 (boot disk).
17. Now start the instance.
18. As soon as it goes from Pending to Started status, get the instance screen shot, you will see it churning, and hopefully not get stuck but show the "Press ALT+CTL+DEL" screen. So the system booted.
19. If you can't already reach the system with Remote Desktop, wait a while, and then on the EC2 console issue a normal stop instance. No need to be impatient and do force stop. Just wait a minute and it should stop. Then restart again.
20. Now when the instance screenshot goes to the "Press ALT+CTL+DEL" screen, try the Remote Desktop again, now it should work.

Details

To point 1. Installing AWS drivers on the old, non AWS machine, it's not trivial. In my case I had a super old Windows 2008 server that wouldn't even run the install.ps1 scripts that come with much of the AWS drivers.

This document here: https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/migrating-latest-types.html was the best guide to what to do.

1. Install AWS PV drivers
2. Install AWS ENA driver
3. Install AWS NVMe drivers
4. Install EC2Config and EC2Launch

The PV driver was hard, because the latest version tells you it doesn't support your old system. But after some digging I found out that the last supported version is 8.3.4, and then I figured out that the URL to that is: https://s3.amazonaws.com/ec2-windows-drivers-downloads/AWSPV/8.3.4/AWSPVDriver.zip. Once that was found, the included .msi installer ran without problems.

The other drivers worked from latest release. But running the install.ps1 scripts for the ENA driver was fairly easy. For the NVMe drivers it was hard. I ended up just reading the script with my own eyes, interpreting what it does, and doing it manually.

The EC2Config and EC2Launch installations were straight forward.

Now re-booting the system, even on the old platform, already came up showing the AWS background with the AWS system information panel on the top right corner. Not showing much, but it tells you that this EC2Launch thing did its magic.

About dd copies.

On cygwin the disk drives are /dev/sda, sdb, sdc, ... You need that raw drive, not the partitions. First you need to get the exact sizes of the partitions you want to copy:

$ wmic partition get name,bootable,size,type
Bootable  Name                   Size          Type
TRUE      Disk #0, Partition #0  104857600     Installable File System
FALSE     Disk #0, Partition #1  104805171712  Installable File System
FALSE     Disk #0, Partition #2  66884468736   Installable File System

Those sizes are in bytes. You add up the boot and system partition, 104857600 + 104805171712 = 104910029312 bytes. To speed up dd, you want to read in bigger blocks, say 10 MiB (bs=10M). In that case, you need to divide by 1024 twice and 104910029312 /1024 /1024 /10 = 10005.00004882.. and then just round up:

dd if=/dev/sda bs=10M count=10006 status=progress |gzip -cv > /cygdrive/d/diskimage.gz

the end of the disk, you just sum up all partitions 66884468736 + 104805171712 + 104857600 = 171794498048 bytes / 1024 = 167768064 kiB blocks:

dd if=/dev/sda bs=1024 skip=167768064 of=/dev/null status=progress
4096+0 records in
4096+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.057467 s, 73.0 MB/s

4 kiB is a nice small size. If this was very large, you'd know how big your disk is exactly and then just take the last 4 kiB.

$ dd if=/dev/sda bs=1024 skip=167768064 of=diskimage.end status=progress
4096+0 records in
4096+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0673959 s, 62.2 MB/s

When these copies are made move the stuff over to AWS, on a UNIX machine with the new EBS volume attached, check carefully where it is, so you don't mistakenly wipe your other disks. In my case:

zcat diskimage.gz |dd of=/dev/nvd4 bs=10M status=progress
  104847114240 bytes (105 GB, 98 GiB) transferred 1550.052s, 68 MB/s
0+1600960 records in
10006+0 records out
104920514560 bytes transferred in 1551.084986 secs (67643305 bytes/sec)

and the end:

dd if=diskimage.end of=/dev/nvd4 bs=1024 seek=167768064  status=progress
  2983936 bytes (2984 kB, 2914 KiB) transferred 2.002s, 1491 kB/s
4096+0 records in
4096+0 records out
4194304 bytes transferred in 2.836086 secs (1478906 bytes/sec)

Now you are ready to boot!

When I first could not connect to it, it was because the network was not being properly set up.

2022/03/14 17:45:53Z: Windows sysprep configuration complete.
2022/03/14 17:46:03Z: Failed to find primary network interface...retrying...
2022/03/14 17:46:13Z: Failed to find primary network interface...retrying...
2022/03/14 17:46:24Z: Failed to find primary network interface...retrying...

With boot logging turned on, and looking at some AWS related setup logs in C:\Windows (E:\Windows when attaching the disk to a running Windows system) sort directory view by descending file time, I could see that the xen drivers were installed, but xenbus was not found and xennet not installed. But on the second boot I noticed suddenly that the xennet was installed now. And all problems were gone.

Answer 3

It seems you're doing a square-peg/round-hole with trying to make this work with Windows on AWS. That said, I've got two ideas for you.

1. Did you try this procedure to gain boot console access? You just say "it didn't work" without saying what you tried.

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/troubleshooting-sac.html

2. You could also run this in nested virtualization. Make a new EC2 instance running a modern version of Windows, run HyperV or VMware, and try to boot your disk image. You'll at least be able to get console access and mount additional install media if you need them.

/Edit - you stated several parts of the process in your problem description but not in any detail. How did you pre-install the drivers? Did you use the packages from here?

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/Upgrading_PV_drivers.html