0

I have a large 200 GB file to extract. I was using WinRAR on a Windows laptop, and it's a very slow process.

I am considering using an Amazon EC2 instance to help speed up the process.

How can I speed up the process of extracting such a large file?

Peter Mortensen
  • 2,318
  • 5
  • 23
  • 24
Kyle
  • 119
  • 1
  • 1
  • 5

2 Answers2

10

Using a cloud service isn't going to make this faster - unless you have an incredible internet connection just uploading it will take longer than extracting it locally. And longer still to download the extracted files.

There are a few things you can do to make it go faster though:

  • Turn off your antivirus software. Realtime scanning can make this kind of process go 10x slower than without for some antivirus scanners. Be sure to scan all the files afterwards.
  • More RAM. Use a system with as much RAM as possible. 8 or 16GB would be good. Make sure you use a 64bit system.
  • Extract to a different hard drive than the one the zip file is stored on. If you extract to the same drive it has to bounce back and forth between reading the zip and writing the files, this slows it down tremendously.
  • Use SSDs for both the zip file and extracted files. Faster disks will help. There is a good chance your laptop has a 5400 RPM hard drive, which are extremely slow vs an SSD.

Also, it doesn't help you now, but if you are creating more of these zip files try using a lower compression method. The really high compression methods use dramatically more CPU and memory to extract. Sometimes it may be the difference between a 200GB file that takes 3 hours to extract, or a 205GB one that takes 20 minutes. So it is worth testing if this is something you plan to do regularly.

Grant
  • 17,859
  • 14
  • 72
  • 103
  • Would it be possible to use the cloud instance to extract and recompress with a faster extracting method? Thereby negating a large portion of the actual network transfer? – birdman3131 Sep 18 '15 at 15:35
  • @birdman3131 maybe that would help a little, but its still going to be significantly slower than extracting locally. – Grant Sep 18 '15 at 21:21
-1

You could also consider using a system with RAID 0 for this. This wil speed up the extracting about 2x

AgeDeO
  • 99
  • 1