-5

I need to encrypt a 200gb file.

Using 7z, on a pretty fast win2012 server, it is going to take 12 hours (using the fastest option).

Is there a faster way?

MadHatter
  • 79,770
  • 20
  • 184
  • 232
Jonesome Reinstate Monica
  • 5,445
  • 10
  • 56
  • 82
  • Why do you need it to be quicker? What are your security considerations (that is, why are you encrypting it)? Does it have to be done on Windows 2012? – Xiong Chiamiov Mar 29 '16 at 05:01
  • 1
    Yes. Get a high end server. Something with 3-4 very multi core processors. SERIOUSLY; what a question. You are CPU bound - so only faster CPU's work, unless you get the source and optimize the program. – TomTom Mar 29 '16 at 05:03
  • @TomTom, `7z` is a file compression program with optional encryption ("`fastest`" means the fastest file compression). As @samsmith said below in his answer to his own question, it's sped up greatly by simply *turning off the compression* (the time was reduced from "12 hours" to "~33 minutes"). The data rate there is about the same rate as a hard drive, so it might actually be IO-bound rather than CPU-bound. – Olathe Mar 29 '16 at 07:00

3 Answers3

7

Alot of this is going to be:

  1. Disk I/O - because you probably don't have 200GB of RAM, it's going to have to load blocks in at a time and then write them out. So, to improve this, use a high-speed (6GB/s) SSD on SATA3, or a RAID card for even faster speeds.
  2. Block size - This may not work as well on Windows, but on OSs where you can choose or tune the filesystem, managing the block read/write sizes relative to the bus and RAM can improve this considerably.
  3. RAM - The more you can do in RAM, the faster it gets, and RAM speed has an effect also, particularly if your CPU can dump unattended I/O directly between RAM and disk.
  4. CPU - CPUs with crypto-related maths extensions can do better for that crypto type (e.g. AES-NI), particularly if they can access the data directly rather than via a pipe.
tudor -Reinstate Monica-
  • 1,805
  • 18
  • 36
  • If a 200GB file is taking 12 hours this is not I/O-bound. It's also unlikely to be memory-bound although the evidence is not so direct. In practice this is a rare case where a problem really is compute-bound only. So *don't* spend lots of money on SSD or memory to improve performance here. –  Mar 29 '16 at 06:31
  • This doesn't answer the question well. The question includes vital details that weren't taken into account in this answer: `7z` is a file archiving and **compression** program and `fastest` is a method of low-quality but quick **compression**. Compression is very slow compared to encryption. The compression can be turned off to greatly speed things up, as shown in @samsmith's answer to their own question, where the time went down from 12 hours to 33 minutes. – Olathe Mar 29 '16 at 07:37
1

Discovery #1: Use the "store" option for compression method, that brings it to ~33 minutes....

Jonesome Reinstate Monica
  • 5,445
  • 10
  • 56
  • 82
  • 1
    It's not going to get much faster unless you have a really fast disk. It's almost surely IO-bound. – Olathe Mar 29 '16 at 04:47
  • @Olathe: it isn't: do the maths: 200GB/43200s is 4.7MB/s. This is hugely below the bandwidth os any modern disk. –  Mar 29 '16 at 06:37
  • 1
    33 minutes is much less than 43200 seconds, which is about half a day. I get about 103 MB/s with 200GB/1980s, which seems like it might be a 7200 RPM hard drive or something like that. – Olathe Mar 29 '16 at 06:42
  • @Olathe: Oh, sorry, I was referring to the original time (there's an answer which suggest buying fancy disk/SSD for that when clearly it's not the bottleneck, and I did not read this answer properly). Yes, in this case it probably is I/O limited: especially as it has to do well over 200GB of I/O (read 200, write quite a lot). Sorry! –  Mar 29 '16 at 08:54
  • Yes, this answer should really have been part of the question because I took is as such when I constructed my answer. From other comments, I believe others here also read it the same way. – tudor -Reinstate Monica- Mar 29 '16 at 22:32
-1

I have found using the 7z command line to be faster than the gui for large files.

A suggestion could be to store the file on an encrypted virtual drive image instead of using 7z.

tdev
  • 14
  • 1
  • 2