How to encrypt big files with php and decrypt over commandline?

Question

I'm writing a small script in PHP to backup my files. Before I transfer the files from the server I want to encrypt them.

I did that in previous versions of my script by using exec() and OpenSSL on my Linuxserver. Now I'm looking for a native PHP-Function to do this job, mainly for better error handling.

The thing is that my files may get big (like 20gb). Furthermore it's a must that I can use a command on my shell to decrypt the file again.

Does anyone know how to encrypt big files in PHP and then decrypt on the commandline?

I'm now using PHP's mcrypt functions to encrypt:

// IV:
$iv = mcrypt_create_iv($ivSize, MCRYPT_RAND);

// Create new random Key:
$key = openssl_random_pseudo_bytes(32);

// Encrypt:
$fileStream = fopen($file, "r");
$encFileStream = fopen($file . ".enc.data", "w");

$opts = [
    'iv'   => $iv,
    'key'  => $key,
    'mode' => 'cbc'
];
stream_filter_append($encFileStream, 'mcrypt.rijndael-256', STREAM_FILTER_WRITE, $opts);
stream_copy_to_stream($fileStream, $encFileStream);

fclose($fileStream);
fclose($encFileStream);

// Encrypt random generated key and save it:
$encryptedKey = null;
openssl_public_encrypt($key, $encryptedKey, $publickey);
file_put_contents($file . ".enc.key", $encryptedKey);

// Save Initial Vetor:
file_put_contents($file . ".enc.iv", $iv);

// Delete unencrypted file:
unlink($file);

Now to decrypt on the linux commandline I tried to use mcrypt as well. But the big problem I have with that is that I have no idea how to add the IV to mdecrypt. My command I have so far is:

mdecrypt --decrypt -m CBC -f key.key archive_hallo.tar.gz.enc.data

Which of course does not work because the IV is missing. So, does anyone know how to add the IV to my mdecrypt command?

score 3 · Accepted Answer · answered Mar 15 '14 at 08:39

3

mdecrypt CLI tool from mcrypt_2.6.8 debian package expects the specific file format with header. Search for write_file_head function, it shows first 3 bytes of the mcrypt/mdecrypt tool format:

buf[0] = '\0';
buf[1] = 'm';
buf[2] = '\3';

Header should have special bit set, and the IV embedded into the file. It will be read here, in decrypt_general function:

if (_mcrypt_iv_is_needed(td, mode, noiv) != 0) {
   IV = read_iv(FROMF, mcrypt_enc_get_iv_size(td));
} else {
   IV = _mcrypt_calloc(1, mcrypt_enc_get_iv_size(td));
}

and read_iv is defined in extra.c as:

void *read_iv(FILE * fstream, int ivsize)
{
char *IV;
     ... // malloc 
fread(IV, 1, ivsize, fstream);
    return IV;
}

So, just check first N bytes of encrypted file (you can post them here after hexdump -C) and check for mcrypt header (0x00 'm' 0x03) and for your IV. If yes - then mdecrypt tool should get IV from file. If not - there is no support for setting IV via command-line in mdecrypt, but you can hack into sources, add new option for iv, then modify decrypt_general to read iv from option.

answered Mar 15 '14 at 08:39

osgx

90,338
53
357
513

Hey, thank you for answering. If I do hexdump the first line is: `00000000 76 80 ee 78 05 07 03 36 72 a7 62 46 78 95 11 91 |v.îx...6r§bFx...|`. How can I find out if the second byte is an 'm'? – Matthias Dunkel Mar 17 '14 at 10:35
'm' is 0x6d - use www.asciitable.com. So, your output from PHP mcrypt is not in format of `mdecrypt` utility. You should add right header and fill IV into it; OR you can hack into `mdecrypt` utility and add IV command-line option. Or you can write small php script to decrypt file using php functions, and then run this php script from the bash as `php script.php` or smth like. – osgx Mar 17 '14 at 11:38
Okey, thank you very much. Gonna try to add the header and the IV to the file. Is there a Documentation of how the header has to be? Or do I have to figure it out by reading the code? Thank you again! – Matthias Dunkel Mar 17 '14 at 12:29
I don't get it. Can you help me? Here is how I tried it (see line 13, 14): https://gist.github.com/DeDu/3804940cffc19db4aedf – Matthias Dunkel Mar 17 '14 at 13:00
Matthias, you can try to encrypt known file with command-line `mcrypt` and compare results with your script (there is `--noiv` option to set IV to all zeroes). Don't know about exact format (reading sources may help here). – osgx Mar 17 '14 at 14:25
Wasn't able to do it, but thank you anyway @osgx . I just use my own script to decrypt... If anyone else was successful, please let me know. – Matthias Dunkel Mar 21 '14 at 09:23

score 0 · Answer 2 · answered Mar 17 '14 at 21:09

0

If your version of PHP can execute an individual file from the command line, you can always create a PHP command-line program to perform the decryption. Depending on your security settings, you may need to execute PHP through a shell program (php -q decrypt.php in the shell file) rather than directly (#!/usr/bin/php/cli/php or somesuch).

answered Mar 17 '14 at 21:09

Mark Leighton Fisher

5,609
2
18
29

That's the solution I use at the moment. But I like to be able to decrypt the files of my backup without needing my script. Thank you anyway! – Matthias Dunkel Mar 18 '14 at 08:18
So how do you think some command-line programs are implemented :)? Take a look at Puppet, for instance -- it is in Ruby. With the proper magic incantation at the front (`#!/usr/bin/perl` for example), you need never think about what language the command is actually implemented in. – Mark Leighton Fisher Mar 21 '14 at 21:11
Yes, I know how a shebang works. The goal was to not rely on my script to decrypt my backups. I wanted to encrypt with the PHP-Encryption methods, and then when I need my backup I want to be able to use mdecrypt (because it's more likely that I will have that handy when). Thanks for your help, but my problem has nothing to do with the shebang ;-) – Matthias Dunkel Mar 24 '14 at 12:04

How to encrypt big files with php and decrypt over commandline?

2 Answers2