7Zip extraction on linux is so slow

Question

Extracting 7zip with many small files on linux xfs, extraction speed is so slow.

My Test System Environment.

System1 : 2vCore/4GB CentOS 6.5 64bit guest on XenServer.
System2 : 2vCore/4GB Windows 2012 guest on XenServer.

Large small files Creation Script

#!/bin/bash

for i in {1..100000}
do
    dd if=/dev/zero of=./dmp/dd$i.dmp bs=1K count=4
done

ok. I got 4KB files of 100,000 in dmp directory. and then I formatted /testfs with xfs.

Test

Compression Time on System1 :
   # time 7za a dmp.7z ./dmp     --> real   0m27.518s
Extraction Time on System1: 
   # time 7za x dmp.7z -o/testfs --> real   9m5.420s
   - 7za use only single core.
   - Dirent system call use most of the CPU time.
Extraction Time on System2:
   - around 25s
   - 7za use multi-core.

My Question

Although system2 use multi-core and system1 use single-core, such a big different result is unreliable. I think. How can I explain this ?
If I use xfs on linux for extracting 7zip file, How can I speed up ? any idea ?

Thanks in advance.

You're missing a *huge* variable here: IO. What do the storage systems of the two systems look like? — EEAA, Jul 14 '15 at 02:01
ok. Two systems share the same nfs storage repository that is abstracted by hypervisor. thanks. — fromcloud, Jul 14 '15 at 02:41
I have absolutely the same problem. 60 MB archive with small files takes minutes on Fedora and less than 1s on Windows. Some shit happens with p7zip on Linux. — Maxim, Jan 19 '20 at 23:33
And it is not related to file system. I have XFS, EXT4, with luks and without. Resut is the same. Considering that question is 4 years old, I have to switch to another archiving tool. — Maxim, Jan 19 '20 at 23:36

score 3 · Answer 1 · answered Jan 16 '23 at 16:22

3

I had the same issue, and the fix was to upgrade 7z. On Ubuntu 20.04 LTS, the 7z you get from apt install is from 2016. Instead, go download the latest version from the official website.

For the same archive with 60k small files on the same server, I went from 8 minutes to 6 seconds(!).

answered Jan 16 '23 at 16:22

bfontaine

133
11

1

The OP specifically specified CentOS vs. Windows. Giving an Ubuntu solution may or may not work. – doneal24 Jan 20 '23 at 21:36
@doneal24 From the question title and its tags I understand it to be about Linux in general and that CentOS was only used as an example. – bfontaine Jan 23 '23 at 15:05
1

This gives a 20x speed up in Ubuntu 18. – Kais Hasan Aug 16 '23 at 07:13

score 0 · Answer 2 · answered Mar 27 '23 at 12:40

Here's what I did to get Nautilus and Archive Manager (file-roller) in Ubuntu 22.04 to update to use 7-Zip 22.01:

Download https://7-zip.org/a/7z2201-linux-x64.tar.xz

In the Terminal (Ctrl-Alt-T) copy and paste the following commands:

sudo apt-get install p7zip-full
sudo 7z e 7z2201-linux-x64.tar.xz 7z2201-linux-x64.tar
sudo 7z e -o/usr/lib/p7zip -y 7z2201-linux-x64.tar 7zz 7zzs
sudo echo exec /usr/lib/p7zip/7zz '"$@"' > 7z
sudo mv 7z /bin/7z
sudo chmod ugo+rx /usr/lib/p7zip/7zz /usr/lib/p7zip/7zzs /bin/7z

7Zip extraction on linux is so slow

2 Answers2