File size with zfs compression

Question

I usually estimate the size of a whole directory tree using du -ks $DIRECTOY_TREE_ROOT, but this method cannot be used when zfs compression is on.

The total displayed by ls -l is ok for a single directory, but which is the simplest way to get the same result for a directory tree?

EDIT:

Operating system is Solaris 10.

I am looking for real file size, not the space used on disk.

Are you looking for actual space used on disk, or true file size? — justarobert, Mar 31 '11 at 08:38

score 13 · Answer 1 · edited May 18 '18 at 20:12

13

Just use du -b example:

# du -sh .
215G    .

# du -sbh .
344G    .

edited May 18 '18 at 20:12

code_dredd

156
1
1
11

answered May 16 '11 at 21:03

Woyteck

131
2

1

Twelve up-votes despite the fact there is no "-b" option with Solaris 10 `du`... – jlliagre Dec 14 '18 at 15:11
1

On Solaris 11, it works via `gdu -bsh foldername` – Copy Run Start Apr 02 '19 at 15:29
@jlliagre Everyone knows that the only computer configuration in the entire universe is an x86 GNU/Linux machine. – anahata Jun 15 '23 at 00:21

score 5 · Accepted Answer · answered Apr 02 '11 at 10:07

5

This should just work:

find . -type f -exec ls -l {} + | nawk '{s=s+$5}
END {print s}'

answered Apr 02 '11 at 10:07

jlliagre

8,861
18
36

score 3 · Answer 3 · answered Dec 30 '11 at 11:47

It is possible to get both file size and approximate disk usage direcly from command 'find' with the parameter '-ls'

 function lsdu() (
    export SEARCH_PATH=$*
    if [ ! -e "$SEARCH_PATH" ]; then
        echo "ERROR: Invalid file or directory ($SEARCH_PATH)"
        return 1
    fi
    find "$SEARCH_PATH" -ls | gawk --lint --posix '
        BEGIN {
            split("B KB MB GB TB PB",type)
            ls=hls=du=hdu=0;
            out_fmt="Path: %s \n  Total Size: %.2f %s \n  Disk Usage: %.2f %s \n  Compress Ratio: %.4f \n"
        }
        NF >= 7 {
            ls += $7
            du += $2
        }
        END {
            du *= 1024
            for(i=5; hls<1; i--) hls = ls / (2^(10*i))
            for(j=5; hdu<1; j--) hdu = du / (2^(10*j))
            printf out_fmt, ENVIRON["SEARCH_PATH"], hls, type[i+2], hdu, type[j+2], ls/du
        }
    '
)

Some sample command and output:

-bash-3.00# lsdu test_sloccount/
Path: test_sloccount/ 
  Total Size: 30.90 MB 
  Disk Usage: 1.43 MB 
  Compress Ratio: 21.6250

score 2 · Answer 4 · answered Mar 31 '11 at 08:13

2

This oneliner should produce the desired result:

find $DIRECTOY_TREE_ROOT -type d -exec ls -l '{}' \; | awk '/^total\ .[0-9]+$/ { sum+=$(NF) }END{ print sum }'

I don't have a ZFS partition to test it on, but on my ext4 partition it outputs the same result as du -ks.

answered Mar 31 '11 at 08:13

Kenny Rasschaert

9,045
3
42
58

The question has been edited to ask for the actual files size, not the one used on disk which both du and ls total are reporting. – jlliagre Apr 02 '11 at 12:53

score 2 · Answer 5 · answered Mar 31 '11 at 09:20

2

man du would probably help here:

 --apparent-size
      print apparent sizes, rather than disk usage;  although
      the  apparent size is usually smaller, it may be larger
      due to holes in (`sparse') files,  internal  fragmenta-
      tion, indirect blocks, and the like

answered Mar 31 '11 at 09:20

the-wabbit

40,737
13
111
174

These is no such option os Solaris 10 du. It is a non standard Gnu extension. – jlliagre Apr 02 '11 at 10:07
Might be. Solaris Express 11 does have it, though. – the-wabbit Apr 08 '11 at 20:15
4

Solaris 10 doesn't bundle Gnu du. Solaris 11 Express has both /usr/bin/du that doesn't support the --apparent-size option and /usr/gnu/bin/du that supports it. – jlliagre Apr 08 '11 at 20:41

score 2 · Answer 6 · answered Feb 06 '16 at 19:55

I'm going to include the answer to this question for FreeBSD for sake of completeness. According to man du:

 -A      Display the apparent size instead of the disk usage.  This can be
         helpful when operating on compressed volumes or sparse files.

File size with zfs compression

6 Answers6

Linked