1

I have a system with over 10,000,000 files where I want to track the disk usage of a couple of users (these users own the millions of files). Based on the following Q&A https://stackoverflow.com/questions/4307692/fastest-way-to-calculate-directory-sizes I want to look into linux disk quotas for this job since the du command also takes over an hour to run on my system.

My question is:

1) What is the expected run time of repquota? is it always very fast (eg. run in under 1 minute)? or does it depend on the number of files and disk size being tracked?

2) do I need to run quotacheck to update the quata information (files? database?) to change the output of repquota? or will repquota always give me up-to-date information?

Update

Example for 2): If I run the following commands what is the expected output:

repquota /tmp
head -c 1024 /dev/urandom > /tmp/new.file
sleep 1
repquota /tmp

3) Will the second repquota /tmp give the same output as the first? or will the quota info be updated only because I wrote to the disk (and not based on running some other background quota updating program)?

Community
  • 1
Alex Q
  • 135
  • 1
  • 8

1 Answers1

3

  1. Repquota should be almost instant, however doing a quota check won't be. On a server with about 500k files 4 disk RAID0 it took around half an hour, however that was with fairly high disk load (50% or so). It does depend on the number of files, not so much disk size as disk speed.

  2. repquota reports based on the quota file (which is updated when quota runs, i believe roughly every 5 minutes), if you want complete up to date information then running quotacheck is a good idea. Bear in mind you need to turn off quota to do that.

We use the following script:

#!/usr/bin/php
<?php
echo date('Y-m-d H:i:s') . ': Checking quota and fixing' . "\n";

if (file_exists('/usr/local/bin/quota')) {
 `rm -rf /usr/bin/quota`;
 `ln -s /usr/local/bin/quota /usr/bin/quota`;
}

`quotacheck -avugm`;
sleep(1);
`quotaon -av`;
`repquota -as`;
Joshua D'Alton
  • 428
  • 3
  • 13
  • 2
    Thank you, this is very helpful. The is one thing that I'd like you to please clarify for me. How does the quota data get updated? and how often? You said "doing a quota check ... took around half an hour" and also "the quota file ... is updated when quota runs, i believe roughly every 5 minutes" These two statements are contradictory to me. How do you run the quotacheck program every five minutes if it takes 30 minutes to run? – Alex Q Mar 11 '11 at 05:37
  • Ah yea it turns out the 30 minutes was because a disk was about to die so performance was low, you are correct that normally it is very quick such that every 5 minutes or so it can update. As for how quota gets updated and how often, it is a crontab of some sort I believe, I can find out more if you want – Joshua D'Alton Mar 16 '11 at 03:32
  • Could you please find out how quota gets updated, or alternatively just run the few commands from my update to the question and post if the output is the same or not, and if it's the same does sleeping for longer (eg. 600 =10min) change the output? Thanks. – Alex Q Mar 16 '11 at 23:47
  • I ran it once with sleep 1, it went from 82 to 83 files, block count stayed the same. Then ran with new2.file and sleep 60, went from 83 to 84, same block count. Does that help? – Joshua D'Alton Mar 18 '11 at 02:12
  • Very late comment, can't actually remember how I knew this, but glad I could help! There have been some small changes to a couple distros on how this works, but none of the main ones you probably use ie debian. – Joshua D'Alton Mar 22 '13 at 14:30