0

Reg HDFS Space Quota - Cloudera

If I have a folder A in HDFS and inside A there is another folder B.

Space Quota for A - 100 MB

Space Quota for B - 200 MB

If I try to copy a file of size 150MB to B, is to going to fail?

Sidharthan PV
  • 38
  • 5

1 Answers1

1

According to Hadoop documentation:

The space quota is a hard limit on the number of bytes used by files in the tree rooted at that directory.

So in general, you can't put a file larger than the quota of a parent directory into a sub-directory or any of its children.

In your specific case, the largest file you can put into B is 100MB, on two conditions:

  1. replication factor for the file is set to 1, and
  2. dfs.blocksize of the file is =100MB (or dfs.blocksize x N = 100MB) since according to the same document

Block allocations fail if the quota would not allow a full block to be written.

mazaneicha
  • 8,794
  • 4
  • 33
  • 52