0

I ran a MR job that filled up a bit of space while testing. It Almost 20 GB space has been taken by running the Job. I have deleted a number of hive tables and hdfs files. But, I still run into errors such as failing the parcel directory thresholds.

Is there any Cloudera Manager command that will clean unused space across all nodes? Or is there some other method I should be using to make free space?

Container [pid=10156,containerID=container_1511328141399_0037_01_000004] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 23.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1511328141399_0037_01_000004 :
User007
  • 113
  • 8

1 Answers1

0

First, your error is about memory, not disk space. You needed 8GB of RAM to have the Cloudera Express desktop app to start, unless you forced it to start otherwise

Deleting HDFS files still keeps them around in a Trash location in a user's home folder.

Hue has an option to empty the trash, but the command line option is 

hdfs dfs -expunge

https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/FileSystemShell.html#expunge

Note that you could also manipulate the VirtualBox hard drive to be larger, which is outside the scope of the quickstart VM. It's not clear if you're using the Docker version, though

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245