45

EC2 machine on aws is filled with multiple linux headers and currently / is 100% full on disk space. I ran commands like apt-get autoremove but it is throwing an error saying

No apport report written because the error message indicates a disk full error
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

Can I remove the old headers by the rm command? Is it safe? and if I remove with rm command, should I do any clean up after that? Here is the list of headers:

du -h -d1 /usr/src/ | sort -hr
2.8G    /usr/src/
106M    /usr/src/linux-aws-headers-4.4.0-1099
106M    /usr/src/linux-aws-headers-4.4.0-1098
106M    /usr/src/linux-aws-headers-4.4.0-1096
106M    /usr/src/linux-aws-headers-4.4.0-1095
106M    /usr/src/linux-aws-headers-4.4.0-1094
106M    /usr/src/linux-aws-headers-4.4.0-1092
106M    /usr/src/linux-aws-headers-4.4.0-1090
106M    /usr/src/linux-aws-headers-4.4.0-1088
106M    /usr/src/linux-aws-headers-4.4.0-1087
106M    /usr/src/linux-aws-headers-4.4.0-1085
106M    /usr/src/linux-aws-headers-4.4.0-1084
106M    /usr/src/linux-aws-headers-4.4.0-1083
106M    /usr/src/linux-aws-headers-4.4.0-1079
106M    /usr/src/linux-aws-headers-4.4.0-1077
106M    /usr/src/linux-aws-headers-4.4.0-1075
106M    /usr/src/linux-aws-headers-4.4.0-1074
106M    /usr/src/linux-aws-headers-4.4.0-1072
106M    /usr/src/linux-aws-headers-4.4.0-1070
106M    /usr/src/linux-aws-headers-4.4.0-1069
106M    /usr/src/linux-aws-headers-4.4.0-1066
106M    /usr/src/linux-aws-headers-4.4.0-1065
106M    /usr/src/linux-aws-headers-4.4.0-1062
106M    /usr/src/linux-aws-headers-4.4.0-1061
106M    /usr/src/linux-aws-headers-4.4.0-1060
13M /usr/src/linux-headers-4.4.0-1100-aws
13M /usr/src/linux-headers-4.4.0-1099-aws
13M /usr/src/linux-headers-4.4.0-1098-aws
13M /usr/src/linux-headers-4.4.0-1096-aws
13M /usr/src/linux-headers-4.4.0-1095-aws
13M /usr/src/linux-headers-4.4.0-1094-aws
13M /usr/src/linux-headers-4.4.0-1092-aws
13M /usr/src/linux-headers-4.4.0-1090-aws
13M /usr/src/linux-headers-4.4.0-1088-aws
13M /usr/src/linux-headers-4.4.0-1087-aws
13M /usr/src/linux-headers-4.4.0-1085-aws
13M /usr/src/linux-headers-4.4.0-1084-aws
13M /usr/src/linux-headers-4.4.0-1083-aws
13M /usr/src/linux-headers-4.4.0-1079-aws
13M /usr/src/linux-headers-4.4.0-1077-aws
13M /usr/src/linux-headers-4.4.0-1075-aws
13M /usr/src/linux-headers-4.4.0-1074-aws
13M /usr/src/linux-headers-4.4.0-1072-aws
13M /usr/src/linux-headers-4.4.0-1070-aws
13M /usr/src/linux-headers-4.4.0-1069-aws
13M /usr/src/linux-headers-4.4.0-1066-aws
13M /usr/src/linux-headers-4.4.0-1065-aws
13M /usr/src/linux-headers-4.4.0-1062-aws
13M /usr/src/linux-headers-4.4.0-1061-aws
13M /usr/src/linux-headers-4.4.0-1060-aws

df command output:

df -k
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              499316       0    499316   0% /dev
tmpfs             101444   10740     90704  11% /run
/dev/xvda1       8065444 8049060         0 100% /
tmpfs             507212       0    507212   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             507212       0    507212   0% /sys/fs/cgroup
/dev/xvdf        5029504   10244   4740732   1% /data
tmpfs             101444       0    101444   0% /run/user/1000
Deepak
  • 3,134
  • 2
  • 24
  • 24
hithyshi
  • 713
  • 1
  • 5
  • 11
  • Usually (in properly baked distributions) those are provided by packages. You may check using either `apt` or `dpkg` in Ubuntu what's going on. And it's safe to purge (remove) redundant **packages**. In your case I would rather to see if there is any `apt` cache left and start from cleaning it (without using `rm` command). – 0andriy Feb 16 '20 at 12:07
  • Thanks @0andriy , is there a specific command that you know for checking `apt` cache and cleaning it? – hithyshi Feb 16 '20 at 12:27
  • @hithyshi Install the `apt-file` command and then run `apt-file search /usr/src/linux-aws-headers-4.4.0-1099` to find the package responsible for those files. – Marco Bonelli Feb 16 '20 at 16:23
  • 1
    Thanks @0andriy and thanks Marco Bonelli . I did a few things and it has helped at the moment to clear up some space. I deleted manually 12 redundant headers with rm command. Then ran the purge commands : example - `rm -rf linux-aws-headers-4.4.0-1060` and then `sudo apt-get purge linux-image-4.4.0-1060-aws`. I did the above for 12 redundant headers. This has cleared a lot of space for now. Will have to take steps to make sure this wont repeat – hithyshi Feb 16 '20 at 16:57

6 Answers6

68

To delete linux headers safely, always use:

$ sudo apt-get autoremove

Sometimes this command can give error like this:

Reading package lists... Error!
E: Could not create temporary file for /var/cache... (28: No space left on device)
E: The package lists or status file could not be parsed or opened.

In this case, delete one or two old headers manually. To do this first go to /usr/src folder and list the files according to date (oldest last) using below commands:

$ cd /usr/src
$ ls -lt

Then choose an old header (e.g. linux-aws-headers-4.4.0-1067) and delete it:

$ sudo rm -r linux-aws-headers-4.4.0-1067

After doing this, the command sudo apt-get autoremove should work without any problem.

Deepak
  • 3,134
  • 2
  • 24
  • 24
  • 1
    This was helpful, as I was getting errors on one specific header while running sudo apt autoremove. I just manually removed it as described here, and then reran autoremove without any issue. – mr.adam Mar 16 '21 at 17:07
15

This command worked for me:

 apt-get -f autoremove
jpereira
  • 648
  • 7
  • 12
4

Context

I was getting the same error as OP. To be able to run the command suggested in jpereira's answer (apt-get -f autoremove), I first had to free about 10MB of space.

But when I ran it, something weird happened where apt-get said it was going to remove all the linux-headers (and some other) packages, but it first tried to install another package (snapd), but because there was no space to install it, the process failed.

The next time I tried running the command, it was as if apt-get had forgotten about the linux-headers (and the other) packages. So I went ahead and attempted a manual removal of one of the packages:

sudo apt-get remove linux-aws-headers-4.4.0-1065

This command, successfully removed 2 directories from the /usr/src directory. It removed the directory with the same name as the package and the package of the same name but with the -aws moved to the end.

To remove all of them I ran this command, BUT DO NOT USE IT because I ended up removing some packages that would have not been removed by the autoremove functionality and I also did not remove all the packages that would have been removed by the autoremove functionality.

cd /usr/src
while read pkg; do sudo apt-get -y remove $pkg; done < <(find . -maxdepth 1 -type d -name "linux*" -printf "%f\n")

Recommended Solution

The perfect solution would have been to run the following commands, but you first need to copy the list of modules that autoremove initially said that it was going to remove:

echo "<PASTE COPIED LIST OF PACKAGES HERE>" | cut -c3- | tr " " "\n"

Make sure that the output of that command is a list of the packages where each package is in one line. If that works, then go ahead and run something similar to the command I used before:

while read pkg; do sudo apt-get -y remove $pkg; done < <(echo "<PASTE COPIED LIST OF PACKAGES HERE>" | cut -c3- | tr " " "\n")

More context

Running the recommended solution (after running the not recommended one) was also good for me because that removed some linux-image and linux-modules packages that were also occupying a lot of space but somewhere else!

The first command (not recommended one) freed about 3.7G of space and the second one (recommended one) freed an additional 1.7G. However, if you first run the recommended command, that command alone should free all the space that needs to be freed.

alejandro
  • 794
  • 7
  • 9
1

I automated removal of the unneeded directories with this shell code:

sudo apt-get autoremove
cd /usr/src
sudo rm -r $(comm -23 <(ls /usr/src) <(apt-file search /usr/src/* | awk -F / '{print $4}' | sort -u))

The last line deletes every directory that is present in ls /usr/src but is not present in a list of owned files in that directory. The command ended up deleting all but the most recent directory. Before running for real, I suggest changing rm to echo rm to see what would be removed.

piojo
  • 6,351
  • 1
  • 26
  • 36
1

I first repaired package state with dpkg and apt-get -f install.

Then i tried to remove old kernels as suggested via apt-get autoremove.

But this would only remove one kernel. There where many more installed that i found with

dpkg --get-selections|grep linux-modules|grep -v deinstall

I had to explicitly remove those kernels:

apt-get remove linux-image-4.4.0-1123-aws linux-image-4.4.0-1121-aws linux-image-4.4.0-1119-aws linux-image-4.4.0-1114-aws ... linux-modules-4.4.0-1124-aws  linux-modules-4.4.0-1121-aws linux-modules-4.4.0-1119-aws linux-modules-4.4.0-1114-aws ...

Afterwards i could clean up using apt-get autoremove which removed headers.

Some leftovers where reported so i had to clean them manually:

cd /lib/modules
rm -rf 4.4.0-1124-aws 4.4.0-1123-aws 4.4.0-1122-aws 4.4.0-1121-aws4.4.0-1119-aws 4.4.0-1118-aws 4.4.0-1117-aws 4.4.0-1114-aws ...

I still don't know why apt-get autoremove didn't work for me.

fheyer
  • 126
  • 1
  • 4
0

I tried apt-get -f autoremove but there is not even space to free space :(

I had to delete 02 headers (the old ones) manually

cd /usr/src
ls -la 
rm -rf  linux-aws-5.4-headers-5.4.0-abcd

After that, I ran again apt-get -f autoremove but a new error appeared:

Reading package lists... Done
Building dependency tree
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 linux-headers-5.4.0-ABCD-aws : Depends: linux-aws-5.4-headers-5.4.0-FGHI but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

As log says, I ran: apt --fix-broken install and then apt-get -f autoremove worked

JRichardsz
  • 14,356
  • 6
  • 59
  • 94