6

How do you create an empty tar file in AIX?

$touch myfile.tar doesn't work.

Roland
  • 7,525
  • 13
  • 61
  • 124
  • It would seem POSIX does not allow empty tar files (`tar -c -f empty.tar` without at least one file name complains about missing files). Is this an x-y-problem? What is your problem solved with an empty tar file? – Jens Jan 19 '23 at 16:56
  • @Jens I need to run several commands and add the returned files to a tar archive. Problem is that each of the commands can return 0 files. The second command would add files to the tar archive. The simplest solution would be to create an empty tar at the beginning and just add to it. Otherwise I will need to check with if-then – Roland Jan 20 '23 at 12:45
  • You can create an empty tar file with `dd if=/dev/zero of=empty.tar bs=512 count=1` but you cannot append files to it with AIX!tar (with Gnu!tar you can). The trivial workaround is adding a dummy file such as `ReadMe.TXT` containing some text like _files are restored from tar-archive_ – Lorinczy Zsigmond Jan 21 '23 at 13:12
  • And at the end of the process you can remove the first 1024 bytes of the archive with `dd skip` – Lorinczy Zsigmond Jan 21 '23 at 13:22

4 Answers4

3

Just creating an empty file will not be recognized as a tar-file. To be recognized as a tar-file, it must satisfy a particular format. There are multiple tar-file formats. An overview can be found here.

The IBM AIX operation system seems support the United States Tape Archiver (USTAR) format as well as the old UNIX format [AIX 1] and does this trough the command tar [AIX 2]. It can also deal with other interchangable formats such as pax [AIX 3] and cpio [AIX 4] via the commands pax and cpio. The USTAR format and the interchangable formats, pax and cpio, are all POSIX defined [POSIX 1]

Not directly supported by AIX, but worth mentioning is the wide-spread GNU-tar format [GNU 1].

All mentioned archiving formats have a similar format: a sequential list of file objects (meta-data and file content) followed by two zero-filled blocks (a block is 512 bytes big).

This means that, in order to create an empty tar-ball, you need to have no file objects but two zero-filled blocks. In other words, have minimally 1024 zero bytes in your file. The following two commands generate such file:

$ dd if=/dev/zero of=empty.tar bs=512 count=2
$ head --bytes 1024 /dev/zero > empty.tar

Note: there is something called blocking which just tells the system to perform IO-operations in chunks of BLOCKING_FACTOR * 512 bytes. In most systems (GNU, AIX) this value is set to 20 and unneeded garbage bytes are converted to zero's. That is why an empty file, generated with GNU tar is 10240 bytes big.

$ tar -cf empty.tar --files-from=/dev/null
$ stat -c "Size: %s" empty.tar
Size: 10240
$ od empty.tar
0000000 000000 000000 000000 000000 000000 000000 000000 000000
*
0024000
kvantour
  • 25,269
  • 4
  • 47
  • 72
2

You can use the POSIX pax command for generating an empty TAR archive:

pax -x ustar -w < /dev/null > empty.tar

Different OSs will generate different file sizes (ranging from 1024 to 10240 Bytes in my tests), all of those containing only NUL Bytes. Hence, an other possibility would be to use dd for generating the empty TAR file.

remark: With the POSIX TAR header being defined as 512 Bytes, I don't understand why the generated files are bigger than that.


When you "untar" any of those files with:

tar xvf empty.tar

It gives no output, no error, and doesn't create any file nor directory.

Fravadona
  • 13,917
  • 1
  • 23
  • 35
  • I can create the empty tar file, but when I run: "tar -rf empty.tar a.txt" I get: "tar: Unexpected end-of-file while reading from the storage media." – Roland Jan 20 '23 at 12:39
  • 1
    @Fravadona The reason why there are bigger files is because tar is optimized for IO-handling on tapes. Apparently it is more efficient to always read/write 20*512 bytes = 10240 bytes in a single go. So no matter what, they always write in chunks of 10240 bytes. If less bytes are needed, they fill it with zeros. So the tar archive by `pax` or `gnu tar` is 10240 bytes big, but only the first 1024 bytes are used as the end-of-archive indicator. The remaining 9210 bytes are just garbage and not looked at. – kvantour Jan 25 '23 at 13:12
  • @Roland in the end, you weren't only trying to create an empty TAR file, you were also looking on how to add a files to it. You should have asked an other question referencing this one. – Fravadona Jan 26 '23 at 10:02
  • @Fravadona For me it was always implicit that you should be able to add files to the empty tar archive. I didn't consider the possibility that it is possible to create a valid empty tar file in AIX yet AIX!tar will not be able to add files to it. – Roland Jan 26 '23 at 14:06
0

An empty file won't do, it has to contain some zeroes (n*512 bytes) -- this comes from tar being tape archiver. (This zeroes are interpreted as EOF, that's why you cannot simply concatenate the output of more tar c commands.)

A file containing n*512 zeroes can be used with t (toc) and x (extract) operations, but not whith r (append) operation (at least with AIX's default tar).

Still you can create an archive containing a dummy file, which you can even remove at the end of the process with dd. Example script:

#!/bin/sh

set -ev

export PATH=/usr/bin

# create 'collect_tmp.tar' with a dummy file in it

echo 'These files have been restored from a tar-archive.' >ReadMeFromTar.TXT
tar -cvf collect_tmp.tar ReadMeFromTar.TXT
rm ReadMeFromTar.TXT
ls -l collect_tmp.tar

if [ $# -gt 0 -a "x$1" = 'xempty' ]; then
    echo empty
else
    # add some files into it 'collect_tmp.tar' with a dummy file in it

    tar -rvf collect_tmp.tar /etc/motd
    tar -rvf collect_tmp.tar /etc/hosts
    tar -rvf collect_tmp.tar "$0"

    tar -tvf collect_tmp.tar
fi

dd if=collect_tmp.tar of=collect.tar bs=1024 skip=1
rm collect_tmp.tar

tar -tvf collect.tar

Notes:

  1. The error message you get from AIX!tar when trying to append an empty (or containing n*512 zeroes) tar file: r tar: Unexpected end-of-file while reading from the storage media.

  2. Why 1024 bytes: tar adds 512 bytes long header (or a multiple of 512 in special cases) before the content of the file, and it stores the content padded with zeroes to n*512 bytes. So an empty file takes 512 bytes, a short file (like above) takes 1024 bytes in the tar-archive.

Off-topic: GNU!tar is way superior to AIX!tar (you can say it is over-featured), but is still adds these tailing zeroes (because of POSIX-compatibility, tape-compatibility, inertia) which is no problem, unless you want append to a compressed archive -- the zeroes between the old and new files won't be removed. Workaround:

/opt/freeware/bin/tar -b 1 ... -cf - |\
/opt/freeware/bin/head -c -1024 |\
/opt/freeware/bin/xz >>appendtome.tar.xz
Lorinczy Zsigmond
  • 1,749
  • 1
  • 14
  • 21
  • Do you know why append on empty tar doesn't work on AIX? – Roland Jan 22 '23 at 14:43
  • How can you be sure that the dummy file will occupy exactly 1024 bytes in the tar? – Roland Jan 22 '23 at 14:48
  • After you remove the file from the tar archive it will have less than 10240 bytes. This might be a problem because each record in a tar file should have 10240 bytes. So maybe it would be a good idea to pad the tar file with 1024 zeros? https://www.gnu.org/software/tar/manual/html_node/Blocking-Factor.html – Roland Jan 23 '23 at 15:34
  • 1
    Nope, the size can be any multiple of 512 bytes. (Also the tailing m*512 byte zeroes are completely useless (maybe except for actual magnetic tape?)) – Lorinczy Zsigmond Jan 24 '23 at 04:08
  • If you are going to remove the file afterwards you could make it empty in the beginning. In this case you should only remove 512 bytes, right? That's what I'm doing in my solution. – Roland Jan 24 '23 at 13:54
  • Sounds okay -- to tell the truth I wouldn't bother with a dummy file, it doesn't require _so_ much resource if you keep it. – Lorinczy Zsigmond Jan 24 '23 at 14:12
  • 1
    It consumes attention/work once you have to explain to other people where this mysterious file comes from. – Roland Jan 24 '23 at 15:12
  • "A file containing 512 zeroes" Actually it has to be two blocks of 512 zeros. See https://www.gnu.org/software/tar/manual/html_node/Standard.html – Roland Jan 25 '23 at 13:31
  • 1
    Well GNU manual isn't necessarily authentic regarding AIX!tar. (Also GNU has no problems with working with an actually empty file.) – Lorinczy Zsigmond Jan 25 '23 at 13:38
  • @LorinczyZsigmond The POSIX standard of USTAR (the archiving format used by AIX tar) states that a tar-file must end with at least 2 times 512 bytes of zero's. Furthermore, if any of the files you have is bigger than 512 bytes, your `dd` will fail (512 bytes of head + blocksize of 512, leads to minimally 1024 bytes per file) – kvantour Jan 25 '23 at 17:51
  • @kvantour Sorry, I don't get your point regarding `dd`; what I meant is starting the archive with a dummy file of size 1..512, which takes altogether 1024 bytes (header included). Then in the final step the first 1024 file is skipped by `dd` -- the other files in the archive are unaffected, so are the leading zeroes. – Lorinczy Zsigmond Jan 25 '23 at 19:23
-1

Try this. Works in Linux.

tar cvf your-empty-tar-filename.tar --files-from /dev/null
bluish
  • 26,356
  • 27
  • 122
  • 180
Rahul
  • 76,197
  • 13
  • 71
  • 125