17

I would guess most people on this site are familiar with tail, if not - it provides a "follow" mode that as text is appended to the file tail will dump those characters out to the terminal.

What I am looking for (and possibly to write myself if necessary) is a version of tail that works on binary files. Basically I have a wireless link that I would like to trickle a file across as it comes down from another network link. Looking over the tail source code it wouldn't be too hard to rewrite, but I would rather not reinvent the wheel! This wouldn't strictly be "tail" as I would like the entire file to be copied, but it would watch as new bytes were added and stream those.

Ideas?

Goyuix
  • 23,614
  • 14
  • 84
  • 128

9 Answers9

22

Pipe it to hexdump:

tail -f somefile | hexdump -C
Adam Pierce
  • 33,531
  • 22
  • 69
  • 89
  • I wasn't 100% sure myself but I tried it and it works just fine. – Adam Pierce Oct 24 '08 at 01:51
  • 1
    Wouldn't tail -f only output new data when it saw a newline in the binary file? I doubt it unbuffers its stdout. – Chris Young Oct 24 '08 at 01:59
  • 1
    Good point Chris, I didn't think of that. So I just tested it now on Debian and yes, it still works if there are no newlines in the stream although that behaviour might be different on different platforms. – Adam Pierce Oct 24 '08 at 03:33
  • 5
    The use of hexdump is a red herring, isn't it? Or perhaps just an illustration of somewhere to send the binary data. I don't see anything in the question asking for a hexdump, that's all... – Jonathan Leffler Oct 24 '08 at 06:44
  • Note hexdump only logs to the screen in increments of 16 bytes. – CletusW Jul 04 '20 at 06:27
6

There is also the bintail application which appears to be more robust than the aforementioned script.

The bintail package contains a single application, bintail. The program reads a normal file from disk, and pipes the output to stdout, byte-by-byte, with no translation, similar to what tail(1) does to text files. This is useful for "tailing" binary files, such as WAV files, while they are being written in realtime. This app is a work in progress, but it already does what it was designed to do for me.

Bin Tail
  • 76
  • 1
  • 1
  • Thank you, it is exactly what I needed, to redirect output from "tcpflow" to a nodejs stream :) It didn't work with "tail -f". – Drasill Oct 08 '13 at 19:21
  • 2
    In linux binutils `tail -c +1 -f somefile` works just as well. – ruief Jul 25 '17 at 13:52
5

Linux coreutils tail(1) works just fine on binary files. For most applications, you just need to avoid its line-orientation, so that the output doesn't begin in some random spot in the middle of a data structure. You can do that by simply starting at the beginning of file, which is also exactly what you asked for:

tail -c +1 -f somefile

works just fine.

ruief
  • 436
  • 6
  • 9
3

This hastily coded Python script for Windows may be of assistance:

# bintail.py -- reads a binary file, writes initial contents to stdout,
# and writes new data to stdout as it is appended to the file.

import time
import sys
import os
import msvcrt
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

# Time to sleep between file polling (seconds)
sleep_int = 1

def main():
    # File is the first argument given to the script (bintail.py file)
    binfile = sys.argv[1]

    # Get the initial size of file
    fsize = os.stat(binfile).st_size

    # Read entire binary file
    h_file = open(binfile, 'rb')
    h_bytes = h_file.read(128)
    while h_bytes:
        sys.stdout.write(h_bytes)
        h_bytes = h_file.read(128)
    h_file.close()


    # Loop forever, checking for new content and writing new content to stdout
    while 1:
        current_fsize = os.stat(binfile).st_size
        if current_fsize > fsize:
            h_file = open(binfile, 'rb')
            h_file.seek(fsize)
            h_bytes = h_file.read(128)
            while h_bytes:
                sys.stdout.write(h_bytes)
                h_bytes = h_file.read(128)
            h_file.close()
            fsize = current_fsize
        time.sleep(sleep_int)

if __name__ == '__main__':
    if len(sys.argv) == 2:
        main()
    else:
        sys.stdout.write("No file specified.")
Bin Tail
  • 31
  • 2
1

less somefile

Then press shift F

mhawke
  • 84,695
  • 9
  • 117
  • 138
1

Strictly speaking, you need to write a program to do this, as tail is not specified to work on binary files. There are also buffering issues you probably want to avoid if you want to receive the new "trickled" data as soon as possible.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • Well, looking again I saw that you tagged your question gnu-coreutils. So if you know you'll be using the gnu implementation of tail, it's probably binary safe and probably does not have problematic buffering (check and see). – R.. GitHub STOP HELPING ICE Oct 04 '10 at 17:18
0

This isn't tail -- this is progressively copying a file. Look at rsync.

wnoise
  • 9,764
  • 37
  • 47
  • I wonder that this answer is the accepted one where there are two answers which match much more the question: http://stackoverflow.com/a/6173419/1353930 http://stackoverflow.com/a/6171491/1353930. rsync is not helpful here because it cannot stream data. it is limited to (relatively static) files on the disk – Daniel Alder Nov 16 '15 at 10:05
  • @Daniel Alder. rsync can be rerun, sending only the new data. – wnoise Nov 16 '15 at 16:01
  • streaming means you are inside a cgi script run or piping to `netcat` etc. But that was more a question at @Goyuix – Daniel Alder Nov 16 '15 at 16:05
0

I use this as it works on live streams too:

cat ./some_file_or_dev | hexdump -C

sample dumping my key presses (and releases):

[user@localhost input]$ sudo cat /dev/input/event2 | hexdump -C
00000000  81 32 b1 5a 00 00 00 00  e2 13 02 00 00 00 00 00  |.2.Z............|
00000010  04 00 04 00 36 00 00 00  81 32 b1 5a 00 00 00 00  |....6....2.Z....|
00000020  e2 13 02 00 00 00 00 00  01 00 36 00 01 00 00 00  |..........6.....|
00000030  81 32 b1 5a 00 00 00 00  e2 13 02 00 00 00 00 00  |.2.Z............|
00000040  00 00 00 00 00 00 00 00  81 32 b1 5a 00 00 00 00  |.........2.Z....|
00000050  a3 af 02 00 00 00 00 00  04 00 04 00 36 00 00 00  |............6...|
00000060  81 32 b1 5a 00 00 00 00  a3 af 02 00 00 00 00 00  |.2.Z............|
^C
Anonymous
  • 2,122
  • 19
  • 26
0

I use this command (1 means the number of bytes to interpret) : tail -f | od -x1

Alex
  • 26
  • 4