3

I have two data sets that I want to run a program on. I want to compare the results to running the analysis on each set individually, and on the combined data. Since they're large files, I'd prefer not to create a whole new file that's the two data sets concatenated, doubling the disk space required. Is there some way in Linux I can create something like a symbolic link, but that points to two files, so that when it's read it will read two other files in sequence, as if they're concatenated into one file? I'm guessing probably not, but I thought I'd check.

Colin
  • 10,447
  • 11
  • 46
  • 54
  • technically, a process could create a socket and simply spit out the two files through that socket. but then you'd have a process-per-conjoined-file, which would be rather painfully inefficient. – Marc B Sep 01 '15 at 17:48
  • I assume the app cannot read from stdin. You can write a custom FUSE driver (huge overkill for a simple usage, imho) – Karoly Horvath Sep 01 '15 at 17:59
  • http://unix.stackexchange.com/questions/94041/a-virtual-file-containing-the-concatenation-of-other-files – Karoly Horvath Sep 01 '15 at 18:02

1 Answers1

2

Can your program read from standard input?

cat file-1 file-2 | yourprogram

If your program can only read from a file that is named on the command line, then this might work:

yourprogram <(cat file-1 file-2)

I think you need to be running the /bin/bash shell for the second example to work. The shell replaces <(foobar) with the name of a named pipe that your program can open and read like a file. The shell runs the foobar command in another process, and sends its output in to the other end of the pipe.

Solomon Slow
  • 25,130
  • 5
  • 37
  • 57
  • Unfortunately the program takes the file name as an argument, so I think any solution would have to be at the OS level. – Colin Sep 01 '15 at 18:30
  • 1
    _the program takes the file name as an argument_ That's what my second example is for. If you are using the bash shell, and if your program only opens the file one time, reads it from beginning to end, and then close it, you can use my second example. – Solomon Slow Sep 01 '15 at 19:54