gnuplot: plot input data obtained through pipe

Question

This is a bit of an odd query, and probably demonstrates my ignorance of bash more than being a simple request for help. Then again, someone may have the perfect answer -- which would save me a lot of head-scratching.

I wanted to create a simple gnuplot command to produce a plot, repeatedly for different data. Whilst at the moment I will probably have to save the data into a temporary file in /dev/shm, I don't really want to. It feels like unnecessary complexity, when what I would instinctively do is pass the data through a pipe. However, just when I thought I had it sussed (thanks to theozh) , experiments prove me wrong.

I start with the file stack.gp, as follows:

### bar chart with conditional color
data = '/dev/stdin'
lhb = ARG1
myColor(col) = (_s=strcol(col), _s eq lhb ? 0xff3333 : 0x3333ff)

set style fill solid 0.5
set key noautotitle

stats data using 0:2 nooutput
set label 1 at STATS_max_x, STATS_mean_y sprintf("mean=%.1f",STATS_mean_y) offset 0,0.7 right

plot data using 0:2:(myColor(1)):xtic(1) w boxes lc rgb var, \
     STATS_mean_y w l lw 2 lc "web-green"

As far as I understand things, the important thing here is:

1. The first line, where the incoming data is stored; and
2. The fact that this data is processed more than once (since I can easily do what I want, so long as I DON'T have the stats command).

The data is a very simple, two-column table:

A 8
B 6
C 4
D 3

and I can generate my output with the command:

< test.dat gnuplot --persist -c stack.gp 'C' 

Perfect! Except it isn't, because this uses a temporary file (test.dat). What I actually intend to do is pipe the data in, similar to this approach:

cat otherfile.txt | awk ... | gnuplot -c stack.gp 'C'

This doesn't work with the original script (no valid data points), but it DOES work if I remove my stats command so that the data is only processed once. So lets try a few other approaches:

1. data = '<cat'
2. data = '-'

(1) again works for a single pass, but otherwise gives "x range is invalid" (2) Either gives "x range is invalid" or "No valid data points found in file" depending on 1- or 2-pass processing. Arghhh!

So, is there another approach? Once could be the reason why I started using gnuplot in the first place, which is the perl helper script feedgnuplot. However, so far I haven't managed to get this working either.

The post https://unix.stackexchange.com/questions/671446 goes into a little more detail. Similar discussions: how to make several plots from the same standard input data in gnuplot?

Being stubborn, logic suggests that this is possible, because I can create random test data (using set table $Data) and then analyse that. If I can do that, then surely I can populate $Data with the contents of piped data? I'm obviously not populating this correctly when using stdin...

Is there a solution? Or is this just not possible with the current version of gnuplot?

Answer 1

I can think of at least two approaches off the top of my head. There are probably others.

Approach 1:

You could wrap the piped data in a start and end line to tell gnuplot to store it as a here document for later use. What gnuplot wants to see is this:

$DATA << EOD
 ... data ...
 ... data ...
 ...
EOD

That wrapping (additional first and last line) could either be done by the program generating the data or it could be added by piping through awk or some other set of command line bits. Here is a mock-up:

cat part2.gp
   print $DATA
   stats $DATA
   plot $DATA

echo '$DATA << EOD\n 2 1 \n 3 0 \n 4 4 \nEOD' | gnuplot '-' part2.gp

Approach 2:

Rather than piping through stdin, you can tell gnuplot to read from some other fd. Here is the relevant text from the manual ("help piped-data"):

 On systems with an fdopen() function, data can be read from an arbitrary file
 descriptor attached to either a file or pipe.  To read from file descriptor
 `n` use `'<&n'`.  This allows you to easily pipe in several data files in a
 single call from a POSIX shell:

       $ gnuplot -p -e "plot '<&3', '<&4'" 3<data-3 4<data-4
       $ ./gnuplot 5< <(myprogram -with -options)
       gnuplot> plot '<&5'

Answer 2

Here is a suggestion with a single pass for the data. You can calculate the mean value during the first plot command and plot the mean value in a 2nd and 3rd plot command with lines and with labels. With the single pass you should be able to get it to run together with the pipe where you have the data only once and you can get the variable lhb from the parameter ARG1.

Script:

### bar chart with mean value in a single pass
reset session

$Data <<EOD
A 8
B 6
C 4
D 3
EOD

# lhb = ARG1
lhb = "C"
myColor(col) = (strcol(col) eq lhb ? 0xff3333 : 0x3333ff)

set style fill solid 0.5
set key noautotitle
set yrange[0:]
set offsets 0.75,0.75,1,0

plot c=s=0 $Data u (c=c+1):(s=s+$2,$2):(myColor(1)):xtic(1) w boxes lc rgb var, \
     '+' u 1:(s/c) w l lw 2 lc "web-green", \
     '+' u (c):(s/c):(sprintf("mean=%.2f",s/c)) every ::::0 w labels offset 0,0.5 right
### end of script

Result:

Answer 3

The system() command returns the result of the execution as a string. This can be combined with "set print" to build DATABLOCK with the contents of the file on the fly. Applied to your script, it looks like this,

### bar chart with conditional color

lhb = "C"

# Read data in /dev/stdin into data block
set print $data 
print system("cat /dev/stdin")
unset print

myColor(col) = (_s=strcol(col), _s eq lhb ? 0xff3333 : 0x3333ff)

set style fill solid 0.5
set key noautotitle

stats $data using 0:2 nooutput
set label 1 at STATS_max_x, STATS_mean_y sprintf("mean=%.1f",STATS_mean_y) offset 0,0.7 right

set yrange[0:STATS_max_y*1.1]
set xrange[STATS_min_x-0.5:STATS_max_x+0.5]

plot $data using 0:2:(myColor(1)):xtic(1) w boxes lc rgb var, \
     STATS_mean_y w l lw 2 lc "web-green"

pause mouse