0

stream create --name TailTest --definition "tail --name=/Users/name/Documents/SpringXD/PoC/input/Try.txt --outputType=text/plain --lines=1 |file --name=output --dir=/Users/name/Documents/SpringXD/PoC/output --mode=APPEND" --deploy

Here I see that whether I give --lines=1 or 2 or 0 as soon as there is an additional line entered in Try.txt and it is saved, it is reading entire Try.txt and dumping it into output file. Again when I add a line to Try.txt it does the same reads entire file content rather than just last line and appends it at the end of the previous content in output, any idea what could be going wrong here, why does tail does not read only last 1 (or specified n) lines I do put enter at the end of each line in input file?

Mogsdad
  • 44,709
  • 21
  • 151
  • 275
user3754631
  • 3
  • 1

1 Answers1

1

It sounds like you are replacing the file when you say "and it is saved".

Tail follows an existing file and, yes, it expects a newline to terminate a new message.

I just ran a test and it works fine...

xd:>stream create --name foo --definition "tail --lines=1 --name=/Users/foo/Documents/foo.txt | log" --deploy

logs the last line of the file,

23:15:58,925  INFO SimpleAsyncTaskExecutor-1 sink.foo - sdsd

then 

$ echo foo >> ~/Documents/foo.txt

(which appends foo\n to the file)

results in

23:17:07,744  INFO SimpleAsyncTaskExecutor-1 sink.foo - foo

The lines=1 only applies to an existing file; if you replace the file with a new one, the whole file is used - this is like log rotation.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Thanks. Works fine with echo now. Earlier in Mac if I add lines to a file and hit enter at the end of the line and then save that file, it does not seem to help to tail stream. When I used echo to write lines to the file that Tail stream is consuming it works fine. I guess ENTER key in mac does not seem to induce a newline character at the end that caused this issue. – user3754631 Jun 19 '14 at 19:21