Grep not matching from serial input

Question

This is a sister question to my other question on a similar topic: I have a few commands, part of a larger script, that looks like this (one of many iterations):

stty -F /dev/ttyUSB0 115200 -echo raw -icrnl iutf8
cat /dev/ttyUSB0 | tee /dev/tty | grep --max-count=1 -F "root@ramfs:"

The intent here is that when an embedded device, connected via RS232 serial-USB cable, finishes booting, it will output the prompt that I am grepping for, and grep will find that prompt and continue in script execution. I am not sure what is going wrong however, as when I can see that target string printed to my terminal, the command continues and grep does not find a match. Some sample output is below (from various tries with different grep and stty options):

[[55;100R[    9.177532] macb ff0c0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx
[    9.185137] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
root@ramfs:~#

[[55;100Rroot@ramfs:~# [   18.393491] macb ff0c0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx
[   18.401103] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
                                                                                                   root@ramfs:~# [   18.393491] macb ff0c0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx
^[[55;100R

[[55;100Rroot@ramfs:~# [   19.417479] macb ff0c0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx
[   19.425087] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
                                                                                                   root@ramfs:~# [   19.417479] macb ff0c0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx
^[[55;100R

I can understand how the first example would not generate a match, as the target string only appears on the last line, and it would seem that grep will not operate on raw data from what I gather in other questions, meaning that another newline would be needed to generate a match. I do not understand, however, why the next two examples do not generate a match, as the target strings clearly appear within their own line, followed by a newline.

I am hoping someone can point out what I am doing wrong here/why it does not work like I would expect (I would expect at least the 2 and 3 examples would work), or suggest a more appropriate alternative that does not require so much coercion in this case. I would also be open to tips on troubleshooting this, as I am not really sure where I would start (printing everything as raw hex bytes is my first thought, but that does not seem like an efficient way to go about it).

===================================================================== Edit: Below is the output of cat /dev/ttyUSB0 > serialdump followed by cat -TEv serialdump

^M^M$
^[7^[[r^[[999;999H^[[6nroot@ramfs:~# [   17.401463] macb ff0c0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx^M$
[   17.409067] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready^M$

And matching with cat serialdump -TEv | grep root@ramfs

$ cat serialdump -TEv | grep root@ramfs
^[7^[[r^[[999;999H^[[6nroot@ramfs:~# [   17.401463] macb ff0c0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx^M

I haven't used a ttyUSB ever, but what are the chances that the things you see as being spewed out on individual lines are in fact on **STDERR**, not **STDOUT**, and the prompt still needing a `\n`? — tink, Mar 15 '23 at 03:58
"*Some sample output is below (from various tries with different grep and stty options): ... why the next two examples do not generate a match*" -- So these samples are worthless without knowing the exact commands that produced them. "*stty -F /dev/ttyUSB0 115200 -echo raw -icrnl iutf8*" -- `raw` already specifies `-icrnl`. `iutf8` seems unreasonable/unnecessary. "*it would seem that grep will not operate on raw data*" -- True, the man page for **grep** clearly states that it scans *lines* rather than unterminated text. — sawdust, Mar 15 '23 at 04:17
@tink good question, stderr/out mixing shouldnt be the problem here; it has worked intermittently before and I am trying to find a more stable solution; the embedded device I am using outputs both to the same fd on its end (the serial port) with no distinction between them afaik — Douglas B, Mar 15 '23 at 04:26
@sawdust those options to `stty` were a scattershot indeed but not the first thing I had tried, and if it is simply unnecessary I doubt it is the problem, but thank you for pointing that out. I am not sure why those samples are 'worthless' though. They are copied directly from the bash script output, the entire bash script in the final unworking form is shown. The output is clearly stated to be the boot sequence of an embedded device over a serial port read as a file, the data being the boot sequence of linux. I am not sure if you are asking for kernel source for the supplied boot seq? — Douglas B, Mar 15 '23 at 04:35
@sawdust [that quote](https://stackoverflow.com/questions/75740272/grep-not-matching-from-serial-input#comment133612481_75740272) is unrelated to `@` appearing in a regexp as in this case. There's nothing special about `@` in a regexp and so it doesn't need to be escaped. If the OP was using `$@` within double quotes then the `$` would need to be escaped to stop `$@` resulting in parameter expansion, but still not the `@`. — Ed Morton, Mar 15 '23 at 14:02
You probably have some control chars among the `root@ramfs:`, e.g. maybe for making part of it bold or colored or something. Try saving the pre-grep output to a file and then running `cat -TEv` or whatever your version of `cat` supports (or `od -c`) to see any control chars in that line. Also run your grep command on that file to see if it matches or not. — Ed Morton, Mar 15 '23 at 14:10
@EdMorton-SOstopbullying thank you for this information. I have edited my post after trying the troubleshooting you suggest, you can see from my edit that it does seem to be able to match when the output is redirected to a file first, oddly enough. I assumed there were some non-printing characters causing trouble (it seems to me that ^[ is an escape sequence for instance) but I am still unsure what makes the behavior different between the serialdump file and the direct tee/pipe — Douglas B, Mar 15 '23 at 17:18
Doesn't seem to be any control chars within the string you're grepping for, `root@ramfs:`. Maybe it's a buffering issue? Maybe the string you're looking for sometimes doesn't have a terminating newline when your grep is looking for it? Try taking a look at `stdbuf` (e.g. at https://stackoverflow.com/q/11337041/1745001) to see if it'd help with buffering. — Ed Morton, Mar 15 '23 at 17:44
Hang on - I just noticed `^M` at the end of some of your output lines. Is it possible the grep is actually producing output but the CR at the end is backing your cursor over it so it LOOKS like there's no output? — Ed Morton, Mar 15 '23 at 17:47
@EdMorton-SOstopbullying Thank you for your replies, I tried setting the option `igncr` and had success, but only once so Im assuming that was another fluke. My current solution is to `cat $OUTPUT_TERM | tee -a tmp.tmp &` and then grep tmp.tmp in a loop every second checking for the output, which does work. Feels hacky/bad and I do wish I was able to get to the bottom of what was interfering with greps ability to find targets in my output shown in the 2/3 example (where missing newline after target wouldnt be the issue), but I think it makes sense to delete this question, thoughts? — Douglas B, Mar 15 '23 at 20:41
Well, it hasn't received any downvotes or close votes yet so maybe leave it up a day or so to see if you get any replies? If you don't get any answers here you might want to try posting it at https://unix.stackexchange.com, I see more of the nitty gritty Unix stuff being addressed there than here. Obviously if you could figure out a way to post a [mcve] with a complete script we could execute to test with that'd be enormously useful in helping us be able to help you, otherwise we're just kind hand waving and guessing. — Ed Morton, Mar 15 '23 at 20:45

Grep not matching from serial input

0 Answers0