grep -f on OS X produces segfault

Question

If you've got a Mac, try this:

echo 'abcd*' > grepfile
echo 'abc$' >> grepfile
echo '^abc' >> grepfile
echo "fojeiwuroiuwet\nljfajsljkfabcdddjlfkajlkj\nabcaaa\nzzzabc\n" | grep -f grepfile

Here's the version:

$ grep --v
grep (BSD grep) 2.5.1-FreeBSD

This is a machine (a rMBP of the 2012 flavor) that's kept up with Apple's software updates, so I am on 10.8.4.

I verified that GNU grep compiled from source does not suffer from this problem. Indeed it is version 2.14, which is a whole lot of versions past 2.5.1.

But how might one achieve the task of testing some input against a series of regexes otherwise, without some vastly inefficient loop that forks a grep for each regex?

Edit: The approach I took to work around this was something akin to: while read REGEX; do [[ ... =~ $REGEX ]] ... done < regexfile.

Question: Is this a known bug with this version of grep? How can we set up our systems so they will work properly with a file of regexes to grep?

Update: Looks like some folks are reporting it works fine (even with this particular FreeBSD 2.5.1 grep). What are some steps I can take to try to figure out which .so/.dylib's it might be using? Can someone do a shasum /usr/bin/grep and tell me if it works for you? (I'm not sure if that would provide much information, but what I'm after is whether my computer's configuration is screwed up, or if this is some actual existing issue with this version of the software.)

$ shasum /usr/bin/grep
eac59389d09642decbb8551e2c975f795934bfbf  /usr/bin/grep

Here is more info:

$ codesign -dvvv /usr/bin/grep
Executable=/usr/bin/grep
Identifier=com.apple.zgrep
Format=Mach-O thin (x86_64)
CodeDirectory v=20100 size=224 flags=0x0(none) hashes=6+2 location=embedded
Hash type=sha1 size=20
CDHash=93b823c000188f8737653d8333c90a6db9361d70
Signature size=4064
Authority=Software Signing
Authority=Apple Code Signing Certification Authority
Authority=Apple Root CA
Info.plist=not bound
Sealed Resources=none
Internal requirements count=2 size=208

Further investigation:

$ gdb /usr/bin/grep
GNU gdb 6.3.50-20050815 (Apple version gdb-1824) (Thu Nov 15 10:42:43 UTC 2012)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries .... done

(gdb) start -f grepfile
Function "main" not defined.
Make breakpoint pending on future shared library load? (y or [n])
Starting program: /usr/bin/grep -f grepfile
Reading symbols for shared libraries +++.............................. done
abc
abc

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000101000000
0x00007fff89b5d1b3 in memchr ()
(gdb) where
#0  0x00007fff89b5d1b3 in memchr ()
#1  0x00007fff89b8e45a in __sfvwrite ()
#2  0x00007fff89b8e861 in fwrite ()
#3  0x0000000100003138 in _mh_execute_header ()
#4  0x0000000100002988 in _mh_execute_header ()
#5  0x0000000100001c28 in _mh_execute_header ()
#6  0x00007fff8e2d57e1 in start ()
(gdb)

I have rebooted the machine as well. It repeatably does the same thing in gdb.

Then I switched to my Macbook Air and it worked, and then I checked and it has homebrew grep 2.14 installed. I believe I was testing grep vs ack back in the day when I learned about ack. So this is another way to gain a proper grep on a Mac, use homebrew. That was kind of sad because I used homebrew to obtain `gettext` in order to compile grep without even checking for `grep` in the first place... — Steven Lu, Jun 22 '13 at 03:55
It's annoying to re-find the answers to things already found by past-me. It is brew formula `homebrew/dupes/grep`. — Steven Lu, Jun 22 '13 at 04:00
FWIW, your example works fine for me with the stock Apple /usr/bin/grep (also 2.5.1-FreeBSD) shipped with OS X 10.8.4. — Ned Deily, Jun 22 '13 at 04:03
@NedDeily interesting. Let me test it with the 2.5.1 on the other macbook... — Steven Lu, Jun 22 '13 at 04:08
This is getting interesting. The macbook air (mid 2011 version I think it is; it's a Sandy Bridge MBA) is running Lion 10.7.5 and has `grep (GNU grep) 2.5.1`, and handles it fine too. I must have borked something with this machine then?? — Steven Lu, Jun 22 '13 at 04:10
BTW, note if you install grep with homebrew it will install it as ggrep, read the Caveats it lists. If you don't wanna re-do the brew install just hop over to `/usr/local/bin` and `ln -s ../Cellar/grep//bin/ggrep grep` — Steven Lu, Jun 22 '13 at 04:22
I'm running 10.8.4, built-in grep, and it works. My shasum is different (350ee11e1868e18c9707ea7035184a114f40badf), but I don't think that's reliable, because the binary can be modified to optimize linked libraries. Try `codesign -dvvv /usr/bin/grep` and see if it lists "Identifier=com.apple.zgrep" and "CDHash=1537b3ed49878d5d18482859a37318164a2a40f1". — Gordon Davisson, Jun 22 '13 at 05:02
Doesn't reproduce on my MacBook Pro (16 GiB main memory; Mac OS X 10.8.4; grep (BSD grep) 2.5.1-FreeBSD). The version of GNU `grep` is irrelevant as a comparison with BSD `grep`. I have had problems with `egrep` running out of space on me in times past; however, the upgrade to 10.8.4 seems to have fixed that. — Jonathan Leffler, Jun 22 '13 at 05:08
@GordonDavisson I see this: `CDHash=93b823c000188f8737653d8333c90a6db9361d70` — Steven Lu, Jun 22 '13 at 05:17
I have edited OP with `gdb` run -- maybe somebody can make some sense out of it. — Steven Lu, Jun 22 '13 at 05:31
I see. Well. I couldn't be bothered to report it now seeing as any potential resolution to the issue would involve doing exactly what I did (compile an appropriate replacement from source). Apple has saved themselves (and me) the work of that bug report by deciding to go with a BSD based UNIX. On Windows, though, something of this sort probably could have resulted in an actual stoppage of progress. — Steven Lu, Jun 23 '13 at 08:59
Able to reproduce. `350ee11e1868e18c9707ea7035184a114f40badf /usr/bin/grep`. `grep (BSD grep) 2.5.1-FreeBSD`. — Luqmaan, Jun 25 '13 at 23:35
Thanks @luqmaan, can you give me more details about your machine? Is it a recent mac on Mountain Lion? — Steven Lu, Jun 26 '13 at 01:00
No seg fault here. `350ee11e1868e18c9707ea7035184a114f40badf /usr/bin/grep`. `grep (BSD grep) 2.5.1-FreeBSD`. 13-inch mid 2009 MacBook Pro, 10.8.3. — Ashley, Jun 27 '13 at 11:00
working on MBP 2009 Os X 10.8.4 and `grep (BSD grep) 2.5.1-FreeBSD` — Thomas Ayoub, Jul 08 '13 at 21:45

kenorb · Accepted Answer · 2013-08-13T14:17:40.593

I've got OSX 10.8.4 on MacBook Air and your example doesn't crash by default, but only when adding --color parameter.

Explanation

This crash usually happens when you're mixing wilcard (asterisk sign) with the terminal colours and this is the software bug.

Also check another simpler example:

echo 'abc*' | grep --color=auto -e ".*" -e a

Here it seems that --color=auto makes the difference (without it or setting to never, then it doesn't crash). So I assume that your grep is using colors in terminal by default.

Solution

Make sure that your grep is not an alias to the grep with colors enabled, or the colors are not enabled by default.

You can always try to run grep with --color=never.

For permanent solution, I've reported the bug report:

http://www.freebsd.org/cgi/query-pr.cgi?pr=181263

so the issue can be fixed in the software it-self.

If you'd like to access more detailed log of your crash, go to Console, Show Log List and find the crash log under User Diagnostic Reports.

E.g.:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib               0x00007fff8a8da1b3 memchr + 15
1   libsystem_c.dylib               0x00007fff8a90b45a __sfvwrite + 578
2   libsystem_c.dylib               0x00007fff8a90b861 fwrite + 114
3   grep                            0x000000010a4a3138 0x10a4a0000 + 12600
4   grep                            0x000000010a4a2988 0x10a4a0000 + 10632
5   grep                            0x000000010a4a1c28 0x10a4a0000 + 7208
6   libdyld.dylib                   0x00007fff8daf77e1 start + 1

Here is some more detailed explanation about the crash:

A quick test

  $ env -i bsdgrep -Fi without_nls usr.bin/grep/grep.c
  $ env -i gnugrep -Fi without_nls usr.bin/grep/grep.c
  #ifndef WITHOUT_NLS
  #ifndef WITHOUT_NLS
  #ifndef WITHOUT_NLS

shows that bsd fgrep already fails to ignore case. And if you throw
a few more options to the mix it'd crash, e.g.

  $ env -i LC_CTYPE=en_US.UTF-8 TERM=xterm bsdgrep --color -Fir without_nls usr.bin/grep/
  [...]
  Program received signal SIGSEGV, Segmentation fault.
  0x0000000801007ff2 in memchr (s=0x61167a, c=10, n=18446744073707490297) at /usr/src/lib/libc/string/memchr.c:48
  48                              if (*p++ == (unsigned char)c)
  (gdb) bt
  #0  0x0000000801007ff2 in memchr (s=0x61167a, c=10, n=18446744073707490297) at /usr/src/lib/libc/string/memchr.c:48
  #1  0x0000000801007b03 in __sfvwrite (fp=0x801247770, uio=0x7fffffffd8f0) at /usr/src/lib/libc/stdio/fvwrite.c:170
  #2  0x0000000801007698 in fwrite (buf=0x608c03, size=18446744073709551606, count=1, fp=0x801247770)
      at /usr/src/lib/libc/stdio/fwrite.c:95
  #3  0x0000000000405498 in printline (line=0x7fffffffdb70, sep=58, matches=0x7fffffffd990, m=9)
      at /usr/src/usr.bin/grep/util.c:500
  #4  0x0000000000404f51 in procline (l=0x7fffffffdb70, nottext=0) at /usr/src/usr.bin/grep/util.c:381
  #5  0x000000000040489f in procfile (fn=0x80140b600 "usr.bin/grep/nls/es_ES.ISO8859-1.msg") at /usr/src/usr.bin/grep/util.c:239
  #6  0x00000000004044d7 in grep_tree (argv=0x7fffffffdd30) at /usr/src/usr.bin/grep/util.c:163
  #7  0x0000000000403ea9 in main (argc=5, argv=0x7fffffffdd10) at /usr/src/usr.bin/grep/grep.c:689

Source: http://lists.freebsd.org/pipermail/freebsd-current/2011-August/026502.html

Also it seems that there are different grep binaries on different OSX even with the same version:

$ grep --v
grep (BSD grep) 2.5.1-FreeBSD
$ shasum /usr/bin/grep
350ee11e1868e18c9707ea7035184a114f40badf  /usr/bin/grep
$ codesign -dvvv /usr/bin/grep
Executable=/usr/bin/grep
Identifier=com.apple.zgrep
Format=Mach-O thin (x86_64)
CodeDirectory v=20100 size=224 flags=0x0(none) hashes=6+2 location=embedded
Hash type=sha1 size=20
CDHash=1537b3ed49878d5d18482859a37318164a2a40f1
Signature size=4064
Authority=Software Signing
Authority=Apple Code Signing Certification Authority
Authority=Apple Root CA
Info.plist=not bound
Sealed Resources=none
Internal requirements count=2 size=176

Hey thanks for digging into this! I'm not sure whether I've got it aliased to show color or not but I definitely prefer to have color (of course the script doesn't need it). I believe it is not aliased but will default to using color. Hopefully when using it in a script (which should make it intelligently disable color) it will then not suffer from the problem! — Steven Lu, Aug 13 '13 at 14:58

grep -f on OS X produces segfault

1 Answers1

Linked