0

I get segmentation fault on a certain scenario(it is C code with DEC VAX FMS(Forms Management System) calls to get a certain field on a CRT screen - pretty old legacy code). I am on an AIX machine, and have only dbx installed on it. GDB, valgrind etc. are not available.

Here is what I get when I try to debug:

Unreadable instruction at address 0x53484950

I do not know how to proceed from here.

I have tried a few things:

1.

(dbx) up
not that many levels
(dbx) down
not that many levels

(dbx) n
where

Segmentation fault in . at 0x53484950 ($t1)
0x53484950 (???) Unreadable instruction at address 0x53484950

  1. Tried tracei(for machine instructions), dump(dump gives so much output, I am unable to make sense of it) etc. but nothing seems to help.

    (dbx) &0x53484950/X
    expected variable, found "1397246288"

I am used to getting a stack trace on "where" and going on from there. This is something I have not encountered before, and it appears I am not very good at dbx either. Any help to get to at least the line of code that is causing trouble is appreciated.

Swarna Gowri
  • 277
  • 2
  • 10
  • 2
    For 1-2, I suspect your program has corrupted its stack, so the debugger is unable to find stack frames or valid caller return addresses. For 3, nobody can help you interpret “so much output” when you show none of it. 4 looks like a syntax error; hex 53484950 is decimal 1397246288, and, if `&` is an address operator in dbx, then it is telling you it expected the operand of `&` to be a variable, not a number. Perhaps `0x53484950/X` would work. However, I expect that number is not useful, having resulted from the stack corruption. You need to catch the program before this point. – Eric Postpischil Jul 17 '21 at 10:43
  • 2
    Somehow you put `x'53484950'`=`c'SHIP'` into `$pc` Most likely you've overwritten an array on the stack. – Lorinczy Zsigmond Jul 17 '21 at 17:21
  • 1
    If the source code is not available, make sure to resize the terminal to size 25x80 before starting the program. – Lorinczy Zsigmond Jul 17 '21 at 18:34
  • You were right. I was doing something wrong(with pointers in the code that I wrote that made use of the FMS calls) and this corrupted the stack, and led to the above problem further up. I inspected code, and found it out, though it took me a lot of time to figure this one out. – Swarna Gowri Jul 22 '21 at 18:10

1 Answers1

1

Once you have hit a segfault, there is no way to continue, so the n command is not going to do anything. At that point, all you can do is examine the stack and the variables, and that will be meaningless unless you have the source code and can recompile it.

In fact, without the source code, I am not sure how you could possibly proceed with fixing the program. Even if you could "decompile" the program, or at least disassemble the program, the risk of making a mistake when trying to patch the binary in order to fix it is virtually 100%.

I'm sorry. Given the limitations you are working under, I would argue the the problem is insolvable. Without tools such as gdb or valgind, it will be difficult to find the problem, and without the source code, it will be very difficult to fix the problem once you have found it.

Jeff Silverman
  • 692
  • 1
  • 8
  • 15
  • You were right. I was doing something wrong(with pointers in the code that I wrote that made use of the FMS calls) and this corrupted the stack, and led to the above problem further up. I inspected code, and found it out, though it took me a lot of time to figure this one out. – Swarna Gowri Jul 22 '21 at 18:14