2

due to recent problems discovered with NDK12 and NDK13b2, i'm thinking of 'porting' libx264's use of signal() (and missing bsd_signal() in ndk12) to use sigaction() instead.

The problem is, I'm not quite sure what's the simple&fastest way to replace signal() calls with sigaction() ones.

For all i see, it's mainly used in x264-snapshot/common/cpu.c in the following manner:

using the following signal handler:

static void sigill_handler( int sig )
{
    if( !canjump )
    {
        signal( sig, SIG_DFL );
        raise( sig );
    }

    canjump = 0;
    siglongjmp( jmpbuf, 1 );
}

This is the problematic x264_cpu_detect function... currently, i'm guessing i only need to tackle the ARM version, but i';; still have to replace all occurances of signal() with sigaction() so i might just cover both of them to get the thing building...

FYI - the NDK13 beta2 still has "unstable" libc and the build doesn't fail on this part, but rather the first invocation of the rand() function somewhere else... So i'm out of luck and replacing the signal() calls might be better than just waiting for the official NDK13 release. I'm doing this to get rid of text-relocations so i can run the library (and doubango) on API 24 (Android N)

the problematic part of function that invokes signal():

#elif SYS_LINUX

uint32_t x264_cpu_detect( void )
{
    static void (*oldsig)( int );

    oldsig = signal( SIGILL, sigill_handler );
    if( sigsetjmp( jmpbuf, 1 ) )
    {
        signal( SIGILL, oldsig );
        return 0;
    }

    canjump = 1;
    asm volatile( "mtspr 256, %0\n\t"
                  "vand 0, 0, 0\n\t"
                  :
                  : "r"(-1) );
    canjump = 0;

    signal( SIGILL, oldsig );

    return X264_CPU_ALTIVEC;
}
#endif

#elif ARCH_ARM

void x264_cpu_neon_test( void );
int x264_cpu_fast_neon_mrc_test( void );

uint32_t x264_cpu_detect( void )
{
    int flags = 0;
#if HAVE_ARMV6
    flags |= X264_CPU_ARMV6;

    // don't do this hack if compiled with -mfpu=neon
#if !HAVE_NEON
    static void (* oldsig)( int );
    oldsig = signal( SIGILL, sigill_handler );
    if( sigsetjmp( jmpbuf, 1 ) )
    {
        signal( SIGILL, oldsig );
        return flags;
    }

    canjump = 1;
    x264_cpu_neon_test();
    canjump = 0;
    signal( SIGILL, oldsig );
#endif

    flags |= X264_CPU_NEON;

    // fast neon -> arm (Cortex-A9) detection relies on user access to the
    // cycle counter; this assumes ARMv7 performance counters.
    // NEON requires at least ARMv7, ARMv8 may require changes here, but
    // hopefully this hacky detection method will have been replaced by then.
    // Note that there is potential for a race condition if another program or
    // x264 instance disables or reinits the counters while x264 is using them,
    // which may result in incorrect detection and the counters stuck enabled.
    // right now Apple does not seem to support performance counters for this test
#ifndef __MACH__
    flags |= x264_cpu_fast_neon_mrc_test() ? X264_CPU_FAST_NEON_MRC : 0;
#endif
    // TODO: write dual issue test? currently it's A8 (dual issue) vs. A9 (fast      mrc)
#endif
    return flags;
}

#else

uint32_t x264_cpu_detect( void )
{
    return 0;
}

So the question is really this: what would be the quickest/easiest//fastest way to replace the signal() calls with sigaction() ones while preserving the current functionality?

EDIT: The reason i'm trying to get rid of signal() are these build errors:

/home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function sigill_handler: error: undefined reference to 'bsd_signal'

/home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function x264_cpu_detect: error: undefined reference to 'bsd_signal'
/home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function x264_cpu_detect: error: undefined reference to 'bsd_signal'

/home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function x264_cpu_detect: error: undefined reference to 'bsd_signal'

I already know that this is a known NDK12 problem, that might be solved by bringing bsd_signal back to the libc in NDK13. However, in it' beta state with it's unstable libc - it's currently missing the rand() function and simply waiting for it might not do the trick. But in the worst-case scenario, i guess i'll just have to wait for it and retry after it's release.

But as it currently is, the prebuilt version of the library i want to use has text-relocations and is being rejected by phones running newer API / version of the android OS.

EDIT2: I also know that signal() usually works by using sigaction() under the hood, but maybe i won't get bsd_signal related build-errors... since i'm suspecting that this one isn't using it. It's obviously using bsd_signal, which may or may not be the same underlying thing :/

Shark
  • 6,513
  • 3
  • 28
  • 50
  • What exactly is wrong with `signal()`? Why do you want to get rid of it? – Sergio Sep 28 '16 at 14:38
  • Because NDK12 does not have `bsd_signal` symbol defined and throws the build errors mentioned in the edit. TL;DR = Because it causes build errors. – Shark Sep 28 '16 at 14:44
  • But you don't present any usage of `bsd_signal()` in your example code, so what does it have to do with your question? – John Bollinger Sep 28 '16 at 15:04
  • Glibc's `signal()` works by calling `sigaction()`, if that's what you mean by "usually". That does not constitute any particular problem, because glibc also provides `sigaction()`. – John Bollinger Sep 28 '16 at 15:12
  • I haven't checked all libc implementations, so i kinda tried to remain reserved by saying "usually". As far as android goes, it's libc - `bionic` - inlines `bsd_signal` which was removed two NDKs ago (might be better to just say in `android-21`) and because it broke alot of third-party libraries, they're considering to bring it back in the pending NDK13. As far as your other question goes - i have no invocations or usage of `bsd_signal` in the code, `grep` only finds it in `.o` and `.a` files, and the `.a` files can't be linked in to a shared library (`.so`) because the symbol is missing. – Shark Sep 28 '16 at 16:59
  • But the answers you and Serhio gave me so far were very helpful and have greatly helped in understanding and confirming what should be done next. – Shark Sep 28 '16 at 17:03
  • @JohnBollinger "But you don't present any usage of bsd_signal() in your example code" - See https://github.com/android-ndk/ndk/issues/160. Old Android API levels defined `signal` as an inline function that called `bsd_signal`. New Android APIs rightly define `signal` out of line and no longer include a declaration for `bsd_signal`. If you build something for android-9 and then link it into something targeting android-21, you get the build failure. r13 puts back `bsd_signal` to handle this use case. – Dan Albert Sep 28 '16 at 21:26

2 Answers2

2

According to info posted in your question, I see next points:

  • You have prebuilt binaries, that were built for android-19 or lower (since they refer to bsd_signal()).

  • You want to compile your code for android-21 or higher and link it against that old-targeted prebuilts.

As you may know, prior to android-21 a lot of libc's functions were declared as static just right in headers. And actually all they were thin wrappers around more generic functions, that are exposed by libc binaries. Old versions of NDK have next definition in <signal.h>:

/* the default is bsd */
static __inline__ __sighandler_t signal(int s, __sighandler_t f)
{
    return bsd_signal(s,f);
}

That is from where bsd_signal() references stand in your binaries. You have two choices how to solve it.

  • Recompile your dependencies totally for android-21 or higher. But, note that they won't run on older platforms.

  • Provide bsd_signal() in your code, e.g. you can add an extra source file with implementation of this function. You can use bionic's implementation as reference. Also note that this function should be marked as hidden: __attribute__ ((visibility ("hidden"))). This is needed to prevent putting it to DSO dynamic symbol table, that may confuse dynamic linker on old platforms that have that function already exposed by libc.so.

Also similar manipulations may be required for another libc functions, that previously were exported via similar static wrappers.

Note, that no one of the proposed two solutions requires modifications in the code of your dependencies.

Sergio
  • 8,099
  • 2
  • 26
  • 52
  • From what i've read, the NDK people are thinking about bringing `bsd_signal()` back in NDK 13, however i'm not entirely sure i can just rely on that piece of information since their libc is currently unstable and kinda broken (because it's missing `rand()`) – Shark Sep 28 '16 at 16:46
  • But basically, your solution #2 merely requires me to write a small piece of C code (copy/paste the bsd_signal() implementation from bionic) and include it in the build process without changing x264's code? Great, sounds promising. (y) – Shark Sep 28 '16 at 16:54
  • @Shark Not at all. Just recompile all the libraries that you use that didn't come as a part of the NDK. – Kuba hasn't forgotten Monica Sep 28 '16 at 18:19
2

The reason i'm trying to get rid of signal() are these build errors [...]

The code you present shows the functions in question calling signal(), whereas the linker errors indicate that they are calling bsd_signal() (directly). If the code presented goes with the error messages presented, then that can only mean that there is an in-scope macro or inline function whose expansion includes a call to bsd_signal(). Presumably, that macro is named signal(), so as to substitute for a call to the real signal() function.

If that's your macro / inline function then you should be able to fix it to call sigaction() instead (see below). On the other hand, if it is the system's macro / inline function then the system's header files do not correspond to its C library. In that case, it is essential to establish a consistent build environment before you do anything else.

So the question is really this: what would be the quickest/easiest//fastest way to replace the signal() calls with sigaction() ones while preserving the current functionality?

In order to answer that, one must first determine / divine what the current functionality is, or is supposed to be. POSIX allows the semantics of using signal() to set a signal's disposition to anything other than SIG_DFL or SIG_IGN to vary. Specifically, if you set a custom handler for a given signal then the disposition for that signal might or might not be reset to its default when the signal is received. Additionally, behavior varies with respect to whether the signal being handled is blocked during the execution of the handler. Resolving that issue was one of the primary purposes for introducing sigaction().

Supposing, based on the linker errors, that BSD behavior is what you are used to getting, the analog of

oldsig = signal( SIGILL, sigill_handler );

would be

struct sigaction old_action;
struct sigaction new_action = {
    .sa_handler = sigill_handler
};

int result = sigaction(SIGILL, &new_action, &old_action);
if (result) {
    // handle error ...
} else {
    oldsig = old_action.sa_handler;
}

If you want different semantics (emulating SysV signal() semantics, for instance) then you would use the sa_flags member of new_action to describe the details. If you want to mask any other signals while the handler runs, then you would want to use the sa_mask member to indicate that. In any case, you might also consider whether you need to preserve the flags or mask along with the previous handler.

You can substitute that for calling signal() by defining your own macro (not inline function) for signal(), after including all needed system headers. If your calls to signal() are spread across multiple files, then presumably you would want to put such a macro definition in a local header file.

John Bollinger
  • 160,171
  • 8
  • 81
  • 157
  • I think NDK's `signal()` implementation inlines the `bsd_signal()` implementation, however i'm not entirely sure on that (yet). But it's my main suspicion so far. Serhio's post confirms it. – Shark Sep 28 '16 at 16:43
  • 1
    @Shark, yes, inlining a call to `bsd_signal()` is one of the alternatives I described. If you have a header that does this and a library implementation that does not provide `bsd_signal()`, then the two do not go together. Possibly this is a major bug, but more likely you have a corrupt build environment with different versions of the headers and library. Sorting that out (and afterward performing a clean build) ought to resolve the linker errors you presented. Switching to `sigaction()` is not a bad idea, but it's probably not obligatory for you. – John Bollinger Sep 28 '16 at 17:25
  • "Possibly this is a major bug, but more likely you have a corrupt build environment with different versions of the headers and library." That's the case here. @serhio's answer covers the most likely cause. – Dan Albert Sep 28 '16 at 21:32
  • It's actually not the case here, NDK removed the `bsd_signal` symbol at some point (android-21) and are considering of bringing it back in NDK13 because they broke alot of third-party libs because of that... – Shark Sep 29 '16 at 15:03
  • @Shark, when the function was removed and whether it might be brought back are irrelevant to the question of building your code *now* against the library you have *now*. It is difficult to believe that NDK12 provides system headers that inline a call to a function that NDK12 does not, in fact, provide. It is much more plausible that you're trying to use header files from an earlier NDK (perhaps unintentionally) with the C library from NDK 12. – John Bollinger Sep 29 '16 at 15:15
  • 1
    @Shark, the reason to consider bringing back `bsd_signal()` in NDK 13 would be to support *existing* binaries built against NDK 11 or earlier, and to some extent to support code that calls it directly. Binaries built against NDK 12 cannot possibly require the system to provide that function. – John Bollinger Sep 29 '16 at 15:16
  • 1
    We make no guarantees that code built with NDK N-1 will work with NDK N. In general C code will be fine, but the C++ STLs are not ABI stable (this is why they're in the NDK and not provided by the system). I don't believe that's the problem you're having (a trivial call to `signal` works fine). It isn't NDK r11 -> NDK r12 that's causing you problems, it's the API level change. r11 and r12 both had a bug with `bsd_signal` where code built against pre-L couldn't be linked into code built with L or newer, for the reasons outlined in @serhio's answer. This is the thing we now support in r13. – Dan Albert Sep 29 '16 at 16:42