I've tried to formulate the problem in an abstract way, but anyway I give details about the actual libraries in the end.
Dynamic library Addon
is statically linked against other library WebRTC
which has some code in assembly and this code is linked into WebRTC
as object files together with WebRTC
's own object files. Lets call this assembly code VP8
. Functions of VP8
are marked extern
inside WebRTC
. Some function Encode()
from Addon
calls functions of WebRTC
which eventually calls functions from VP8
.
Now, the application Firefox
which is going to load library Addon
is quite complex and has its own version (means statically linked) of library WebRTC
(let's call it WebRTC2
), but an older one.
So, here is a problem: if a call of Encode()
is made from the application Firefox
, WebRTC
functions get called (not WebRTC2
, which is correct) BUT when WebRTC
tries to call VP8
functions, they get called from the WebRTC2
version (means application's version of WebRTC
), but not from WebRTC
.
Is there are way to force WebRTC
make calls only from local copy of VP8
?
Application Firefox
is a Firefox browser, WebRTC
is a WebRTC library, VP8
is a VP8 codec library (inside WebRTC) and Addon
is my Firefox C++ add-on.
UPDATE - DETAILED DESCRIPTION
Here is "unabstract" description of the problem:
So there is a C++ XPCOM add-on which is statically linked against latest version of WebRTC library.
At some point inside add-on a call for encoding a frame is made (method Encode
of VP8Encoder
class) and it crashes in Firefox all the time, while continue to work well on test programs using gtest framework.
The problem is that at some point inside WebRTC there is VP8 assembly code which is get called for encoding and functions of this assembly code are declared as extern
in implementation files. Actually, it crashes on vp8_intra_pred_y_ve_sse2
function.
I've compared three assembly codes of this function: one is from my version of WebRTC (used in add-on), second - where debugger crashed and the third one - from source code of Mozilla's WebRTC.
It turned out that for some weird reason, Mozilla's code get called instead of add-on's WebRTC (they both have same names of course) and as Mozilla's WebRTC code is outdated, it crashes with EXC_BAD_ACCESS.