I am new to Firefox extension development.
We got a firefox Extension that signs PDF (encapsulated signature) working for all versions of firefox prior to 22 (22 not included) Using NSS 3.12. After that Firefox version, there is a modification in the NSS used by it. This extension is a JS extension calling C++ fonctions using ctypes.
My goal is to make this extension work on firefox version superior to 22.
I got in my extension libs nspr4.dll, nss3.dll, nssutil3.dll, plc4.dll, plds4.dll generated with mozilla-build from NSS release.
the Code we got is pretty similar to what you can find here : https://github.com/metajack/nss/blob/master/cmd/p7sign/p7sign.c
the code works well ( it gets the certificates information sucessfully, creates the SEC_PKCS7ContentInfo with SEC_PKCS7CreateSignedData(...), set it's content with SEC_PKCS7SetContent(..), includes the certs chains) until the call of SEC_PKCS7Encode. when using this method, fireFox crashes.
I used WinDBG to get the stacktrace and here is what I got:
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
FAULTING_IP:
plds4!PL_HashTableLookupConst+c
721b1c7c 8b4f0c mov ecx,dword ptr [edi+0Ch]
EXCEPTION_RECORD: ffffffffffffffff -- (.exr 0xffffffffffffffff)
.exr 0xffffffffffffffff
ExceptionAddress: 00000000721b1c7c (plds4!PL_HashTableLookupConst+0x000000000000000c)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 000000000000000c
Attempt to read from address 000000000000000c
CONTEXT: 0000000000000000 -- (.cxr 0x0;r)
.cxr 0x0;r
eax=063e7b90 ebx=00000000 ecx=dd4c128b edx=063e7c98 esi=063e7b90 edi=00000000
eip=721b1c7c esp=0067c1e8 ebp=063e7898 iopl=0 nv up ei pl nz na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00210206
plds4!PL_HashTableLookupConst+0xc:
721b1c7c 8b4f0c mov ecx,dword ptr [edi+0Ch] ds:002b:0000000c=????????
.cxr
FAULTING_THREAD: 00000000000016cc
PROCESS_NAME: firefox.exe
ERROR_CODE: (NTSTATUS) 0xc0000005 - L
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - L
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: 000000000000000c
READ_ADDRESS: 000000000000000c
FOLLOWUP_IP:
plds4!PL_HashTableLookupConst+c
721b1c7c 8b4f0c mov ecx,dword ptr [edi+0Ch]
APPLICATION_VERIFIER_FLAGS: 0
APP: firefox.exe
ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) amd64fre
BUGCHECK_STR: APPLICATION_FAULT_NULL_CLASS_PTR_READ_ZEROED_STACK
PRIMARY_PROBLEM_CLASS: NULL_CLASS_PTR_READ
DEFAULT_BUCKET_ID: NULL_CLASS_PTR_READ
LAST_CONTROL_TRANSFER: from 00000000716b9231 to 00000000721b1c7c
STACK_TEXT:
WARNING: Stack unwind information not available. Following frames may be wrong.
0067c1f4 716b9231 00000000 063e7b90 063e7ca0 plds4!PL_HashTableLookupConst+0xc
00000000 00000000 00000000 00000000 00000000 nssutil3!SECOID_FindOIDTag_Util+0x11
STACK_COMMAND: .cxr 0x0 ; kb
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: plds4!PL_HashTableLookupConst+c
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: plds4
IMAGE_NAME: plds4.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 53a04c1e
FAILURE_BUCKET_ID: NULL_CLASS_PTR_READ_c0000005_plds4.dll!PL_HashTableLookupConst
BUCKET_ID:
APPLICATION_FAULT_NULL_CLASS_PTR_READ_ZEROED_STACK_plds4!PL_HashTableLookupConst+c
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING: um:null_class_ptr_read_c0000005_plds4.dll!pl_hashtablelookupconst
FAILURE_ID_HASH: {2f7bb160-6820-dd99-35e3-03bb56e8aca4}
Followup: MachineOwner
---------
It means that when calling the SEC_PKCS7Encode, it calls a method in PLDS4.DLL that make Firefox crash.
I tried to call SEC_PKCS7Encode using an empty SEC_PKCS7ContentInfo (with SEC_PKCS7CreateData()) and it didn't crash. It crashes only when the SEC_PKCS7ContentInfo is filled.
After some reaserchs , I find this issue : https://bugzilla.mozilla.org/show_bug.cgi?id=702307 I don't know if this issue is what makes my Firefox crash.
I tried to use different version of the NSS (3.12, 3.15, 3.16.2) the results are exactly the same.
Is there some way to avoid this problem ? is there a way to encode my SEC_PKCS7ContentInfo whithout using SEC_PKCS7Encode ?
thanks in advance.
EDIT 11/07/2014
As nmaier mentioned in is response, the probleme seems to be the utilisation of two different NSS (even if it is the same version of the nss because the one in mozilla has different dependencies for exemple).
The signature (and other processes) are done in a C++ project (the Dll used by my extension is generated from this project with visual studio 2013). But in order to compile and build the DLL, the project must have the NSS libs and includes (it is quite normal, I use NSS methods in my project). The main question is how can I refer the MOZILLA FIREFOX NSS libs and include. I compiled a NSS of the same version of the Firefox's NSS, but the dependecies are different. I also tried to get the FireFox sources and compile it's NSS with mozilla build but I got errors. If I can get the Firefox's NSS libs, I could just load the Firefox's NSS dll in my extension.
How can I refers Mozilla firefox's NSS libs in my C++ project to build my custom DLL using the right libs ?