How to Consume a RESTful Web Service in Lotus Domino using SSL

Question

I'm developing a background agent (and a web agent) in LotusScript that has to consume RESTful web services supplied by a 3rd party. For obvious reasons, they protected their services using SSL.

In the Notes client, I managed to accept the certificate the site emits. The agent does exactly what I want it to do, all communication works as it should. But when I try the background agent on the server (R8.5.3FP2):

12/07/2013 11:46:17 Connection interrupted: SSL Error: Bad or missing remote certificate

Some details about the code:

NotesDocument.GetDocumentByURL
the Web Retriever (web.exe) is running on the server (the task is required)

I opened the same page, using the URL I use to connect to the services, in Firefox. The page opens correctly, and the Page Info allows me to create a certificate file (either .der or .p7c, they are both accepted by the Domino Admin). I then opened the Admin, went to Configuration/Security/Certificates/Certificates, and used the action Import Internet Certificates. Both files can be imported successfully. Still, no luck...

If you have a Good Idea, please share, I sure need one. Or could it be that difficult??

Thanks!

Trace info from the console:

12/07/2013 22:47:16,82 [01E8:0005-0E40] ReadKeyfile> Recovering password from stash file
12/07/2013 22:47:16,83 [01E8:0005-0E40] ReadKeyfile> Password is 
12/07/2013 22:47:16,83 [01E8:0005-0E40] ReadKeyfile> Reading keyfile E:\Lotus\Domino\data\keyfile.kyr
12/07/2013 22:47:16,83 [01E8:0005-0E40] ReadKeyfile> Read failed: file not found
12/07/2013 22:47:16,83 [01E8:0005-0E40] ReadKeyfile> Exit status = 259
12/07/2013 22:47:16,85 [01E8:0005-0E40] int_MapSSLError> Mapping SSL error 0 to 0
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Handshake> Enter
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Handshake> SSL Undetermined attempt
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Write> Enter len = 66
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Xmt> 00000000: 80 40 01 03 00 00 27 00 00 00 10 00 00 04 00 00   '.@....'.........'
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Xmt> 00000010: 05 00 00 2F 00 00 35 00 00 0A 00 00 09 00 00 62   '.../..5........b'
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Xmt> 00000020: 00 00 03 00 00 02 00 00 01 00 00 01 01 00 80 02   '................'
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Xmt> 00000030: 00 80 B0 AC 0E 39 84 F7 C8 D1 1A A6 D6 10 CA C9   '..0,.9.wHQ.&V.JI'
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_Xmt> 00000040: 9C 00                                             '..'
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Write> Switching Endpoint to sync
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Write> Posting a nti_snd for 66 bytes
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_EncryptData> SSL not init exit
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Write> Switching Endpoint to async
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_EncryptDataCleanup> SSL not init exit
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Write> nti_done return 66 bytes rc = 0
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Write> Exit, wrote 66 bytes
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Read> Enter len = 1
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Read> Switching Endpoint to sync
12/07/2013 22:47:16,85 [01E8:0005-0E40] S_Read> Posting a nti_rcv for 1 bytes
12/07/2013 22:47:16,85 [01E8:0005-0E40] SSL_RcvSetup> SSL not init exit
12/07/2013 22:47:16,96 [01E8:0005-0E40] S_Read> Switching Endpoint to async
12/07/2013 22:47:16,96 [01E8:0005-0E40] S_Read> nti_done return 1 bytes rc = 0
12/07/2013 22:47:16,96 [01E8:0005-0E40] SSL_RCV> 00000000: 00                                                '.'
12/07/2013 22:47:16,96 [01E8:0005-0E40] S_Read> Exit, read 1 bytes
12/07/2013 22:47:16,96 [01E8:0005-0E40] S_Read> Enter len = 4
12/07/2013 22:47:16,96 [01E8:0005-0E40] S_Read> Switching Endpoint to sync
12/07/2013 22:47:16,96 [01E8:0005-0E40] S_Read> Posting a nti_rcv for 4 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RcvSetup> SSL not init exit
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Switching Endpoint to async
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> nti_done return 4 bytes rc = 0
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RCV> 00000000: 00 00 00 00                                       '....'
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Exit, read 4 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Enter len = 74
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Switching Endpoint to sync
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Posting a nti_rcv for 74 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RcvSetup> SSL not init exit
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Switching Endpoint to async
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> nti_done return 74 bytes rc = 0
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RCV> -- 64 (0x0040) bytes of 0 --
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Exit, read 74 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLProcessHandshakeMessage Enter> Message: 2 State: 6 Key Exchange: 0 Cipher: 0x0000 (Unknown Cipher)
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLProcessHandshakeMessage Enter> Message: SSL_server_hello
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLProcessHandshakeMessage Exit> Message: 2 State: 6 Key Exchange: 1 Cipher: 0x0004 (RSA_WITH_RC4_128_MD5)
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLAdvanceHandshake Enter> Processed : 2 State: 6
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLAdvanceHandshake Enter> Processed : SSL_server_hello
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLAdvanceHandshake Exit> State : 8
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> After handshake state= 8 Status= -5000
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Exit Status = -5000
12/07/2013 22:47:16,97 [01E8:0005-0E40] int_MapSSLError> Mapping SSL error -5000 to 4176
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Enter
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Current Cipher 0x0004 (RSA_WITH_RC4_128_MD5)
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Enter len = 5
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Switching Endpoint to sync
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Posting a nti_rcv for 5 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RcvSetup> SSL not init exit
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Switching Endpoint to async
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> nti_done return 5 bytes rc = 0
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RCV> 00000000: 02 00 00 46 03                                    '...F.'
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Exit, read 5 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Enter len = 3661
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Switching Endpoint to sync
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Posting a nti_rcv for 3661 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RcvSetup> SSL not init exit
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Switching Endpoint to async
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> nti_done return 3661 bytes rc = 0
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_RCV> -- 3648 (0x0E40) bytes of 0 --
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Read> Exit, read 3661 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLProcessHandshakeMessage Enter> Message: 11 State: 8 Key Exchange: 1 Cipher: 0x0004 (RSA_WITH_RC4_128_MD5)
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLProcessHandshakeMessage Enter> Message: SSL_certificate
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLCheckCertChain> Invalid certificate chain received
Cert Chain Evaluation Status: err: 3659, Cannot establish trust in a certificate or CRL.
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLSendAlert> Sending an alert of 0x0 level 0x2
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSLProcessHandshakeMessage Exit> Message: 11 State: 2 Key Exchange: 1 Cipher: 0x0004 (RSA_WITH_RC4_128_MD5)
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Changing SSL status from -6986 to -5000 to flush write queue
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> After handshake2 state 2
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Exit Status = -5000
12/07/2013 22:47:16,97 [01E8:0005-0E40] int_MapSSLError> Mapping SSL error -5000 to 4176
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Enter
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Current Cipher 0x0004 (RSA_WITH_RC4_128_MD5)
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Write> Enter len = 7
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Xmt> 00000000: 15 03 00 00 02 02 00                              '.......'
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Write> Switching Endpoint to sync
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Write> Posting a nti_snd for 7 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_EncryptData> SSL not init exit
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Write> Switching Endpoint to async
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_EncryptDataCleanup> SSL not init exit
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Write> nti_done return 7 bytes rc = 0
12/07/2013 22:47:16,97 [01E8:0005-0E40] S_Write> Exit, wrote 7 bytes
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> After handshake2 state 2
12/07/2013 22:47:16,97 [01E8:0005-0E40] SSL_Handshake> Exit Status = -6986
12/07/2013 22:47:16,97 [01E8:0005-0E40] int_MapSSLError> Mapping SSL error -6986 to 4163
12/07/2013 22:47:17   Connection interrupted: SSL Error: Bad or missing remote certificate

Again, more info. A cross-certificate was created (it is in fact Go Daddy, the certifier): cross-certificate created Still, no luck. I asked the Admin to cross-certify the other party as well, it won't hurt and might indeed be the missing link. More to follow.

That didn't help either, the cross-certificates were created and still the server complains about the remote certificate.

So then I thought: well, let's mimic the actions I did to get my personal certificate. I created a trust in my personal address book, copied that document over to the server, and lo and behold, my web agent suddenly started to work!!

Remains the question: is this how it's supposed to work?? And where is this documented? If at all?

Thanks for your support!

Have a look at this. The bits you want are near the end (Java code to debug SSL). It should give you more details as to why the certificate is not being found in your CACERTS. You can update your question with the logs (remove confidential stuff). http://www-10.lotus.com/ldd/ddwiki.nsf/dx/Connecting_to_a_Domino_server_over_SSL_in_Java_using_a_self_signed_certificate._#+Creating+a+sample+Java+application+to+connect+to+the+server+ — Simon O'Doherty, Jul 12 '13 at 13:11
I did say that it's all in LotusScript, didn't I? Oh dear me... Question updated. — D.Bugger, Jul 12 '13 at 15:10
Ah ha! You can try DEBUG_SSL_ALL=3 + SSL_TRACE_KEYFILEREAD=1 in the server notes.ini. Might give you more details on console log. — Simon O'Doherty, Jul 12 '13 at 16:03
Brilliant! Tried it right away, dumped the result in the question text, now studying it... Thanks! — D.Bugger, Jul 12 '13 at 20:58
Still not sure of the answer, but have you tried creating a cross certificate with your server the agent is running on? See: http://serverfault.com/questions/505273/java-certificateexception-in-domino-9-when-trying-to-access-https-url/515189#515189 . Or have you installed the full certificate chain of the server you are connecting to? — Simon O'Doherty, Jul 12 '13 at 21:25
I asked the Admin to X-cert the certificate. Didn't even know it's required... I'll be back... — D.Bugger, Jul 15 '13 at 09:13
May also be related to this. Probably need to open a PMR to confirm: http://www-01.ibm.com/support/docview.wss?uid=swg1LO41163 — Simon O'Doherty, Jul 15 '13 at 09:42

How to Consume a RESTful Web Service in Lotus Domino using SSL

0 Answers0