JavaScript gets null response or empty responseXML from Python SimpleXMLRPCServer

Question

I found the answer to this question. It is resolved.

Narrative: I am accessing a Python API, a set of methodCalls on top of SimpleXMLRPCServer. Server responds to browser GET request with a html page, "web_interface.html". The HTML page is a very simple script that sends a XHR POST request of xml params to the XMLRPC server. Server responds to XHR POST with headers but empty document. Server responds to cURL with correct data. Why is JavaScript not getting any readable data in response from server?

| web_interface.html |

<!DOCTYPE html>
<html>
<head>

<script>

var xrequest = '<?xml version="1.0?"><methodCall><methodName>helloWorld<methodName><params><param><firstWord><string>hello</string><firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>';

function hello() {
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
    if (this.readyState == XMLHttpRequest.DONE) {
        alert(this.responseText);
        alert(this.status);
        alert(this.response);
    }
}
xhr.open('POST', '/', true);
xhr.setRequestHeader("Authorization", "Basic " + "aGVsbG8=" + ":" + "dGVzdA==");
xhr.send(xrequest);    
}    

</script>

</head>
<body>

<div>
  <h2 id="msgoutput">HelloWorld API Test</h2>
  <button type="button" onclick="hello(); return false;">SAY HELLO!</button>
</div>
</body>
</html>

Note: Clicking the button produces the alert dialogs. Status dialog shows "200" while Text and response dialogs are null.

| Mozilla Inspector Data & Headers |

POST Raw Data:

<?xml version="1.0?"><methodCall><methodName>helloWorld<methodName><params><param><firstWord><string>hello</string><firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>

Response Headers:

Content-Length 332
Content-Type text/html
Date Sat, 10 Dec 2016 23:51:21 GMT
Server BaseHTTP/0.3 Python/2.7.12

Request Headers:

Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-US,en;q=0.5
AuthorizationBasic aGVsbG8=:dGVzdA== (not my actual creds, swapped fakes)
Connection keep-alive
Content-Length 218
Content-Type text/plain;charset=UTF-8
DNT1
Hostlocalhost:8442
Referer http://localhost:8442/
User-Agent Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0

| Test with cURL |

:~$ curl -i --data '<?xml version="1.0"?><methodCall><methodName>helloWorld</methodName><params><param><firstWord><string>hello</string></firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>' http://username:password@localhost:8442
HTTP/1.0 200 OK
Server: BaseHTTP/0.3 Python/2.7.12
Date: Sun, 11 Dec 2016 00:00:13 GMT
Content-type: text/html
Content-length: 137

<?xml version='1.0'?>
<methodResponse>
<params>
<param>
<value><string>hello-world</string></value>
</param>
</params>
</methodResponse>

Note: No problem, cURL returns the XML response as text. I pointed cURL at a netcat socket to see exactly what it is sending to the XMLRPC server. Here's what netcat shows when cURL hits:

| cURL POST Data |

POST / HTTP/1.1
Host: localhost:8442
Authorization: Basic YWRtaW46Z2liYmVyc2g=
User-Agent: curl/7.47.0
Accept: */*
Content-Length: 220
Content-Type: application/x-www-form-urlencoded

<?xml version="1.0"?><methodCall><methodName>helloWorld</methodName><params><param><firstWord><string>hello</string></firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>

It's not CORS. Already tested a GET request to xhr.responseText with same browser on same machine. Setup is using same host, same port, same directory for both the GET page and the XHR POST XMLRPC request.

What am I missing?

so .. basically the problem is that GET works, and POST doesn't - is the curl a GET or a POST? — Jaromanda X, Dec 11 '16 at 00:37
The cURL command is doing both. The GET (URL) user:pass are sent like GET but converted to authentication headers. The XML params are sent as POST data with the --data switch. POST works; it's the response that is not working. Inspector shows the browser is sending POST data and the server is sending a response with empty XML being reported by JavaScript. — R. Strack, Dec 11 '16 at 00:53
are you saying that the curl command shown makes two requests? — Jaromanda X, Dec 11 '16 at 01:03
@JaromandaX cURL takes the get URL parameter and converts the username and password to a basic authentication header. It only makes a single request as far as I can tell. I just edited the question and I pasted the data that cURL is sending so you can see the headers. — R. Strack, Dec 11 '16 at 01:23
`Authorization: Basic YWRtaW46Z2liYmVyc2g=` vs `AuthorizationBasic aGVsbG8=:dGVzdA==` - completely different header — Jaromanda X, Dec 11 '16 at 01:28
@JaromandaX, correct. I was using that previously with no success. I changed the script to send Auth basic header with creds in base64. It doesn't seem to make a difference either way. — R. Strack, Dec 11 '16 at 01:29
it's interesting that the server response header includes `Content-type: text/html` - yet the content is clearly XML not HTML — Jaromanda X, Dec 11 '16 at 01:33
The difference in the basic headers was that one is cut-paste from the console inspector, which doesn't preserve the punctuation, and I swapped bunk passwords into the code that I pasted here online. Response from browser is empty in console inspector; except for headers and 200 OK. I have no idea what I am doing wrong. I have read dozens of articles and tried a hundred tricks. I have changed the python code to send numerous Content-Type headers and none of them work. Response to browser xhr is blank for text/xml, text/html, text/plain, application/xml, yada yada. — R. Strack, Dec 11 '16 at 05:45

R. Strack · Answer 1 · 2016-12-11T06:47:40.657

The Python SimpleXMLRPCServer had some code hacked into it to handle cookies. Problem is, the code couldn't handle the cookie objects when a browser makes connection. This code threw an error on each invocation. It was stopping the server from sending the response text to the browser. I learned this after writing a Python snippet to output debug information to a file. I removed the cookie code and then the response XML was successfully sent from the server to the browser.

I also discovered that sending the username/password from the XHR Send statement would throw an authentication error from the server. The server is expecting credentials to be in the POST header as Authentication: Basic base64 [btoa(username:password)].

The content-type header was not the culprit in this case. Now content-type is set to text/xml on both client and server.

Here is the modified JavaScript code that works:

<!DOCTYPE html>
<html>
<head>

<script>

var xrequest = '<?xml version="1.0"?><methodCall><methodName>helloWorld</methodName><params><param><firstWord><string>hello</string></firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>';

function hello() {
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
    if (this.readyState == XMLHttpRequest.DONE) {

    }
}
xhr.open("POST", "/", true);
xhr.setRequestHeader("Authorization", "Basic " + btoa("admin" + ":" + "1234"));
xhr.setRequestHeader("Content-Type", "text/xml")
xhr.send(xrequest);    
}    

</script>

</head>
<body>

<div>
  <h2 id="msgoutput">HelloWorld API Test</h2>
  <button type="button" onclick="hello(); return false;">SAY HELLO!</button>
</div>
</body>
</html>