PHP xmlrpc client and Python 2.5 xmlrpc server: incomplete data and Connection reset by peer error

Question

I have a XML RPC server running with Python.

It is implemented as an instance of the SimpleXMLRPCServer class.

from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler

class MyClass:
    def getGeneExtraInfo(self,genome,infoType,elements):
        print 'DEBUG:\ngenome: %s\ninfoType: %s, elements %s' % (genome,infoType,elements)
        return 'A' * 10000

csfServer = MyClass()

server = SimpleXMLRPCServer((serverHost, serverPort), SimpleXMLRPCRequestHandler)
server.register_instance(csfServer) 
server.serve_forever()

The csfServer has the method def getGeneExtraInfo(self,genome,infoType,elements) that returns a long string, lets consider, just for sake of simplicity, that is returs 10000 repetitions of "A".

I access this web service through PHP by this code:

function sendRequest($host, $url, $request, $port = 80) {                                                                                                                                      

  $socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
  if ($socket === false) {
      echo "socket_create() failed: reason: " . socket_strerror(socket_last_error()) . "\n";
      return FALSE;
  }

  $address = gethostbyname($host);
  if (!socket_connect ($socket, $address, $port)) {
      echo socket_strerror(socket_last_error());
      return FALSE;
  }

  $httpQuery = "POST ". $url ." HTTP/1.0\r\n";
  $httpQuery .= "User-Agent: xmlrpc\r\n";
  $httpQuery .= "Host: ". $host ."\r\n";
  $httpQuery .= "Content-Type: text/xml\r\n";
  $httpQuery .= "Content-Length: ". strlen($request) ."\r\n\r\n";
  $httpQuery .= $request ."\r\n";

  if (!socket_send($socket, $httpQuery , strlen($httpQuery), 0)) {
      echo socket_strerror(socket_last_error());
      return FALSE;
  }

  $xmlResponse = "";
  $buff = "";
  while ($bytes = socket_recv($socket, $buff, 1024, MSG_WAITALL) > 0) {
      $xmlResponse .= $buff;
  }

  // Just for debugging
  echo "socket_recv() failed; reason: " . socket_strerror(socket_last_error($socket)) . "\n";

  socket_close($socket);

  return  $xmlResponse;
}

The $request variable is build:

$xmlrpc_output_options = array( 
                        "output_type" => "xml", 
                        "verbosity" => "no_white_space", 
                        "escaping" => array("markup", "non-ascii", "non-print"), 
                        "version" => "xmlrpc", 
                        "encoding" => "UTF-8");


$xmlRequest = xmlrpc_encode_request('getGeneExtraInfo', array($genome,$infoType,$elements), $xmlrpc_output_options);

(It is possible to see the built request inside the following C and Python code)

The php client and server works properly when the server is running with python 2.4, but when the server is executed with python 2.5 or 2.6, sometimes (50% of the times), the data comes incomplete and with 'Connection reset by peer' error.

To verify if the problem is with Python or PHP, I wrote two clients, one in C and other in Python. Both uses sockets and mainly how the PHP code works. The "request" content was copied from the PHP output, it means that is exactly the request for the PHP, C, and Python.

The C code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h> 

void error(const char *msg)
{
        perror(msg);
            exit(0);
}

int main(int argc, char *argv[])
{
        int sockfd, n;
        struct sockaddr_in serv_addr;
        struct hostent *server;

        char* hostname=  "wks-13-15";
        int portno = 56572;

        char buffer[1024];

        char request[] = "POST / HTTP/1.0\r\nUser-Agent: xmlrpc\r\nHost: wks-13-15\r\nContent-Type: text/xml\r\nContent-Length: 4479\r\n\r\n<?xml version=\"1.0\" encoding=\"UTF-8\"?><methodCall><methodName>getGeneExtraInfo</methodName><params><param><value><string>hg19</string></value></param><param><value><string>GO</string></value></param><param><value><array><data><value><string>GO:0044428</string></value><value><string>GO:0044422</string></value><value><string>GO:0044425</string></value><value><string>GO:0044424</string></value><value><string>GO:0043412</string></value><value><string>GO:0006464</string></value><value><string>GO:0009889</string></value><value><string>GO:0003824</string></value><value><string>GO:0016020</string></value><value><string>GO:0016021</string></value><value><string>GO:0048522</string></value><value><string>GO:0048523</string></value><value><string>GO:0090304</string></value><value><string>GO:0019538</string></value><value><string>GO:0051171</string></value><value><string>GO:0001882</string></value><value><string>GO:0001883</string></value><value><string>GO:0080090</string></value><value><string>GO:0042221</string></value><value><string>GO:0048869</string></value><value><string>GO:0019222</string></value><value><string>GO:0005488</string></value><value><string>GO:0005886</string></value><value><string>GO:0005524</string></value><value><string>GO:0031090</string></value><value><string>GO:0050896</string></value><value><string>GO:0010556</string></value><value><string>GO:0010468</string></value><value><string>GO:0016740</string></value><value><string>GO:0003677</string></value><value><string>GO:2000112</string></value><value><string>GO:0005622</string></value><value><string>GO:0019219</string></value><value><string>GO:0006139</string></value><value><string>GO:0032502</string></value><value><string>GO:0032501</string></value><value><string>GO:0050794</string></value><value><string>GO:0009058</string></value><value><string>GO:0032991</string></value><value><string>GO:0044249</string></value><value><string>GO:0044260</string></value><value><string>GO:0044267</string></value><value><string>GO:0035639</string></value><value><string>GO:0009987</string></value><value><string>GO:0044464</string></value><value><string>GO:0051252</string></value><value><string>GO:0043170</string></value><value><string>GO:0005634</string></value><value><string>GO:0005737</string></value><value><string>GO:0050789</string></value><value><string>GO:0031326</string></value><value><string>GO:0051716</string></value><value><string>GO:0016787</string></value><value><string>GO:0031323</string></value><value><string>GO:0006810</string></value><value><string>GO:0048856</string></value><value><string>GO:0065007</string></value><value><string>GO:0043227</string></value><value><string>GO:0043167</string></value><value><string>GO:0044459</string></value><value><string>GO:0043169</string></value><value><string>GO:0008150</string></value><value><string>GO:0008152</string></value><value><string>GO:0006355</string></value><value><string>GO:0005575</string></value><value><string>GO:0046914</string></value><value><string>GO:0003674</string></value><value><string>GO:0006807</string></value><value><string>GO:0003676</string></value><value><string>GO:0044446</string></value><value><string>GO:0044444</string></value><value><string>GO:0051234</string></value><value><string>GO:0032555</string></value><value><string>GO:0043228</string></value><value><string>GO:0043229</string></value><value><string>GO:0043226</string></value><value><string>GO:0045449</string></value><value><string>GO:0032559</string></value><value><string>GO:0031224</string></value><value><string>GO:0017076</string></value><value><string>GO:0071842</string></value><value><string>GO:0071841</string></value><value><string>GO:0071840</string></value><value><string>GO:0060255</string></value><value><string>GO:0016043</string></value><value><string>GO:0034641</string></value><value><string>GO:0008270</string></value><value><string>GO:0000166</string></value><value><string>GO:0046872</string></value><value><string>GO:0044237</string></value><value><string>GO:0044238</string></value><value><string>GO:0043234</string></value><value><string>GO:0043231</string></value><value><string>GO:0043232</string></value><value><string>GO:0032553</string></value><value><string>GO:0005515</string></value><value><string>GO:0007165</string></value><value><string>GO:0048519</string></value><value><string>GO:0048518</string></value><value><string>GO:0030554</string></value></data></array></value></param></params></methodCall>";

        fprintf(stderr, "%s\n", request);

        sockfd = socket(AF_INET, SOCK_STREAM, 0);
        if (sockfd < 0) 
            error("ERROR opening socket");
        server = gethostbyname(hostname);

        if (server == NULL) {
            fprintf(stderr,"ERROR, no such host\n");
            exit(0);
        }

        bzero((char *) &serv_addr, sizeof(serv_addr));
        serv_addr.sin_family = AF_INET;
        bcopy((char *)server->h_addr, 
                (char *)&serv_addr.sin_addr.s_addr,
                server->h_length);

        serv_addr.sin_port = htons(portno);

        if (connect(sockfd,(struct sockaddr *) &serv_addr,sizeof(serv_addr)) < 0) 
            error("ERROR connecting");

        fprintf(stderr, "%d\n", strlen(request));

        n = write(sockfd,request,strlen(request));
        if (n < 0) 
            error("ERROR writing to socket");

        fprintf(stderr, "%d\n", n);

        bzero(buffer,1024);
        while (read(sockfd,buffer,1023) > 0) {
            printf("%s",buffer);
            bzero(buffer,1024);
        }

        if (n < 0) 
            error("ERROR reading from socket");

        close(sockfd);

        return 0;
}

The python code:

import socket

request = """POST / HTTP/1.0
User-Agent: xmlrpc
Host: wks-13-15
Content-Type: text/xml
Content-Length: 4479

<?xml version="1.0" encoding="UTF-8"?><methodCall><methodName>getGeneExtraInfo</methodName><params><param><value><string>hg19</string></value></param><param><value><string>GO</string></value></param><param><value><array><data><value><string>GO:0044428</string></value><value><string>GO:0044422</string></value><value><string>GO:0044425</string></value><value><string>GO:0044424</string></value><value><string>GO:0043412</string></value><value><string>GO:0006464</string></value><value><string>GO:0009889</string></value><value><string>GO:0003824</string></value><value><string>GO:0016020</string></value><value><string>GO:0016021</string></value><value><string>GO:0048522</string></value><value><string>GO:0048523</string></value><value><string>GO:0090304</string></value><value><string>GO:0019538</string></value><value><string>GO:0051171</string></value><value><string>GO:0001882</string></value><value><string>GO:0001883</string></value><value><string>GO:0080090</string></value><value><string>GO:0042221</string></value><value><string>GO:0048869</string></value><value><string>GO:0019222</string></value><value><string>GO:0005488</string></value><value><string>GO:0005886</string></value><value><string>GO:0005524</string></value><value><string>GO:0031090</string></value><value><string>GO:0050896</string></value><value><string>GO:0010556</string></value><value><string>GO:0010468</string></value><value><string>GO:0016740</string></value><value><string>GO:0003677</string></value><value><string>GO:2000112</string></value><value><string>GO:0005622</string></value><value><string>GO:0019219</string></value><value><string>GO:0006139</string></value><value><string>GO:0032502</string></value><value><string>GO:0032501</string></value><value><string>GO:0050794</string></value><value><string>GO:0009058</string></value><value><string>GO:0032991</string></value><value><string>GO:0044249</string></value><value><string>GO:0044260</string></value><value><string>GO:0044267</string></value><value><string>GO:0035639</string></value><value><string>GO:0009987</string></value><value><string>GO:0044464</string></value><value><string>GO:0051252</string></value><value><string>GO:0043170</string></value><value><string>GO:0005634</string></value><value><string>GO:0005737</string></value><value><string>GO:0050789</string></value><value><string>GO:0031326</string></value><value><string>GO:0051716</string></value><value><string>GO:0016787</string></value><value><string>GO:0031323</string></value><value><string>GO:0006810</string></value><value><string>GO:0048856</string></value><value><string>GO:0065007</string></value><value><string>GO:0043227</string></value><value><string>GO:0043167</string></value><value><string>GO:0044459</string></value><value><string>GO:0043169</string></value><value><string>GO:0008150</string></value><value><string>GO:0008152</string></value><value><string>GO:0006355</string></value><value><string>GO:0005575</string></value><value><string>GO:0046914</string></value><value><string>GO:0003674</string></value><value><string>GO:0006807</string></value><value><string>GO:0003676</string></value><value><string>GO:0044446</string></value><value><string>GO:0044444</string></value><value><string>GO:0051234</string></value><value><string>GO:0032555</string></value><value><string>GO:0043228</string></value><value><string>GO:0043229</string></value><value><string>GO:0043226</string></value><value><string>GO:0045449</string></value><value><string>GO:0032559</string></value><value><string>GO:0031224</string></value><value><string>GO:0017076</string></value><value><string>GO:0071842</string></value><value><string>GO:0071841</string></value><value><string>GO:0071840</string></value><value><string>GO:0060255</string></value><value><string>GO:0016043</string></value><value><string>GO:0034641</string></value><value><string>GO:0008270</string></value><value><string>GO:0000166</string></value><value><string>GO:0046872</string></value><value><string>GO:0044237</string></value><value><string>GO:0044238</string></value><value><string>GO:0043234</string></value><value><string>GO:0043231</string></value><value><string>GO:0043232</string></value><value><string>GO:0032553</string></value><value><string>GO:0005515</string></value><value><string>GO:0007165</string></value><value><string>GO:0048519</string></value><value><string>GO:0048518</string></value><value><string>GO:0030554</string></value></data></array></value></param></params></methodCall>"""

HOST = "wks-13-15"
PORT = 56572

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(request)

while 1:
        data = s.recv(1024)
        print data
        if not data: break

s.close()

Both programs work and receives the entire response data.

I already tried to change the encoding, change how the data is read by the PHP, but the incomplete data and Connection reset by peer problem persist.

My questions: (feel free to answer any of them :-) )

Why the C and Python programs works, while PHP sometimes doesn't?
Why sometimes PHP code works, and others doesn't?
Why with the server running on a python 2.4, the PHP client code works, and with python 2.5 and 2.6 doesn't?
How can I solve it? Suggestions are welcome.

@Electronick thank you for observation, but the problem persists. (I will update the code example). — Pih, Feb 23 '12 at 06:23
maybe try to change socket for fsockopen? "Connection reset by peer" indicates that you have connection problems (ping? socket writing? packet lost?) — Electronick, Feb 23 '12 at 06:35
I suggest using Wireshark, a packet sniffer, to find out when the server resets the connection. — rodion, Feb 29 '12 at 17:05

Mike L. · Answer 1 · 2012-03-07T12:47:25.810

I use the xml_rpc class, I am not sure which version of PHP this started with but it works for me. It even will encode your request from an array, but if you already have your XML for the request I would do this:

PHP

function sendRequest($host, $url, $request, $port = 80) {

//create the context to send to the xmlrpc server
$context = stream_context_create(array('http' => array(
    'method' => "POST",
    'header' => "Content-Type: text/xml\r\nUser-Agent: PHPRPC/1.0\r\n",
    'content' => $request
)));

//i am not sure how to get the url, normally something like http://server/api/xml
$server = "http://$host:$port"; //?

//store the response
$file = file_get_contents($server, false, $context);
//decode the response to xml
$return xmlrpc_decode($file);
}

I updated to mimic your function, I am unsure as to what your server url looks like, just glancing at the code I see you are decoding the host name so you can figure out how to point the `$server` url to your host but this works for every xmlrpc request I have ever attempted to make with PHP. — Mike L., Mar 07 '12 at 12:49

PHP xmlrpc client and Python 2.5 xmlrpc server: incomplete data and Connection reset by peer error

1 Answers1