POST request remake from Java to python gets null

Question

I am totally new to python flask and encountered a problem when writing some code using the requests and flask modules.

I am working on a project using the web API offered by the Panther platform. The project provided an example using Apache Java.

The source code is as below (see more for details).

public class TestProject {

public static void main(String args[]) throws Exception {
    CloseableHttpClient httpclient = HttpClients.createDefault();
    try {
        HttpPost httppost = new HttpPost("http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?");

        StringBody organism = new StringBody("Homo sapiens", ContentType.TEXT_PLAIN);
        FileBody fileData = new FileBody(new File("c:\\data_files\\gene_expression_files\\7_data\\humanEnsembl"), ContentType.TEXT_PLAIN);
        StringBody enrichmentType = new StringBody("process", ContentType.TEXT_PLAIN);       
        StringBody testType = new StringBody("FISHER", ContentType.TEXT_PLAIN);    
        //StringBody cor = new StringBody("FDR", ContentType.TEXT_PLAIN);
        //StringBody cor = new StringBody("BONFERRONI", ContentType.TEXT_PLAIN);
        //StringBody cor = new StringBody("NONE", ContentType.TEXT_PLAIN);
        StringBody type = new StringBody("enrichment", ContentType.TEXT_PLAIN);

        HttpEntity reqEntity = MultipartEntityBuilder.create()
                .addPart("organism", organism)
                .addPart("geneList", fileData)
                .addPart("enrichmentType", enrichmentType)
                .addPart("test_type", testType)
                .addPart("type", type)
                //.addPart("correction", cor)
                .build();
        httppost.setEntity(reqEntity);

        CloseableHttpResponse response = httpclient.execute(httppost);
        try {
        //System.out.println("----------------------------------------");
       //System.out.println(response.getStatusLine());
            HttpEntity resEntity = response.getEntity();
            if (resEntity != null) {
                System.out.println(IOUtils.toString(resEntity.getContent(), StandardCharsets.UTF_8));

            }
            EntityUtils.consume(resEntity);
        } finally {
            response.close();
        }
    } finally {
        httpclient.close();
    }
  }
}

The part I am mostly interested in is .addPart("organism", organism) and all the other code with similar structures. They will help pass the parameters from a third-party website to the web API offered by Panther.

I remade the JAVA code into python3 using requests. The code is as follows:

uploadTemp = {'file':open('./app/static/data_temp/temp.txt','rb')}

url="http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?"
params = {"organism":organism,"geneList":pantherName,"enrichmentType":"fullGO_process","test_type":"BINOMIAL","type":"enrichment","correction":"BONFERRONI"}

# or params = {"organism":organism,"geneList":uploadTemp,"enrichmentType":"fullGO_process","test_type":"BINOMIAL","type":"enrichment","correction":"BONFERRONI"} 

Pantherpost= requests.post(url, params = params)
print(Pantherpost.text)

I am expecting an XML object from the web API including some basic biological information. However, the result I got was null (or \n\n\rnull\n when I print Pantherpost.content)

It seems that the parameters I have got from my own web were not correctly sent to the web API.

In addition to this getting null problem, as a beginner, I am also not quite sure about whether the "geneList" part should be receiving a plain-text object or a file. The manual says it is expecting a file, however, it may have been reformatted into plain-text by this command

FileBody fileData = new FileBody(new File("c:\\data_files\\gene_expression_files\\7_data\\humanEnsembl"), ContentType.TEXT_PLAIN);

Anyway, I did try both interpretations: pantherName is a list with name correctly formatted in plain-text and uploadTemp is a .txt file generated for the project. There must be some extra bugs in my code since it returned null in both cases.

Can someone please help out? Thank you very much.

Answer 1

I've found the following issues with your python code:

One. If you want to POST a file using requests, you should use keyword files=.

Two. Keys in files object should match respective parameters of the request (you're using file instead).

Three. You put your parameters in the wrong place of the request by writing params=params.

Function annotation from requests source code:

:param params: (optional) Dictionary or bytes to be sent in the query string for the :class:Request.

In example Java code StringBody is used to create parameters, which implies that parameters should be placed inside the body of HTTP request, not query string. So you should use data= keyword instead. If you use params=, output will be null.

SO article on difference between data and params keywords in requests.

---

So I've spent some time reading thier manual and made a test script:

import requests


url = "http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?"
filepath = "C:\\data\\YOUR_DATA.txt"  # change to your file location

# all required parameters according to manual, except geneList which is a file (see below)
params = {  # using defaults from manual
    "type": "enrichment",
    "organism": "Homo sapiens",
    "enrichmentType": "process",
    "test_type": "FISHER",
    "correction": "FDR",
}

# note that the key here is the name of paramter: geneList
files = {'geneList': open(filepath, 'rb')}  

# it outputs null, when 'params=params' is used
r = requests.post(url, data=params, files=files)  
print(r.status_code)
print(r.text)

Output:

200
Id  Name    GeneId  raw P-value FDR