How to trigger authenticated Jenkins job with file parameter using standard Python library

Question

We are currently triggering Jenkins jobs from a Python script with the help of PycURL. We would like, however, to get rid of the PycURL dependency, but have had little success so far. What makes our scenario more complicated is that we need to post a file as a parameter. Our current PycURL logic for posting the request looks as follows:

url = "https://myjenkins/job/myjob/build"
with contextlib.closing(pycurl.Curl()) as curl:
    curl.setopt(pycurl.URL, url)
    curl.setopt(pycurl.USERPWD, "myuser:mypassword")
    curl.setopt(pycurl.SSL_VERIFYPEER, False)
    curl.setopt(pycurl.SSL_VERIFYHOST, False)
    curl.setopt(pycurl.FAILONERROR, True)
    data = [
            ("name", "integration.xml"),
            ("file0", (pycurl.FORM_FILE, "integration.xml")),
            ("json", "{'parameter': [{'name': 'integration.xml', 'file': 'file0'}]}"),
            ("Submit", "Build"),
            ]
    curl.setopt(pycurl.HTTPPOST, data)
    try:
        curl.perform()
    except pycurl.error, err:
        raise JenkinsTriggerError(curl.errstr())

How can we replace this with facilities from the standard Python library?

We've tried before, but had to give up as we could not see how to upload files successfully, as you can see from my question on that issue.

You seem to be using `https`. Beware that `httplib` and `urllib2` don't verify the server certificate when connecting to an HTTPS website (see official documentation), so it won't be able to establish the connection securely. (You can work around this by wrapping the `httplib` connection using the `ssl` module.) — Bruno, Dec 05 '11 at 10:55
If you look at my code, you'll see that I'm telling cURL to ignore HTTPS certificate :) It's an intranet site, so it shouldn't matter. — aknuds1, Dec 05 '11 at 11:05
@dragosrsupercool Because pycURL is a dependency I'd rather avoid. — aknuds1, Mar 15 '12 at 08:59
pycURL is based on libcurl (pure C code) which has proven better performance and stability over urllib and httplib. Moreover, every linux machine comes with cURL while pycURL is just a single api code to that cURL. — codersofthedark, Mar 15 '12 at 09:54
@aknuds1: too heavy? how? Can u elaborate ur observation? I m the new admin of pycURL and can ensure some solution in the next release we are planning for next month.. . — codersofthedark, Mar 15 '12 at 13:01
@dragosrsupercool We have to build/install both that and cURL manually on Windows — aknuds1, Mar 15 '12 at 14:02
@aknuds1: hw abt an exe file ? one click installation fr windows? — codersofthedark, Mar 15 '12 at 14:08
@dragosrsupercool That would definitely be an improvement. I guess automatic installation via EasyInstall wouldn't be feasible (due to the libcurl dependency)? — aknuds1, Mar 19 '12 at 09:26
lemme discuss if we can include libcurl with pycurl within a single package.. . — codersofthedark, Mar 20 '12 at 03:34
@aknuds1: any updates? Can you post code (maybe what you had in that other question)? — TryPyPy, Apr 24 '12 at 02:21
@TryPyPy See my answer, that's the best I could come up with. — aknuds1, Aug 12 '12 at 19:30

score 9 · Answer 1 · answered Mar 04 '17 at 01:14

We can do it with the help of requests library only.

import requests

payload = ( ('file0', open("FILE_LOCATION_ON_LOCAL_MACHINE", "rb")), 
            ('json', '{ "parameter": [ { 
                                         "name":"FILE_LOCATION_AS_SET_IN_JENKINS", 
                                         "file":"file0" }]}' ))

resp = requests.post("JENKINS_URL/job/JOB_NAME/build", 
                   auth=('username','password'), 
                   headers={"Jenkins-Crumb":"9e1cf46405223fb634323442a55f4412"}, 
                   files=payload )

Jekins-Crumb if required can be obtained using:

requests.get('http://username:password@JENKINS_URL/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')

The Jenkins Crumb isn't needed if an API token is used in place of the user's password — recvfrom, May 21 '21 at 00:03
Also it seems like you would need to call `read(...)` in addition to `open(..., "rb"))`. Otherwise a zero-byte file would be transferred to your Jenkins workspace. aknuds1's answer below calls both functions on the uploaded file. — jrbe228, May 12 '22 at 15:12

aknuds1 · Accepted Answer · 2014-04-18T17:47:40.347

6

I found a solution, using the requests and urllib3 libraries. Not entirely standard, but more lightweight than the PycURL dependency. It should be possible to do this directly with requests (avoiding the urllib3 part), but I ran into a bug.

import urllib3, requests, json

url = "https://myjenkins.com/job/myjob"

params = {"parameter": [
    {"name": "integration.xml", "file": "file0"},
    ]}
with open("integration.xml", "rb") as f:
    file_data = f.read()
data, content_type = urllib3.encode_multipart_formdata([
    ("file0", (f.name, file_data)),
    ("json", json.dumps(params)),
    ("Submit", "Build"),
    ])
resp = requests.post(url, auth=("myuser", "mypassword"), data=data,
        headers={"content-type": content_type}, verify=False)
resp.raise_for_status()

edited Apr 18 '14 at 17:47

answered Aug 09 '12 at 10:27

aknuds1

65,625
67
195
317

So if the build requires a token, it would be added along with the parameters? I tried that and it doesn't seem to build – asm Aug 15 '14 at 18:21
* doesn't seem to build with the parameter being set – asm Aug 15 '14 at 18:31
@Joiya550 You should replicate how tokens are normally sent (ie. In the browser). – aknuds1 Aug 16 '14 at 00:18

mainframer · Answer 3 · 2015-05-21T09:48:20.993

If you are familiar with python, then you can use the jenkins REST APT python wrapper provided by the official site. see this link.

Trigger a build is unbelievably easy by using this python wrapper. Here is my example:

#!/usr/bin/python
import jenkins

if __name == "main":
    j = jenkins.Jenkins(jenkins_server_url, username="youruserid", password="yourtoken")
    j.build_job(yourjobname,{'param1': 'test value 1', 'param2': 'test value 2'},
                    {'token': "yourtoken"})

For those who don't know where to find the token, here is how:

login to jenkins -> click your userid from the top of the webpage -> Configure ->Show API Token...

Enjoy it.

It doesn't include a file parameter however. – aknuds1 Apr 12 '16 at 12:31 — aknuds1, Apr 12 '16 at 12:31

score 1 · Answer 4 · answered Apr 24 '12 at 11:34

Probably it can look something like this:

url = "https://myjenkins/job/myjob/build"
req = urllib2.Request(url)

auth = 'Basic ' + base64.urlsafe_b64encode("myuser:mypassword")
req.add_header('Authorization', auth)

with open("integration.xml", "r") as f:
  file0 = f.read()
  data = {
            "name": "integration.xml",
            "file0": file0,
            "json": "{'parameter': [{'name': 'integration.xml', 'file': 'file0'}]}",
            "Submit": "Build"
         }
  req.add_data(urllib.urlencode(data))

urllib2.urlopen(req)

Sorry, I don't have installed Jenkins around to test it out.

Kamesh Jungi · Answer 5 · 2016-04-12T07:28:41.010

0

Another Alternative that I used :

import requests
import json
url = "https://myjenkins/job/myjob/build"
payload = {'key1': 'value1', 'key2': 'value2'}
resp = requests.post(url, params=payload, auth=("username", "password"),verify=False)
json_data = json.loads(resp.text)

For more details you can refer :Make a Request

edited Apr 12 '16 at 07:28

answered Apr 12 '16 at 05:20

Kamesh Jungi

6,203
6
39
50

@aknuds1 Improved answer to post request with parameters. Hope this will resolve your question. – Kamesh Jungi Apr 12 '16 at 07:29
Does it show how to post a file however? Take a look at my question and answer for reference. – aknuds1 Apr 12 '16 at 07:42
Yes, when you print json_data you can retrieve response text. Try once.. you will find output. – Kamesh Jungi Apr 12 '16 at 07:46
I don't think you understand what I mean. The question is how to post a file to Jenkins, my answer handles this. – aknuds1 Apr 12 '16 at 08:05

jmunsch · Answer 6 · 2016-12-07T05:55:50.357

Here is a similar version to aknuds1 answer where test_result is the xml string:

j_string = "{'parameter': [{'name': 'integration_tests.xml', 'file': 'someFileKey0'}]}"
data = {
          "name": "integration_tests.xml",
          "json": j_string, 
        }
for xml_string in tests.values():
    post_form_result = requests.post('http://jenkins/job/deployment_tests/build',
                                     data=data,
                                     files={'someFileKey0': xml_string})
    print(post_form_result.status_code)

Taking a guess, additional parameters would be passed in as part of the json string array, or additional files, etc. Let me know if this is the case, also, if I find out, i'll update this answer. This solution worked nicely to trigger JUnit tests.

Version:

master* $ pip show requests                                                                                                                                                                      [21:45:05]
Name: requests
Version: 2.12.3
Summary: Python HTTP for Humans.
Home-page: http://python-requests.org
Author: Kenneth Reitz
Author-email: me@kennethreitz.com
License: Apache 2.0
Location: /usr/local/lib/python2.7/site-packages

How to trigger authenticated Jenkins job with file parameter using standard Python library

6 Answers6