bulk size text parameters in jenkins report error

Question

I have a build with a text parameters in Jenkins, when I input big data into text area and try to build, it comes

[TEST-save_txt_to_file] $ /bin/sh -xe /tmp/hudson3834163952953567847.sh
FATAL: command execution failed
java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/jenkins/workspace/TEST-save_txt_to_file"): java.io.IOException: error=7, Argument list too long
    at java.lang.ProcessBuilder.start(Unknown Source)
    at hudson.Proc$LocalProc.(Proc.java:244)
    at hudson.Proc$LocalProc.(Proc.java:216)
    at hudson.Launcher$LocalLauncher.launch(Launcher.java:763)
    at hudson.Launcher$ProcStarter.start(Launcher.java:353)
    at hudson.Launcher$ProcStarter.join(Launcher.java:360)
    at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
    at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:60)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814)
    at hudson.model.Build$BuildExecution.build(Build.java:199)
    at hudson.model.Build$BuildExecution.doRun(Build.java:160)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:593)
    at hudson.model.Run.execute(Run.java:1568)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:237)
Caused by: java.io.IOException: java.io.IOException: error=7, Argument list too long
    at java.lang.UNIXProcess.(Unknown Source)
    at java.lang.ProcessImpl.start(Unknown Source)
    ... 17 more
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Is there any workaround? Thanks!

my ENV:

java.runtime.version 1.6.0_41-b02 org.jenkins-ci.main:jenkins-war:1.502

A typical workaround is to write the argument value to a file and then use an alternate command-line argument that can read all or some arguments form a file. Are you trying to do this already ? If not maybe that could work for your command. — Patrice M., Aug 27 '13 at 00:16
Thank you. It won't help, my job is save_txt_to_file, which already got the error before I can write to a file. — valpa, Aug 29 '13 at 00:35

Tylee · Accepted Answer · 2015-06-08T12:35:43.913

This is caused by a linux limitation whereby the arguments can't exceed 128kb in size See Linux Kernel constant: MAX_ARG_STRLEN https://github.com/torvalds/linux/blob/master/include/uapi/linux/binfmts.h

Within Jenkins once you read from/to a variable where this value is exceeded, you will hit this error. In my case, I had a github webhook that launched a Jenkins job and set a payload parameter to some string > than this limit. Attempting to read this parameter would throw this error.

In order to work around the problem, I have a child job that uses a rest-api call to read the value from parent

You can let the parent job throw a failure, but allow the child job to be launched in all cases. Below is a slightly refined function I used to pull the information (Stripped out error checking and comments for brevity)

def get_parameter_value_from_parent():
    host = 'https://[YOUR_COMPANY].ci.cloudbees.com'
    this_build_url = os.environ.get('BUILD_URL')
    request_auth = (JENKINS_USER, JENKINS_TOKEN)

    url = '{0}/api/json'.format(this_build_url)
    parameter_name = 'payload'
    payload = ''

    #
    # Get the upstreamBuild number, and the upstreamUrl
    # so we can put together a link to the upstream job
    #

    response = requests.get(url, auth=request_auth)
    this_build = json.loads(response)

    build_number = ''
    short_url = ''
    actions = this_build['actions']
    for action in actions:
        if action.get('causes'):
            for cause in action.get('causes'):
                build_number = cause['upstreamBuild']
                short_url = cause['upstreamUrl']

    parent_url = '{host}/{short_url}{build}/api/json'.format(host=host,
            short_url=short_url, build=build_number)

    #
    # Now get the payload from the parent job by making REST api call
    #

    response = requests.get(parent_url, auth=request_auth)
    upstream_build = json.loads(response)

    actions = upstream_build['actions']
    for action in actions:
        if action.get('parameters'):
            for parameter in action.get('parameters'):
                if parameter['name'] == parameter_name:
                    value = parameter['value']
                    payload = value
                    return payload

    print 'Error: Unable to return payload from parent jenkins job: {0}'.format(parent_url)
    sys.exit(1)

score 0 · Answer 2 · answered Aug 29 '13 at 20:10

0

I can get this to work for my purpose by first echoing the parameter value to a file, e.g.:

echo "${BIG_TEXT_PARAM}" > ${WORKSPACE}/BIG_TEXT_PARAM.txt

cat ${WORKSPACE}/BIG_TEXT_PARAM.txt

Note the use of double-quotes in the 'echo' command.

answered Aug 29 '13 at 20:10

Patrice M.

213
1
7

How big is your data? Mine is 132KB. – valpa Sep 02 '13 at 00:24
1

I got it working for >75KB, but when I get close to your value indeed it seems that linux itself has a limit that you can't really overcome anyway (http://www.cyberciti.biz/faq/linux-unix-arg_max-maximum-length-of-arguments/). I'd look next for maybe a different shell (than the standard one) that supports higher (or no) limits. – Patrice M. Sep 03 '13 at 05:40

bulk size text parameters in jenkins report error

2 Answers2