-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major
-
Component/s: workflow-job-plugin
-
None
-
Environment:Jenkins: 2.164.1
Pipeline plugins:
* Build Step: 2.8
* Job: 2.32
We have a pipeline which in Test stage executes a lot of new jobs by using build step (we cannot move the jobs logic into this pipeline due to JENKINS-52391 - we don't want to rebuild 120 jobs because 1 has randomly failed):
stage('Test') { steps { script { def parallelTests = [:] def recipes = env.TESTED_RECIPES.split(',') def testSplit = env.TEST_SPLIT != null ? env.TEST_SPLIT.toInteger() : 1 def testWebSplit = env.WEBTEST_SPLIT != null ? env.WEBTEST_SPLIT.toInteger() : 1 for (def recipe in recipes) { for (def index = 1; index <= testSplit; ++index) { parallelTests["Test JUnit $recipe $index/$testSplit"] = createTestExecutionStage('generic-test-junit', [ string(name: 'PIPELINE_ID', value: params.PIPELINE_ID), string(name: 'RECIPE', value: recipe), string(name: 'TEST_SPLIT_IDX', value: "${index}"), string(name: 'TEST_SPLIT_MAX', value: "${testSplit}"), string(name: 'PRIORITY', value: params.PRIORITY) ]) } for (def index = 1; index <= testWebSplit; ++index) { parallelTests["Test Web JUnit $recipe $index/$testWebSplit"] = createTestExecutionStage('generic-test-junit', [ string(name: 'PIPELINE_ID', value: params.PIPELINE_ID), string(name: 'RECIPE', value: recipe), booleanParam(name: 'WEB_TESTS', value: true), string(name: 'TEST_SPLIT_IDX', value: "${index}"), string(name: 'TEST_SPLIT_MAX', value: "${testWebSplit}"), string(name: 'PRIORITY', value: params.PRIORITY) ], false) } parallelTests["Test Initialization $recipe"] = createTestExecutionStage('generic-test-initialization', [ string(name: 'PIPELINE_ID', value: params.PIPELINE_ID), string(name: 'RECIPE', value: recipe), string(name: 'PRIORITY', value: params.PRIORITY) ]) parallelTests["Test Server $recipe"] = createTestExecutionStage('generic-test-server', [ string(name: 'PIPELINE_ID', value: params.PIPELINE_ID), string(name: 'RECIPE', value: recipe), string(name: 'PRIORITY', value: params.PRIORITY) ]) } parallel parallelTests } } }
def createTestExecutionStage(name, parameters, propagate = true) { return { build(job: name, parameters: parameters, propagate: propagate) } }
Unfortunately, very rarely we see the following entries in logs:
Apr 04, 2019 4:30:04 AM WARNING hudson.model.ParametersAction filter Skipped parameter `PIPELINE_ID` as it is undefined on `generic-test-server`. Set `-Dhudson.model.ParametersAction.keepUndefinedParameters=true` to allow undefined parameters to be injected as environment variables or `-Dhudson.model.ParametersAction.safeParameters=[comma-separated list]` to whitelist specific parameter names, even though it represents a security breach or `-Dhudson.model.ParametersAction.keepUndefinedParameters=false` to no longer show this message. Apr 04, 2019 4:30:04 AM WARNING hudson.model.ParametersAction filter Skipped parameter `RECIPE` as it is undefined on `generic-test-server`. Set `-Dhudson.model.ParametersAction.keepUndefinedParameters=true` to allow undefined parameters to be injected as environment variables or `-Dhudson.model.ParametersAction.safeParameters=[comma-separated list]` to whitelist specific parameter names, even though it represents a security breach or `-Dhudson.model.ParametersAction.keepUndefinedParameters=false` to no longer show this message. Apr 04, 2019 4:30:04 AM WARNING hudson.model.ParametersAction filter Skipped parameter `PRIORITY` as it is undefined on `generic-test-server`. Set `-Dhudson.model.ParametersAction.keepUndefinedParameters=true` to allow undefined parameters to be injected as environment variables or `-Dhudson.model.ParametersAction.safeParameters=[comma-separated list]` to whitelist specific parameter names, even though it represents a security breach or `-Dhudson.model.ParametersAction.keepUndefinedParameters=false` to no longer show this message.
It of course breaks the build because parameters are missing. This is how generic-test-junit looks after I clicked Build with Parameters:

In most cases it looks correct:

But sometimes it is broken:

The generic-junit-test job is generated by Job DSL plugin:
pipelineJob('generic-test-junit') { displayName('Generic JUnit Test') description('''Multi-line description is here''') logRotator { daysToKeep(90) } parameters { stringParam('PIPELINE_ID', '', 'Identifier of the pipeline which will be built.') stringParam('RECIPE', '', 'Recipe which will be tested.') booleanParam('WEB_TESTS', false, 'Defines whether web tests should be executed instead of the non-web tests.') stringParam('TEST_SPLIT_IDX', '', 'Index of the test package to be performed.') stringParam('TEST_SPLIT_MAX', '', 'Number of the test packages.') booleanParam('DEBUG', false, 'Defines whether test target should be executed in debug mode.') stringParam('PRIORITY', '', 'Pipeline priority (number from 1 to 5, where 1 is the highest).') } definition { cps { script(readFileFromWorkspace('jobs/genericTestJunit.jenkinsfile')) sandbox() } } }
This is the definition of generic-junit-test job (jobs/genericTestJunit.jenkinsfile):
pipeline {
agent {
label 'swarm'
}
options {
ansiColor('xterm')
}
stages {
stage('Test') {
steps {
cleanWs()
sh """#!/bin/bash
rsync -az jkmaster:build-scripts .
rsync -az jkmaster:resources/${params.PIPELINE_ID}/pipeline.properties .
"""
script {
def environmentVariables = ""
readProperties(file: 'pipeline.properties')
.findAll { !params.containsKey(it.key) }
.each { k, v -> environmentVariables = "${environmentVariables}env.${k}='${v}'\n" }
writeFile file: 'env.groovy', text: environmentVariables
}
load 'env.groovy'
script {
currentBuild.displayName = "#${env.BUILD_NUMBER}-${env.PRODUCT_NAME}-${params.PIPELINE_ID}-${params.RECIPE}-${params.WEB_TESTS ? 'WEB-' : ''}${params.TEST_SPLIT_IDX}-${params.TEST_SPLIT_MAX}"
}
timestamps {
lock(resource: null, label: 'rsync', quantity: 1) {
sh './build-scripts/synchronize-with-jkmaster.sh'
}
}
wrap([$class: 'LogfilesizecheckerWrapper', failBuild: true, maxLogSize: 0, setOwn: false]) {
wrap([$class: 'Xvfb', displayNameOffset: 2]) {
timestamps {
timeout(time: 6, unit: 'HOURS') {
timeout(time: 15, activity: true) {
sh "./build-scripts/jobs/test-junit.sh ${params.WEB_TESTS ? 'web' : ''}"
}
}
}
}
}
junit 'junit/**/*.xml'
archiveArtifacts artifacts: 'junit/**/*.xml'
}
}
}
post {
always {
cleanWs()
}
}
}
(I know the code is not very pretty (rsync etc.) - we are migrating very old Jenkins to newer technologies
).
At this moment we are forced to use this flag -Dhudson.model.ParametersAction.keepUndefinedParameters=true.