Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-21589

Unresolvable (environment) variables lead to failing maven builds

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major
    • core, maven-plugin
    • None
    • Jenkins 1.532.1 (LTS); JRE 7; Unix Daemon

      maven-plugin 2.0.3

      The following scenario leads to failing builds because of faulty behaviour of Jenkins:

      • create a global environment variable:
      • create a job (maven or freestyle; did not test it on matrix)
      • configure maven: (enter the global env variable)
      • run the job
      • you get something like:
        Modules changed, recalculating dependency graph
        maven-agent.jar already up to date
        classworlds.jar already up to date
        maven-interceptor.jar already up to date
        maven2.1-interceptor.jar already up to date
        [amavenjob] $ java -Xmx512m -XX:MaxPermSize=256m -cp PATH/jenkins-slave/maven-agent.jar:PATH/jenkins-slave/classworlds.jar hudson.maven.agent.Main PATH/maven/current PATH/jenkins-slave/slave.jar PATH/jenkins-slave/maven-interceptor.jar 40919 PATH/jenkins-slave/maven2.1-interceptor.jar
        
        <===[JENKINS REMOTING CAPACITY]===>channel started
        
        log4j:WARN No appenders could be found for logger (org.apache.commons.beanutils.converters.BooleanConverter).
        log4j:WARN Please initialize the log4j system properly.
        
        Executing Maven:  -B -f PATH/jenkins-slave/workspace/amavenjob/pom.xml clean install ${JOB_MVN_PARAM}
        [INFO] Scanning for projects...
        [INFO] ------------------------------------------------------------------------
        [ERROR] BUILD FAILURE
        [INFO] ------------------------------------------------------------------------
        [INFO] Invalid task '${JOB_MVN_PARAM}': you must specify a valid lifecycle phase, or a goal in the format plugin:goal or pluginGroupId:pluginArtifactId:pluginVersion:goal
        [INFO] ------------------------------------------------------------------------
        [INFO] For more information, run Maven with the -e switch
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: < 1 second
        [INFO] Finished at: Fri Jan 31 09:12:40 CET 2014
        [INFO] Final Memory: 3M/59M
        [INFO] ------------------------------------------------------------------------
        
        channel stopped
        Finished: FAILURE
        

          [JENKINS-21589] Unresolvable (environment) variables lead to failing maven builds

          Stephan Krull added a comment -

          Maven is not able to ignore a placeholder in its commandline. Jenkins has to do that just before calling maven.

          Maybe other tools are affected too.

          I propose to filter the maven goals before committing them to a Builder.

          For a private hotfix I used this method:

          private List<String> getListWithoutUnresolvableVariables(ArgumentListBuilder alb)
          {
          	List<String> result = new LinkedList<String>();
          	//  pattern taken from hudson.Util.VARIABLE
          	final Pattern varPattern = Pattern.compile("\\$([A-Za-z0-9_]+|\\{[A-Za-z0-9_.]+\\}|\\$)");
          	
          	for( String entry : alb.toList())
          	{
          		Matcher m = varPattern.matcher(entry);
          		final String intern = m.replaceAll("").trim();
          		if (0 < intern.length())
          		{
          			result.add(intern);
          		}
          	}
          	return result;
          }
          

          to modify the calculated hudson.util.ArgumentListBuilder in hudson.maven.MavenModuleSetBuild.MavenModuleSetBuildExecution.doRun(BuildListener). This works well for Maven jobs but not for freestyle jobs.

          Stephan Krull added a comment - Maven is not able to ignore a placeholder in its commandline. Jenkins has to do that just before calling maven. Maybe other tools are affected too. I propose to filter the maven goals before committing them to a Builder . For a private hotfix I used this method: private List< String > getListWithoutUnresolvableVariables(ArgumentListBuilder alb) { List< String > result = new LinkedList< String >(); // pattern taken from hudson.Util.VARIABLE final Pattern varPattern = Pattern.compile( "\\$([A-Za-z0-9_]+|\\{[A-Za-z0-9_.]+\\}|\\$)" ); for ( String entry : alb.toList()) { Matcher m = varPattern.matcher(entry); final String intern = m.replaceAll("").trim(); if (0 < intern.length()) { result.add(intern); } } return result; } to modify the calculated hudson.util.ArgumentListBuilder in hudson.maven.MavenModuleSetBuild.MavenModuleSetBuildExecution.doRun(BuildListener) . This works well for Maven jobs but not for freestyle jobs.

          Stephan Krull added a comment -

          Maybe someone is interested in the usecase of the scenario described above:
          We are using homogeneous maven commands for all maven jobs that are based on company standard. Additionally these jobs are created from a template where the global environment variables are configured. Now and then a developer needs additional parameters added to the standard maven command. With Hudson I defined a placeholder environment variable without a value, the developer did an overwrite of this variable on his maven job. With Jenkins overwriting an environment variable does not work. I deleted the placeholder from the global environments an got the error message that is base to this issue because the placeholders are not configured in every job.

          Stephan Krull added a comment - Maybe someone is interested in the usecase of the scenario described above: We are using homogeneous maven commands for all maven jobs that are based on company standard. Additionally these jobs are created from a template where the global environment variables are configured. Now and then a developer needs additional parameters added to the standard maven command. With Hudson I defined a placeholder environment variable without a value, the developer did an overwrite of this variable on his maven job. With Jenkins overwriting an environment variable does not work. I deleted the placeholder from the global environments an got the error message that is base to this issue because the placeholders are not configured in every job.

          Oleg Nenashev added a comment -

          Jenkins does not support resolution on nested macros in global variables.
          I suggest to use EnvInject in job or node configuration if you want to get the described behavior.

          Oleg Nenashev added a comment - Jenkins does not support resolution on nested macros in global variables. I suggest to use EnvInject in job or node configuration if you want to get the described behavior.

          Oleg Nenashev added a comment -

          Marked as "not a defect".
          Please reopen the issue if you don't agree

          Oleg Nenashev added a comment - Marked as "not a defect". Please reopen the issue if you don't agree

          Stephan Krull added a comment - - edited

          The scenario described here does not fail with maven-plugin version 2.7 on Jenkins LTS 1.565.3 anymore.

          Although the plugin is able to resolve the global environment variable key while running (see log excerpt below) the output of the "resolved" maven command line still shows the placeholder ("${VAL_UE}")

          Current scenario:

          • global environment variable: MAVEN_VAR_TEST=-Dclean.skip=§{VAL_UE}
          • maven job with simple pom, no clean-plugin config
          • job parameter boolean: VAL_UE
          • running maven goal: "clean ${MAVEN_VAR_TEST}"

          build log with VAL_UE=true (excerpt):

          ...
          Parsing POMs
          maven31-agent.jar already up to date
          maven31-interceptor.jar already up to date
          maven3-interceptor-commons.jar already up to date
          ...
          <===[JENKINS REMOTING CAPACITY]===>channel started
          Executing Maven:  -B -f SLAVE_HOME/workspace/maventest/pom.xml clean -Dclean.skip=${VAL_UE}
          [INFO] Scanning for projects...
          [INFO]                                                                         
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Maven Project 4.1-SNAPSHOT
          [INFO] ------------------------------------------------------------------------
          [INFO] 
          [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ maven.project ---
          [INFO] Clean is skipped.
          [JENKINS] Archiving disabled
          Started calculate disk usage of build
          Finished Calculation of disk usage of build in 0 seconds
          Started calculate disk usage of workspace
          Finished Calculation of disk usage of workspace in 0 seconds
          [JENKINS] Archiving disabled[INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 1.000 s
          [INFO] Finished at: 2014-10-15T13:56:47+02:00
          
          [INFO] Final Memory: 7M/107M
          [INFO] ------------------------------------------------------------------------
          

          build log with VAL_UE=false (excerpt):

          ...
          Parsing POMs
          maven31-agent.jar already up to date
          maven31-interceptor.jar already up to date
          maven3-interceptor-commons.jar already up to date
          ...
          <===[JENKINS REMOTING CAPACITY]===>channel started
          Executing Maven:  -B -f /home/cc/jenkins-slave/workspace/maventest/pom.xml clean -Dclean.skip=${VAL_UE}
          [INFO] Scanning for projects...
          [INFO]                                                                         
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Maven Project 4.1-SNAPSHOT
          [INFO] ------------------------------------------------------------------------
          [INFO] 
          [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ maven.project ---
          [JENKINS] Archiving disabled
          Started calculate disk usage of build
          Finished Calculation of disk usage of build in 0 seconds
          Started calculate disk usage of workspace
          Finished Calculation of disk usage of workspace in 0 seconds
          [JENKINS] Archiving disabled[INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 0.969 s
          
          [INFO] Finished at: 2014-10-15T14:07:34+02:00
          [INFO] Final Memory: 6M/106M
          [INFO] ------------------------------------------------------------------------
          

          Stephan Krull added a comment - - edited The scenario described here does not fail with maven-plugin version 2.7 on Jenkins LTS 1.565.3 anymore. Although the plugin is able to resolve the global environment variable key while running (see log excerpt below) the output of the "resolved" maven command line still shows the placeholder ("${VAL_UE}") Current scenario: global environment variable: MAVEN_VAR_TEST=-Dclean.skip=§{VAL_UE } maven job with simple pom, no clean-plugin config job parameter boolean: VAL_UE running maven goal: "clean ${MAVEN_VAR_TEST}" build log with VAL_UE=true (excerpt): ... Parsing POMs maven31-agent.jar already up to date maven31-interceptor.jar already up to date maven3-interceptor-commons.jar already up to date ... <===[JENKINS REMOTING CAPACITY]===>channel started Executing Maven: -B -f SLAVE_HOME/workspace/maventest/pom.xml clean -Dclean.skip=${VAL_UE} [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Maven Project 4.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ maven.project --- [INFO] Clean is skipped. [JENKINS] Archiving disabled Started calculate disk usage of build Finished Calculation of disk usage of build in 0 seconds Started calculate disk usage of workspace Finished Calculation of disk usage of workspace in 0 seconds [JENKINS] Archiving disabled[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.000 s [INFO] Finished at: 2014-10-15T13:56:47+02:00 [INFO] Final Memory: 7M/107M [INFO] ------------------------------------------------------------------------ build log with VAL_UE=false (excerpt): ... Parsing POMs maven31-agent.jar already up to date maven31-interceptor.jar already up to date maven3-interceptor-commons.jar already up to date ... <===[JENKINS REMOTING CAPACITY]===>channel started Executing Maven: -B -f /home/cc/jenkins-slave/workspace/maventest/pom.xml clean -Dclean.skip=${VAL_UE} [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Maven Project 4.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ maven.project --- [JENKINS] Archiving disabled Started calculate disk usage of build Finished Calculation of disk usage of build in 0 seconds Started calculate disk usage of workspace Finished Calculation of disk usage of workspace in 0 seconds [JENKINS] Archiving disabled[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.969 s [INFO] Finished at: 2014-10-15T14:07:34+02:00 [INFO] Final Memory: 6M/106M [INFO] ------------------------------------------------------------------------

          Stephan Krull added a comment -

          Regarding the unresolved variable in the execution statement (see former comment) there is an open bug: JENKINS-18030

          Stephan Krull added a comment - Regarding the unresolved variable in the execution statement (see former comment) there is an open bug: JENKINS-18030

            Unassigned Unassigned
            krulls Stephan Krull
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: