Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40484

Unable to use withMaven() step inside docker container for old versions of Docker

      I'm not able to use withMaven step inside docker container.

      [Pipeline] withMaven
      $ docker exec ffff env printenv MAVEN_HOME
      $ docker exec ffff env printenv M2_HOME
      $ docker exec ffff env /bin/sh -c "which mvn"
      Using maven exec: /opt/apache-maven-3.3.9/bin/mvn
      Using global settings config with name MavenGlobal
      Replacing all maven server entries not found in credentials list is false
      [Pipeline] {
      [Pipeline] sh
      
      [test-XXX] Running shell script
      nohup: failed to run command `sh': No such file or directory
      
      [Pipeline] }
      [Pipeline] // withMaven

      My Jenkinsfile pipeline:

      node('docker') {
          stage('Checkout') {
              checkout scm
          }
          buildInDocker('linux') {
              stage('Maven') {
                  withMaven(globalMavenSettingsConfig: '11111111-2222-3333-4444-555555555555') {
                          sh 'mvn clean test'
                  }
              }
          }
      }

      It looks like PATH env gets overwritten.

          [JENKINS-40484] Unable to use withMaven() step inside docker container for old versions of Docker

          dan tran added a comment -

          I cant test with beta ATM, however, the fix looks good on my sample basic declarative pipeline. I need do

          here is the result of your request

          [INFO]
          Effective user-specific configuration settings:

          <?xml version="1.0" encoding="UTF-8"?>
          <!-- ====================================================================== -->
          <!-- -->
          <!-- Generated by Maven Help Plugin on 2017-11-08T01:04:15 -->
          <!-- See: http://maven.apache.org/plugins/maven-help-plugin/ -->
          <!-- -->
          <!-- ====================================================================== -->

          <!-- ====================================================================== -->
          <!-- -->
          <!-- Effective Settings for '?' on '58a31892ecc5' -->
          <!-- -->
          <!-- ====================================================================== -->

          <settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
          <localRepository xmlns="http://maven.apache.org/SETTINGS/1.1.0">/space/jenkins/skysandbox/workspace/pipeline-test/?/.m2/repository</localRepository>
          <pluginGroups xmlns="http://maven.apache.org/SETTINGS/1.1.0">
          <pluginGroup>org.apache.maven.plugins</pluginGroup>
          <pluginGroup>org.codehaus.mojo</pluginGroup>
          </pluginGroups>
          </settings>

          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 28.352 s

          dan tran added a comment - I cant test with beta ATM, however, the fix looks good on my sample basic declarative pipeline. I need do here is the result of your request [INFO] Effective user-specific configuration settings: <?xml version="1.0" encoding="UTF-8"?> <!-- ====================================================================== --> <!-- --> <!-- Generated by Maven Help Plugin on 2017-11-08T01:04:15 --> <!-- See: http://maven.apache.org/plugins/maven-help-plugin/ --> <!-- --> <!-- ====================================================================== --> <!-- ====================================================================== --> <!-- --> <!-- Effective Settings for '?' on '58a31892ecc5' --> <!-- --> <!-- ====================================================================== --> <settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd "> <localRepository xmlns="http://maven.apache.org/SETTINGS/1.1.0">/space/jenkins/skysandbox/workspace/pipeline-test/?/.m2/repository</localRepository> <pluginGroups xmlns="http://maven.apache.org/SETTINGS/1.1.0"> <pluginGroup>org.apache.maven.plugins</pluginGroup> <pluginGroup>org.codehaus.mojo</pluginGroup> </pluginGroups> </settings> [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 28.352 s

          Tim Downey added a comment -

          dantran and cleclerc – I'm not sure I understand the original problem that led to this issue.  I've been using withDockerContainer & withMaven for months with no issue.  What is the problem that led to this change?

          Since this change any attempt to change the configured settings.xml is ignored and instead goes looking in <workspace>/?/.m2.

          cleclerc – In trying your example, you can see that  [withMaven] is attempting to configure things properly

          [withMaven] use Maven settings.xml 'my-maven-settings' with NO Maven servers credentials provided by Jenkins

          but later when Maven actually runs, this is ignored

          [DEBUG] Reading user settings from ?/.m2/settings.xml

           

          Tim Downey added a comment - dantran and cleclerc – I'm not sure I understand the original problem that led to this issue.  I've been using withDockerContainer & withMaven for months with no issue.  What is the problem that led to this change? Since this change any attempt to change the configured settings.xml is ignored and instead goes looking in <workspace>/?/.m2. cleclerc – In trying your example, you can see that  [withMaven] is attempting to configure things properly [withMaven] use Maven settings.xml 'my-maven-settings' with NO Maven servers credentials provided by Jenkins but later when Maven actually runs, this is ignored [DEBUG] Reading user settings from ?/.m2/settings.xml  

          dan tran added a comment -

          i can confirm that my build does not see config provider settings

          dan tran added a comment - i can confirm that my build does not see config provider settings

          Cyrille Le Clerc added a comment - - edited

          Cyrille Le Clerc added a comment - - edited Please follow-up on JENKINS-47823 . See my comment https://issues.jenkins-ci.org/browse/JENKINS-47823?focusedCommentId=319156&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-319156

          Steve Todorov added a comment -

          cleclerc I would like to start by saying that really appreciate your work and efforts in providing a fix/workaround for this issue. 

          However, I'm quite disappointed by this change. When we upgraded back in November 2017 (JENKINS-47805) all of our builds turned red because of the `$PATH` changes in the Docker Pipeline plugin. This makes me doubt what's the purpose of using `withMaven` if it can't do the things it's supposed to do and we're constantly having to use "workarounds"? To name just a few - we need to always have a `post -> always` block that ensures `jUnit` records the tests, now we have to use `configFileProvider` for the `settings.xml`, maybe use the mvn wrapper script... We could just as "easily" add 4 or 5 more pipeline declarations and do things manually instead of using `withMaven`. So what's exactly does `withMaven` do other than decorate the syntax?

          Steve Todorov added a comment - cleclerc I would like to start by saying that really appreciate your work and efforts in providing a fix/workaround for this issue.  However, I'm quite disappointed by this change. When we upgraded back in November 2017 ( JENKINS-47805 ) all of our builds turned red because of the `$PATH` changes in the Docker Pipeline plugin. This makes me doubt what's the purpose of using `withMaven` if it can't do the things it's supposed to do and we're constantly having to use "workarounds"? To name just a few - we need to always have a `post -> always` block that ensures `jUnit` records the tests, now we have to use `configFileProvider` for the `settings.xml`, maybe use the mvn wrapper script... We could just as "easily" add 4 or 5 more pipeline declarations and do things manually instead of using `withMaven`. So what's exactly does `withMaven` do other than decorate the syntax?

          Joshua Noble added a comment - - edited

          I hit this issue again today for the second time. Docker Pipeline 1.14 is unusable and causes `withMaven` to break when used from within a Docker container. Using containers is one of the best features of pipelines, so this is pretty critical functionality IMO.

          Joshua Noble added a comment - - edited I hit this issue again today for the second time. Docker Pipeline 1.14 is unusable and causes `withMaven` to break when used from within a Docker container. Using containers is one of the best features of pipelines, so this is pretty critical functionality IMO.

          Joshua Noble added a comment -

          Any updates on this?

          Joshua Noble added a comment - Any updates on this?

          Steve Todorov added a comment - - edited

          ++acejam try reverting to an older version of Docker Pipeline. I have been using the older version since this issue hit me in JENKINS-47805. It seems that you can fix this either by using a mvnw, downgrade to an older version of Docker Pipeline or wait for JENKINS-48050 which looks like it will fix it for good.

          Steve Todorov added a comment - - edited ++ acejam try reverting to an older version of Docker Pipeline. I have been using the older version since this issue hit me in JENKINS-47805 . It seems that you can fix this either by using a mvnw, downgrade to an older version of Docker Pipeline or wait for JENKINS-48050  which looks like it will fix it for good.

          Joshua Noble added a comment -

          I spent a good amount of time looking into this and came up with an acceptable workaround that works for my team. We have hundreds of branches across hundreds of repos, so updating all of the Java/Maven related Jenkinsfiles (on every branch) to use $MVN_CMD instead of "mvn" was not going to happen anytime soon. We also had several pending plugin updates, many of which required the latest version of the pipeline-maven plugin. In our case, we have a private internal Docker image that we use for all maven builds. It's forked off the official Docker Hub Maven image and includes a few additional tools and packages that we need.

          What I ended up doing was creating a bash script that simply wraps the "mvn" command and checks for existence of $MVN_CMD. If $MVN_CMD exists, it calls that and passes along the same parameters. If $MVN_CMD doesn't exist, it simply calls the real maven binary located at /usr/bin/mvn. This bash script lives at /usr/local/bin/mvn, which exists in our $PATH before the real maven binary, which is /usr/bin/mvn.

          Simply create a bash script like below, and ensure it has execute permissions: (chmod +x mvn.sh) 

          #!/bin/bash
          
          if [[ -v MVN_CMD ]]; then
            $MVN_CMD "$@"
          else
            /usr/bin/mvn "$@"
          fi
          

          Then inside of your Dockerfile, add the following instruction:

          COPY mvn.sh /usr/local/bin/mvn
          

          This has been tested with the maven:3.5.3-jdk-8 Docker image.

          Joshua Noble added a comment - I spent a good amount of time looking into this and came up with an acceptable workaround that works for my team. We have hundreds of branches across hundreds of repos, so updating all of the Java/Maven related Jenkinsfiles (on every branch) to use $MVN_CMD instead of "mvn" was not going to happen anytime soon. We also had several pending plugin updates, many of which required the latest version of the pipeline-maven plugin. In our case, we have a private internal Docker image that we use for all maven builds. It's forked off the official Docker Hub Maven image and includes a few additional tools and packages that we need. What I ended up doing was creating a bash script that simply wraps the "mvn" command and checks for existence of $MVN_CMD. If $MVN_CMD exists, it calls that and passes along the same parameters. If $MVN_CMD doesn't exist, it simply calls the real maven binary located at /usr/bin/mvn. This bash script lives at /usr/local/bin/mvn, which exists in our $PATH before the real maven binary, which is /usr/bin/mvn. Simply create a bash script like below, and ensure it has execute permissions: (chmod +x mvn.sh)   #!/bin/bash if [[ -v MVN_CMD ]]; then $MVN_CMD "$@" else /usr/bin/mvn "$@" fi Then inside of your Dockerfile, add the following instruction: COPY mvn.sh /usr/local/bin/mvn This has been tested with the maven:3.5.3-jdk-8 Docker image.

          Tim Downey added a comment -

          Thanks for the writeup acejam – That's pretty much exactly what joshtrow and I have done to deal with the problem.

          Tim Downey added a comment - Thanks for the writeup acejam – That's pretty much exactly what joshtrow and I have done to deal with the problem.

            Unassigned Unassigned
            testuser7 Test User
            Votes:
            13 Vote for this issue
            Watchers:
            25 Start watching this issue

              Created:
              Updated: