Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53012

Unable to configure grapeConfig or Ivy config for declarative pipeline

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • Jenkins ver. 2.107.2, Pipeline 2.5, Pipeline Groovy 2.4.9, Windows 10 x64

      I am attempting to execute some code (an API client wrapper) in a declarative pipeline, within a script block which relies on a jar library in our private artifact repository (Artifactory).  I am able to use @Grab to pull libs from the public maven repository, but cannot get Jenkins to pick up a custom config containing my Artifactory instance in order to resolve our private library set.  I've tried several combinations of grapeConfig.xml and ivysettings.xml in different locations, but cannot seem to override the default config that is picked up via the plugin jar files.

      If this is supported, it's documented very poorly.  If it's not supported, I'd like to request support be added given this should be a typical use case.  If this cannot be done directly in the pipeline is it possible via the shared library functionality?  Documentation there also seems very poor on the subject - all examples are trivial and make no mention of importing 3rd party dependencies or configuring dependency resolution.

      I've attempted to add the following JVM system properties, without much luck:

      -Dgroovy.grape.report.downloads=true
      -Divy.message.logger.level=4
      -Dgrape.config=%USERPROFILE%\.groovy\grapeConfig.xml
      -Dgrape.root=%USERPROFILE%\.groovy
      -Divy.default.settings.dir=%USERPROFILE%\.ivy2
      -Divy.default.conf.dir=%USERPROFILE%\.ivy2

       

      I can tell from the log files that it's not picking up my custom grape/ivy config and that it's not even attempting to resolve from my Artifactory installation.

          [JENKINS-53012] Unable to configure grapeConfig or Ivy config for declarative pipeline

          Mads Nielsen added a comment -

          I'm having similar issues. I work at a corporation where access to the internet is closed down, so i need to remove the default lookup to use our own private mirrors. 

          For us, this results in the jobs timing out failing to download artifacts.

          Mads Nielsen added a comment - I'm having similar issues. I work at a corporation where access to the internet is closed down, so i need to remove the default lookup to use our own private mirrors.  For us, this results in the jobs timing out failing to download artifacts.

          Devops Kanjer added a comment -

          The settings seemingly need to be added in the Jenkins main installation. We have put the grapeConfig.xml in the <jenkingshome>/.groovy directory and its being picked up.

          Devops Kanjer added a comment - The settings seemingly need to be added in the Jenkins main installation. We have put the grapeConfig.xml in the <jenkingshome>/.groovy directory and its being picked up.

          Carlos Rodríguez López added a comment - - edited

          Alternatively, you can integrate private repos by using the `@GrabResolver` annotation see http://docs.groovy-lang.org/latest/html/documentation/grape.html#Grape-SpecifyAdditionalRepositories

          Carlos Rodríguez López added a comment - - edited Alternatively, you can integrate private repos by using the `@GrabResolver` annotation see http://docs.groovy-lang.org/latest/html/documentation/grape.html#Grape-SpecifyAdditionalRepositories

          Note: Jenkins Project does not longer recommend to use @grab. See

          While possible, accessing third-party libraries using @Grab from trusted libraries has various issues and is not recommended. Instead of using @Grab, the recommended approach is to create a standalone executable in the programming language of your choice (using whatever third-party libraries you desire), install it on the Jenkins agents that your Pipelines use, and then invoke that executable in your Pipelines using the bat or sh step.
          

          Carlos Rodríguez López added a comment - Note: Jenkins Project does not longer recommend to use @grab. See https://www.jenkins.io/doc/book/pipeline/shared-libraries/#using-third-party-libraries https://github.com/jenkins-infra/jenkins.io/pull/3535 While possible, accessing third-party libraries using @Grab from trusted libraries has various issues and is not recommended. Instead of using @Grab, the recommended approach is to create a standalone executable in the programming language of your choice (using whatever third-party libraries you desire), install it on the Jenkins agents that your Pipelines use, and then invoke that executable in your Pipelines using the bat or sh step.

          Carlos Rodríguez López added a comment - - edited

          Reproduction steps

          By following https://www.jenkins.io/doc/book/pipeline/shared-libraries/#using-third-party-libraries

          1. Place Jenkins behind a proxy and blacklist the access to https://mvnrepository.com/
          2. Add the following Properties to the instance (I used squid proxy for the testing)

          http.proxyHost=squid-proxy.example.svc.cluster.local
          http.proxyPort=xxxx
          https.proxyHost=squid-proxy.example.svc.cluster.local
          https.proxyPort=xxxx
          ivy.settings.file=/var/jenkins_home/.groovy/grapeConfig.xml
          

          3. Upload https://mvnrepository.com/artifact/org.apache.commons/commons-math3/3.4.1 to your artifactory server repository (e.g. http://artifactory-oss.example.com/artifactory/my-repo-libs-release-local/)
          4.- Create a Shared library Global Variable as follows

          @Grab('org.apache.commons:commons-math3:3.4.1')
          import org.apache.commons.math3.primes.Primes
          
          def isPrime(int count) {
            if (!Primes.isPrime(count)) {
              error "${count} was not prime"
            } else {
              echo "${count} is a prime"
            }
          }
          

          5.- Then a Jenkinsfile

          library identifier: 'my-shared-libraries@master', retriever: modernSCM(
            [$class: 'GitSCMSource',
             remote: 'https://github.com/my_org/my-jenkins-demos.git'])
          
          thirdPartyLib.isPrime 2
          

          6.- Place the /var/jenkins_home/.groovy/grapeConfig.xml

          <?xml version="1.0" encoding="UTF-8"?>
          <ivy-settings
            <settings defaultResolver="downloadGrapes" />
            <resolvers>
              <chain name="downloadGrapes">
                <ibiblio name="public" m2compatible="true" root="http://artifactory-oss.example.com/artifactory/my-repo-libs-release-local/" />
              </chain>
            </resolvers>
          </ivy-settings>
          

          7.- Run the build

          ...
          org.jenkinsci.plugins.workflow.cps.CpsCompilationErrorsException: startup failed:
          General error during conversion: Error grabbing Grapes -- [unresolved dependency: org.apache.commons#commons-math3;3.4.1: not found]
          
          java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: org.apache.commons#commons-math3;3.4.1: not found]
          	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
          	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
          	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
          	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
          ...
          

          Workaround

          Use GrapeResolver

          @GrabResolver(name='artifactory', root='http://artifactory-oss.example.com/artifactory/my-repo-libs-release-local/', m2Compatible=true)
          @Grab('org.apache.commons:commons-math3:3.4.1')
          import org.apache.commons.math3.primes.Primes
          
          def isPrime(int count) {
            if (!Primes.isPrime(count)) {
              error "${count} was not prime"
            } else {
              echo "${count} is a prime"
            }
          }
          

          Carlos Rodríguez López added a comment - - edited Reproduction steps By following https://www.jenkins.io/doc/book/pipeline/shared-libraries/#using-third-party-libraries 1. Place Jenkins behind a proxy and blacklist the access to https://mvnrepository.com/ 2. Add the following Properties to the instance (I used squid proxy for the testing) http.proxyHost=squid-proxy.example.svc.cluster.local http.proxyPort=xxxx https.proxyHost=squid-proxy.example.svc.cluster.local https.proxyPort=xxxx ivy.settings.file=/ var /jenkins_home/.groovy/grapeConfig.xml 3. Upload https://mvnrepository.com/artifact/org.apache.commons/commons-math3/3.4.1 to your artifactory server repository (e.g. http://artifactory-oss.example.com/artifactory/my-repo-libs-release-local/ ) 4.- Create a Shared library Global Variable as follows @Grab( 'org.apache.commons:commons-math3:3.4.1' ) import org.apache.commons.math3.primes.Primes def isPrime( int count) { if (!Primes.isPrime(count)) { error "${count} was not prime" } else { echo "${count} is a prime" } } 5.- Then a Jenkinsfile library identifier: 'my-shared-libraries@master' , retriever: modernSCM( [$class: 'GitSCMSource' , remote: 'https: //github.com/my_org/my-jenkins-demos.git' ]) thirdPartyLib.isPrime 2 6.- Place the /var/jenkins_home/.groovy/grapeConfig.xml <?xml version= "1.0" encoding= "UTF-8" ?> <ivy-settings <settings defaultResolver= "downloadGrapes" /> <resolvers> <chain name= "downloadGrapes" > <ibiblio name= " public " m2compatible= " true " root= "http: //artifactory-oss.example.com/artifactory/my-repo-libs-release-local/" /> </chain> </resolvers> </ivy-settings> 7.- Run the build ... org.jenkinsci.plugins.workflow.cps.CpsCompilationErrorsException: startup failed: General error during conversion: Error grabbing Grapes -- [unresolved dependency: org.apache.commons#commons-math3;3.4.1: not found] java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: org.apache.commons#commons-math3;3.4.1: not found] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ... Workaround Use GrapeResolver @GrabResolver(name= 'artifactory' , root= 'http: //artifactory-oss.example.com/artifactory/my-repo-libs-release-local/' , m2Compatible= true ) @Grab( 'org.apache.commons:commons-math3:3.4.1' ) import org.apache.commons.math3.primes.Primes def isPrime( int count) { if (!Primes.isPrime(count)) { error "${count} was not prime" } else { echo "${count} is a prime" } }

            Unassigned Unassigned
            mrokitka Mike Rokitka
            Votes:
            6 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: