Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-2963

crawler fails to fetch Maven versions

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      It all began with me trying to install Maven 3.8.1 via the Global Tool UI in Jenkins. That version came out a couple of weeks ago, so it should be available. After some digging I found that the crawler that fetches the information fails its runs, so no new (or old) versions are fetched and the file hudson.tasks.Maven.MavenInstaller is not updated.

      There is an automated job that runs all the crawlers at:
      https://ci.jenkins.io/job/Infra/job/crawler/job/master/42/consoleFull
      and it fails when it tries to execute the maven.groovy script. See the relevant part of the log below. Unfortunately that does not fail the build itself, so it propably goes under the radar.

      It looks like a bug in the maven crawler groovy script, but I don't know how to fix it.

      Since Maven 3.8.1 contains security fixes it would be great if it could be made available soon to Jenkins users for easy installation.

      20:39:41  + groovy -Dgrape.config=./grapeConfig.xml ./lib/runner.groovy maven.groovy
      20:39:43  loading dependencies...done
      20:39:49  Caught: org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'com.gargoylesoftware.htmlunit.UnexpectedPage@7915bca3' with class 'com.gargoylesoftware.htmlunit.UnexpectedPage' to class 'com.gargoylesoftware.htmlunit.html.HtmlPage'
      20:39:49  org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'com.gargoylesoftware.htmlunit.UnexpectedPage@7915bca3' with class 'com.gargoylesoftware.htmlunit.UnexpectedPage' to class 'com.gargoylesoftware.htmlunit.html.HtmlPage'
      20:39:49  	at maven$_listFromUrl_closure3.doCall(maven.groovy:52)
      20:39:49  	at maven.listFromUrl(maven.groovy:46)
      20:39:49  	at maven$listFromUrl.callCurrent(Unknown Source)
      20:39:49  	at maven.listFromNewUrl(maven.groovy:75)
      20:39:49  	at maven.listAll(maven.groovy:79)
      20:39:49  	at maven.run(maven.groovy:105)
      20:39:49  	at runner$_run_closure1.doCall(runner.groovy:13)
      20:39:49  	at runner.run(runner.groovy:10)
      20:39:49  + true
      

        Attachments

          Activity

          Hide
          dennisl Dennis Lundberg added a comment -

          There is no components for crawler in JIRA, perhaps one should be added?

          Show
          dennisl Dennis Lundberg added a comment - There is no components for crawler in JIRA, perhaps one should be added?
          Hide
          dennisl Dennis Lundberg added a comment -

          I downloaded the code and was able to figure out what was going wrong. There are message digest files for the Maven metadata files in Maven central. Due to increased security these now include .sha256 and .sha512 files, and these files match the pattern being used to find versions. The crawler fails to read these files as html-pages, so we need to ignore them.

          There is a pull request available for this issue at:
          https://github.com/jenkins-infra/crawler/pull/100

          Show
          dennisl Dennis Lundberg added a comment - I downloaded the code and was able to figure out what was going wrong. There are message digest files for the Maven metadata files in Maven central. Due to increased security these now include .sha256 and .sha512 files, and these files match the pattern being used to find versions. The crawler fails to read these files as html-pages, so we need to ignore them. There is a pull request available for this issue at: https://github.com/jenkins-infra/crawler/pull/100
          Hide
          markewaite Mark Waite added a comment -

          Confirmed resolved by the pull request. Maven 3.8.1 installs as a tool for me.

          Show
          markewaite Mark Waite added a comment - Confirmed resolved by the pull request. Maven 3.8.1 installs as a tool for me.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            dennisl Dennis Lundberg
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: