-
Bug
-
Resolution: Unresolved
-
Critical
-
Jenkins 1.504 - RHEL 6 - JDK 1.6 Hotspot
9 slaves node (1 AIX, 5 RHEL, 3 Windows)
I've configured automatic tools installation (JDK, Maven, Git, ...) in Jenkins with "Extract .zip/.tar" feature pointing to a Nexus URL (like this : "http://nexus:8081/nexus/content/repositories/jenkins-ci-tools/fr/mipih/ic/tools/jdk/oracle-hotspot/1.6.0_25_64bits/oracle-hotspot-1.6.0_25_64bits-linux.zip").
An example of JDK tools declaration in config.xml is in attachment (config-JDK-installs.txt).
The problem is that Jenkins floods Nexus of HTTP requests to download some configured tools every minute.
This feature has been improved by https://issues.jenkins-ci.org/browse/JENKINS-16215 in the version 1.500 of Jenkins but Nexus still faces with a flood of requests returning 304 response.
I've activated access logs of Nexus (nexus-access-log.txt).
Each minute, between 26 and 28 seconds, Jenkins creates more than 130 HTTP requests trying to download some tools parametered in global configuration.
I've 13 tools configured in global configuration.
96 HTTP requests only for oracle-hotspot-1.6.0_25_64bits-linux.zip in 3 seconds.
Each time, the repartition of HTTP requests is the same :
96 - oracle-hotspot-1.6.0_25_64bits-linux.zip
22 - maven-2.2.1.zip
5 - maven-3.0.3.zip
3 - ibm-j9-1.5.0_sr8a_32bits.zip
3 - git-1.7.5.1-aix.zip
1 - maven-3.0.4.zip
1 - oracle-hotspot-1.5.0_22_64bits-linux.zip
1 - oracle-hotspot-1.6.0_26_32bits-windows.zip
I've created this ticket with Critical priority because Nexus is overloaded each day.
I've tried to look in Jenkins core but I cannot see why Jenkins is trying to download again and again these tools. Any idea ?
Jenkins is supposed to call the auto-installer feature only when a build which need a non installed tool is triggered right ? Are there any other cases ?
Thanks in advance!
- is related to
-
-
- Resolved
-
[JENKINS-17170] Jenkins auto install tools feature floods Nexus of HTTP request
Code changed in jenkins
User: Slawomir Jodynski
Path:
src/main/java/hudson/plugins/groovy/SystemGroovy.java
http://jenkins-ci.org/commit/groovy-plugin/612689f61c596a6065c833fa036b3a9c3d2ae689
Log:
fixes JENKINS-17171 - token macro expansion not working
Untested, but based on the code this is still unresolved in ~1.580.
Whenever something asks for the home directory of a tool on a node, all ToolLocationTranslator are queried for getToolHome(...). InstallerTranslator is one of them. It actually runs ToolInstaller.performInstallation(...) when doing so, which in the case of ZipExtractionInstaller calls FilePath.installIfNecessaryFrom(...) and that ultimately open a URLConnection and checks whether getResponseCode() == HttpURLConnection.HTTP_NOT_MODIFIED.
Weird.
https://github.com/jenkinsci/jenkins/blob/70f5968160b0a1cf81dc451fd2aacf1e0f9201ea/core/src/main/java/hudson/tools/DownloadFromUrlInstaller.java#L67 has check before. Both DownloadFromUrlInstaller and ZipExtractionInstaller do the same: "download and extract".
JDKInstaller also has such check https://github.com/jenkinsci/jenkins/blob/70f5968160b0a1cf81dc451fd2aacf1e0f9201ea/core/src/main/java/hudson/tools/JDKInstaller.java#L126
integer: AFAIU the difference is that DownloadFromURL installer is for publicly available distributions based on files with specific assigned version numbers, while ZipExtraction can use arbitrary user-provided paths, including e.g. "currentTools.zip", a file updated whenever there's a change. So the former can rather safely assume that something downloaded once doesn't need to be redownloaded, while the latter cannot.
Might be sufficient to cache any given URL's state after a check for ~10 seconds. One request per Jenkins instance and URL per 10 seconds should be reasonable.
Is it reproduced with a recent Jenkins version?