Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58853

nexusPublisher hangs when uploading multiple large files to Nexus 3

XMLWordPrintable

      Looks like something might be wrong with nexusPublisher when running on a build agent. The upload gets stuck if I try to upload multiple artifacts over a few MB in size to a Nexus 3 repo, and only if the build is aborted it throws an IOException (see the build log with the stacktrace below).

      How to reproduce:

      • it must run on a jenkins agent, in my setup the agent is connected via ssh and runs jdk 1.8u181 (using an agent is important - if you run it on master it works fine)
      • OS does not seem to matter (it fails both linux and windows agents)
      • you must upload 2 files at the same time in one package to a maven2 hosted repository (on a Nexus 3 repo)
      • file size seems to matter, for the small files (20KB) it always works, for the large ones it fails (2 files about 3.8 MB each)
      • there are no errors on the Nexus 3 logs
      • i was also able to reproduce it by setting up everything with 3 containers: nexus3, jenkins and one agent container based on ubuntu:18.04

      Sample Pipeline

       

      node('slave') {
// 2 large size (around 3.8 MB each) -- gets stuck
 nexusPublisher nexusInstanceId: 'nex3', nexusRepositoryId: 'releases', packages: [[$class: 'MavenPackage', mavenAssetList: [[classifier: 'release', extension: 'txt', filePath: '/tmp/jenkinsWS/file1.txt'], [classifier: 'debug', extension: 'txt', filePath: '/tmp/jenkinsWS/file2.txt']], mavenCoordinate: [artifactId: 'artifact1', groupId: 'testGroup', packaging: 'txt', version: "1.0+${BUILD_NUMBER}"]]]

 // 2 small size (aroud 22 KB each) --- works fine 
 // nexusPublisher nexusInstanceId: 'nex3', nexusRepositoryId: 'releases', packages: [[$class: 'MavenPackage', mavenAssetList: [[classifier: 'release', extension: 'txt', filePath: '/tmp/jenkinsWS/file1small.txt'], [classifier: 'debug', extension: 'txt', filePath: '/tmp/jenkinsWS/file2small.txt']], mavenCoordinate: [artifactId: 'artifact1', groupId: 'testGroup', packaging: 'txt', version: "1.0+${BUILD_NUMBER}"]]]

 // 1 large size (10+ MB) -- works fine
 // nexusPublisher nexusInstanceId: 'nex3', nexusRepositoryId: 'releases', packages: [[$class: 'MavenPackage', mavenAssetList: [[classifier: 'release', extension: 'txt', filePath: '/tmp/jenkinsWS/file3.txt']], mavenCoordinate: [artifactId: 'artifact1', groupId: 'testGroup', packaging: 'txt', version: "1.0+${BUILD_NUMBER}"]]]
}

       

      Files to be uploaded (on the build agent filesystem)

      root@388cdf249duq3:/tmp/jenkinsWS# ls -lah
total 20M
drwxr-xr-x 1 root root 4.0K Aug 7 21:33 .
drwxrwxrwt 1 root root 4.0K Aug 7 20:43 ..
-rw-r--r-- 1 root root 3.7M Aug 7 20:48 file1.txt
-rw-r--r-- 1 root root 18K Aug 7 20:55 file1small.txt
-rw-r--r-- 1 root root 3.8M Aug 7 20:48 file2.txt
-rw-r--r-- 1 root root 22K Aug 7 20:56 file2small.txt
-rw-r--r-- 1 root root 11M Aug 7 21:33 file3.txt

      Build log

      Started by user ... 
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on slave1 in /tmp/jenkinsWS/workspace/testUploadNexus3
[Pipeline] {
[Pipeline] nexusPublisher
Uploading Maven asset with groupId: testGroup artifactId: artifact1 version: 1.0+25 packaging: txt To repository: releases
Aborted by ...
Upload of maven component with GAV [testGroup:artifact1:1.0+25] failed
Failing build due to failure to upload file to Nexus Repository Manager Publisher
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.InterruptedException
 at java.lang.Object.wait(Native Method)
 at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:175)
Caused: java.io.IOException
 at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:177)
 at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:151)
 at org.apache.http.entity.mime.content.InputStreamBody.writeTo(SourceFile:92)
 at org.apache.http.entity.mime.AbstractMultipartForm.doWriteTo(SourceFile:134)
 at org.apache.http.entity.mime.AbstractMultipartForm.writeTo(SourceFile:157)
 at org.apache.http.entity.mime.MultipartFormEntity.writeTo(SourceFile:113)
 at org.apache.http.impl.execchain.RequestEntityProxy.writeTo(SourceFile:121)
 at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(SourceFile:156)
 at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(SourceFile:160)
 at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(SourceFile:238)
 at org.apache.http.protocol.HttpRequestExecutor.execute(SourceFile:123)
 at org.apache.http.impl.execchain.MainClientExec.execute(SourceFile:272)
 at org.apache.http.impl.execchain.ProtocolExec.execute(SourceFile:185)
 at org.apache.http.impl.execchain.RetryExec.execute(SourceFile:89)
Caused: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity
 at org.apache.http.impl.execchain.RetryExec.execute(SourceFile:108)
 at org.apache.http.impl.execchain.RedirectExec.execute(SourceFile:111)
 at org.apache.http.impl.client.InternalHttpClient.doExecute(SourceFile:185)
Caused: org.apache.http.client.ClientProtocolException
 at org.apache.http.impl.client.InternalHttpClient.doExecute(SourceFile:187)
 at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:72)
 at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:221)
 at org.apache.http.impl.client.CloseableHttpClient.execute(SourceFile:165)
 at com.sonatype.nexus.api.zz.ev.a(SourceFile:84)
Caused: com.sonatype.nexus.api.exception.RepositoryManagerException: Upload component was unsuccessful
 at com.sonatype.nexus.api.zz.ev.a(SourceFile:93)
 at com.sonatype.nexus.api.zz.ev.a(SourceFile:65)
 at com.sonatype.nexus.api.zz.eu.upload(SourceFile:157)
 at com.sonatype.nexus.api.repository.v3.RepositoryManagerV3Client$upload.call(Unknown Source)
 at org.sonatype.nexus.ci.nxrm.v3.ComponentUploaderNxrm3$_upload_closure2.doCall(ComponentUploaderNxrm3.groovy:83)
Caused: java.io.IOException
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
 at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
 at org.sonatype.nexus.ci.nxrm.v3.ComponentUploaderNxrm3$_upload_closure2.doCall(ComponentUploaderNxrm3.groovy:86)
Caused: java.io.IOException: Upload of maven component with GAV [testGroup:artifact1:1.0+25] failed
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
 at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:255)
 at org.sonatype.nexus.ci.nxrm.v3.ComponentUploaderNxrm3$_upload_closure2.doCall(ComponentUploaderNxrm3.groovy:96)
 at sun.reflect.GeneratedMethodAccessor401.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
 at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
 at groovy.lang.Closure.call(Closure.java:414)
 at org.codehaus.groovy.runtime.DefaultGroovyMethods.callClosureForMapEntry(DefaultGroovyMethods.java:5276)
 at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2117)
 at org.codehaus.groovy.runtime.dgm$164.invoke(Unknown Source)
 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
 at org.sonatype.nexus.ci.nxrm.v3.ComponentUploaderNxrm3.upload(ComponentUploaderNxrm3.groovy:55)
 at org.sonatype.nexus.ci.nxrm.v3.ComponentUploaderNxrm3$upload.callCurrent(Unknown Source)
 at org.sonatype.nexus.ci.nxrm.ComponentUploader.uploadComponents(ComponentUploader.groovy:84)
 at org.sonatype.nexus.ci.nxrm.ComponentUploader$uploadComponents.call(Unknown Source)
 at org.sonatype.nexus.ci.nxrm.PackagePublisherExecution.run(PackagePublisherExecution.groovy:44)
 at org.sonatype.nexus.ci.nxrm.PackagePublisherExecution.run(PackagePublisherExecution.groovy)
 at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
 at hudson.security.ACL.impersonate(ACL.java:290)
 at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
Finished: ABORTED

       

            whyjustin Justin Young
            pbc Pbc pbc
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: