Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-1762

Archiving artifacts on M2 project *really* slow

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: other
    • Labels:
      None
    • Environment:
      Platform: All, OS: Linux
    • Similar Issues:

      Description

      I just upgraded from v211 to v219. When trying to build, the Archiving step
      takes an extremely long time now. Wheras it used to take a few seconds for a
      23M WAR file to be archived, it now takes about 2 hours+

      Here's the relevant log file message on which occurs right before the archiving:

      [HUDSON] Archiving /c01/hudson/data/jobs/Dealer
      Inventory/workspace/caps/caps-web/target/caps-web-1.0-SNAPSHOT.war to
      /c01/hudson/data/jobs/Dealer
      Inventory/modules/com.ca.caps$caps-web/builds/2008-05-29_15-40-30/archive/com.ca.caps/caps-web/1.0-SNAPSHOT/caps-web-1.0-SNAPSHOT.war

      Note that this is a native Maven project.

      Reverting back to v211 solved the problem.

        Attachments

          Activity

          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          Does this involve distributed builds? Or just on the master?

          What is the OS of machines involved in this?

          Show
          kohsuke Kohsuke Kawaguchi added a comment - Does this involve distributed builds? Or just on the master? What is the OS of machines involved in this?
          Hide
          nhajratw Nayan Hajratwala added a comment -

          I only have a master configuration .. no slaves. My server is on Redhat Linux.

          Show
          nhajratw Nayan Hajratwala added a comment - I only have a master configuration .. no slaves. My server is on Redhat Linux.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/www/changelog.html
          http://fisheye4.cenqua.com/changelog/hudson/?cs=9760
          Log:
          [JENKINS-1762 JENKINS-1772] noting the improvement in buffering. Hope this fixes them.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/www/changelog.html http://fisheye4.cenqua.com/changelog/hudson/?cs=9760 Log: [JENKINS-1762 JENKINS-1772] noting the improvement in buffering. Hope this fixes them.
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          I believe I fixed this issue. Can you try the snapshot build from
          https://hudson.dev.java.net/servlets/ProjectDocumentList?folderID=4939&expandFolder=4939&folderID=2761
          and see if that fixes the problem?

          Show
          kohsuke Kohsuke Kawaguchi added a comment - I believe I fixed this issue. Can you try the snapshot build from https://hudson.dev.java.net/servlets/ProjectDocumentList?folderID=4939&expandFolder=4939&folderID=2761 and see if that fixes the problem?
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          Copied from a comment to issue #1772:
          ----------------
          Is there any way to turn off the artifacts copying? It does seem to spend a
          very long time copying the artifacts

          java.net.SocketOutputStream.socketWrite0(Native Method)
          java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
          java.net.SocketOutputStream.write(SocketOutputStream.java:115)
          java.io.FilterOutputStream.write(FilterOutputStream.java:60)
          java.io.FilterOutputStream.write(FilterOutputStream.java:108)
          java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1685)
          java.io.ObjectOutputStream$BlockDataOutputStream.write(ObjectOutputStream.java:1656)
          java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1218)
          java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1075)
          java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1375)
          java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1347)
          java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
          java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079)
          java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:302)
          hudson.remoting.Channel.send(Channel.java:323)
          hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:87)
          hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:71)
          hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:85)
          hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:85)
          hudson.Util.copyStream(Util.java:315)
          hudson.FilePath$21.invoke(FilePath.java:741)
          hudson.FilePath$21.invoke(FilePath.java:738)
          hudson.FilePath.act(FilePath.java:317)
          hudson.FilePath.copyTo(FilePath.java:738)
          hudson.FilePath.copyTo(FilePath.java:726)
          hudson.maven.reporters.MavenArtifact.archive(MavenArtifact.java:161)
          hudson.maven.reporters.MavenArtifactArchiver.postBuild(MavenArtifactArchiver.java:55)
          hudson.maven.MavenModuleSetBuild$Builder.postModule(MavenModuleSetBuild.java:555)
          hudson.maven.MavenBuilder$Adapter.fireLeaveModule(MavenBuilder.java:261)
          hudson.maven.MavenBuilder$Adapter.endModule(MavenBuilder.java:232)
          org.apache.maven.lifecycle.LifecycleExecutorInterceptor$EventMonitorImpl.endEvent(LifecycleExecutorInterceptor.java:69)
          ----------------
          FilterOutputStream is used to copy things one byte at a time. I now just need to
          find out who's using FilterOutputStream or its derived types.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - Copied from a comment to issue #1772: ---------------- Is there any way to turn off the artifacts copying? It does seem to spend a very long time copying the artifacts java.net.SocketOutputStream.socketWrite0(Native Method) java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) java.net.SocketOutputStream.write(SocketOutputStream.java:115) java.io.FilterOutputStream.write(FilterOutputStream.java:60) java.io.FilterOutputStream.write(FilterOutputStream.java:108) java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1685) java.io.ObjectOutputStream$BlockDataOutputStream.write(ObjectOutputStream.java:1656) java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1218) java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1075) java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1375) java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1347) java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290) java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079) java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:302) hudson.remoting.Channel.send(Channel.java:323) hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:87) hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:71) hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:85) hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:85) hudson.Util.copyStream(Util.java:315) hudson.FilePath$21.invoke(FilePath.java:741) hudson.FilePath$21.invoke(FilePath.java:738) hudson.FilePath.act(FilePath.java:317) hudson.FilePath.copyTo(FilePath.java:738) hudson.FilePath.copyTo(FilePath.java:726) hudson.maven.reporters.MavenArtifact.archive(MavenArtifact.java:161) hudson.maven.reporters.MavenArtifactArchiver.postBuild(MavenArtifactArchiver.java:55) hudson.maven.MavenModuleSetBuild$Builder.postModule(MavenModuleSetBuild.java:555) hudson.maven.MavenBuilder$Adapter.fireLeaveModule(MavenBuilder.java:261) hudson.maven.MavenBuilder$Adapter.endModule(MavenBuilder.java:232) org.apache.maven.lifecycle.LifecycleExecutorInterceptor$EventMonitorImpl.endEvent(LifecycleExecutorInterceptor.java:69) ---------------- FilterOutputStream is used to copy things one byte at a time. I now just need to find out who's using FilterOutputStream or its derived types.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/jnlp-agent/src/main/java/hudson/jnlp/Engine.java
          http://fisheye4.cenqua.com/changelog/hudson/?cs=9786
          Log:
          this is not by itself a fix to JENKINS-1762 but this should improve the performane of JNLP slaves

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/jnlp-agent/src/main/java/hudson/jnlp/Engine.java http://fisheye4.cenqua.com/changelog/hudson/?cs=9786 Log: this is not by itself a fix to JENKINS-1762 but this should improve the performane of JNLP slaves
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/maven-agent/src/main/java/hudson/maven/agent/Main.java
          trunk/hudson/main/maven-agent/src/main/java/hudson/maven/agent/RealFilterOutputStream.java
          http://fisheye4.cenqua.com/changelog/hudson/?cs=9787
          Log:
          [FIXED JENKINS-1762] this should really fix #1762.
          FilterOutputStream is evil.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/maven-agent/src/main/java/hudson/maven/agent/Main.java trunk/hudson/main/maven-agent/src/main/java/hudson/maven/agent/RealFilterOutputStream.java http://fisheye4.cenqua.com/changelog/hudson/?cs=9787 Log: [FIXED JENKINS-1762] this should really fix #1762. FilterOutputStream is evil.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            nhajratw Nayan Hajratwala
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: