Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22492

java.io.IOException: Pipe broken

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Duplicate
    • unity3d-plugin
    • None
    • Windows Master. OSX Slave Node running Unity.
      unity3d-plugin: 0.6.

    Description

      When executing the build on the slave, sometimes (it seems to happen when the build duration is considerable, like building for the webplayer), the plugin fails with:

      ... [NORMAL BUILD OUTOUT] ...

      LZMA 4.43 Copyright (c) 1999-2006 Igor Pavlov 2006-06-04
      Failure on remote
      java.io.IOException: Pipe broken
      at java.io.PipedInputStream.read(Unknown Source)
      at java.io.PipedInputStream.read(Unknown Source)
      at java.io.InputStream.read(Unknown Source)
      at org.jenkinsci.plugins.unity3d.io.StreamCopyThread.run(StreamCopyThread.java:64)
      ... [Jenkins internal stack trace] ...

      It fails after the LZMA compression ends (the process lasts several minutes).

      Attachments

        Issue Links

          Activity

            lacostej lacostej added a comment -

            Do you have a way to reproduce this often ?
            Was this happening with v0.5 ?
            Are you using the -logFile option ? (supported properly from 0.6)

            lacostej lacostej added a comment - Do you have a way to reproduce this often ? Was this happening with v0.5 ? Are you using the -logFile option ? (supported properly from 0.6)
            juan_batovi Juan Fornos added a comment - - edited

            Hi lacostej,

            Thanks for your feedback. With that job, it fails every time I try to build for the webplayer. Didn't try with v0.5 (Jenkins doesn't let me downgrade) but I remember having this issue less frequently. I'm not using the -logFile option. The full stack trace is:

            LZMA 4.43 Copyright (c) 1999-2006 Igor Pavlov 2006-06-04
            Failure on remote
            java.io.IOException: Pipe broken
            at java.io.PipedInputStream.read(Unknown Source)
            at java.io.PipedInputStream.read(Unknown Source)
            at java.io.InputStream.read(Unknown Source)
            at org.jenkinsci.plugins.unity3d.io.StreamCopyThread.run(StreamCopyThread.java:64)
            FATAL: Unity3d command line execution failed with status 1
            Build step 'Invoke Unity3d Editor' marked build as failure
            FATAL: Remote call on MacServer failed
            java.io.IOException: Remote call on MacServer failed
            at hudson.remoting.Channel.call(Channel.java:731)
            at hudson.Launcher$RemoteLauncher.kill(Launcher.java:887)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:588)
            at hudson.model.Run.execute(Run.java:1670)
            at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:231)
            Caused by: java.lang.NoClassDefFoundError: Could not initialize class hudson.util.ProcessTree$UnixReflection
            at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:552)
            at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:572)
            at hudson.util.ProcessTree$Unix.killAll(ProcessTree.java:496)
            at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:899)
            at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:890)
            at hudson.remoting.UserRequest.perform(UserRequest.java:118)
            at hudson.remoting.UserRequest.perform(UserRequest.java:48)
            at hudson.remoting.Request$2.run(Request.java:328)
            at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:744)

            Let me know if I can be of any help.

            Thanks,
            Juan

            juan_batovi Juan Fornos added a comment - - edited Hi lacostej, Thanks for your feedback. With that job, it fails every time I try to build for the webplayer. Didn't try with v0.5 (Jenkins doesn't let me downgrade) but I remember having this issue less frequently. I'm not using the -logFile option. The full stack trace is: LZMA 4.43 Copyright (c) 1999-2006 Igor Pavlov 2006-06-04 Failure on remote java.io.IOException: Pipe broken at java.io.PipedInputStream.read(Unknown Source) at java.io.PipedInputStream.read(Unknown Source) at java.io.InputStream.read(Unknown Source) at org.jenkinsci.plugins.unity3d.io.StreamCopyThread.run(StreamCopyThread.java:64) FATAL: Unity3d command line execution failed with status 1 Build step 'Invoke Unity3d Editor' marked build as failure FATAL: Remote call on MacServer failed java.io.IOException: Remote call on MacServer failed at hudson.remoting.Channel.call(Channel.java:731) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:887) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:588) at hudson.model.Run.execute(Run.java:1670) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Caused by: java.lang.NoClassDefFoundError: Could not initialize class hudson.util.ProcessTree$UnixReflection at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:552) at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:572) at hudson.util.ProcessTree$Unix.killAll(ProcessTree.java:496) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:899) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:890) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) Let me know if I can be of any help. Thanks, Juan
            lacostej lacostej added a comment -

            Are you running JDK 1.8 on your slave ? see JENKINS-21341 (I don't think it is the cause, but wondering)

            Also look at the troubleshooting steps: https://wiki.jenkins-ci.org/display/JENKINS/Unity3dBuilder+Plugin#Unity3dBuilderPlugin-Troubleshootingfailures

            lacostej lacostej added a comment - Are you running JDK 1.8 on your slave ? see JENKINS-21341 (I don't think it is the cause, but wondering) Also look at the troubleshooting steps: https://wiki.jenkins-ci.org/display/JENKINS/Unity3dBuilder+Plugin#Unity3dBuilderPlugin-Troubleshootingfailures
            juan_batovi Juan Fornos added a comment - - edited

            I found the issue by examining the Editor.log file. Unity is failing when trying to access some SVN files previously copied to the Temp folder (I still didn't figured out why the Unity process doesn't have permissions to access those files). The workaround I found is deleting the Temp folder before the build.

            However, there is a lot of data (including the file permissions issue) after the LZMA line in the Editor.log and the plugin isn't displaying it in the Jenkins build log. That's why I thought the issue was related to long builds as the LZMA compresses takes several minutes.

            I guess you can reproduce the issue by denying some files permissions to the Unity process in the Temp folder.

            juan_batovi Juan Fornos added a comment - - edited I found the issue by examining the Editor.log file. Unity is failing when trying to access some SVN files previously copied to the Temp folder (I still didn't figured out why the Unity process doesn't have permissions to access those files). The workaround I found is deleting the Temp folder before the build. However, there is a lot of data (including the file permissions issue) after the LZMA line in the Editor.log and the plugin isn't displaying it in the Jenkins build log. That's why I thought the issue was related to long builds as the LZMA compresses takes several minutes. I guess you can reproduce the issue by denying some files permissions to the Unity process in the Temp folder.
            lacostej lacostej added a comment -

            Juan, WRT to the permissions, I just read in http://unity3d.com/unity/whats-new/unity-4.5.3:

            Editor: Spawned child processes will no longer inherit files opened by the Editor on Windows. Fixes issues with VCS or shader compiler processes holding locks to files.

            That was maybe the cause of your issues ?!

            As to reproducing your issue, I haven't yet looked at it. There's another Pipe Broken issue that I am looking into : JENKINS-23958

            lacostej lacostej added a comment - Juan, WRT to the permissions, I just read in http://unity3d.com/unity/whats-new/unity-4.5.3: Editor: Spawned child processes will no longer inherit files opened by the Editor on Windows. Fixes issues with VCS or shader compiler processes holding locks to files. That was maybe the cause of your issues ?! As to reproducing your issue, I haven't yet looked at it. There's another Pipe Broken issue that I am looking into : JENKINS-23958
            juan_batovi Juan Fornos added a comment -

            Sorry for the delay, I've been really busy at work. The issue was fixed in Unity 4.5.3 indeed.

            Anyway, IMHO the plugin should catch that kind of exceptions and display the remaining Unity log so the user can understand what was the real underlying problem.

            Greetings,
            Juan

            juan_batovi Juan Fornos added a comment - Sorry for the delay, I've been really busy at work. The issue was fixed in Unity 4.5.3 indeed. Anyway, IMHO the plugin should catch that kind of exceptions and display the remaining Unity log so the user can understand what was the real underlying problem. Greetings, Juan

            I`m still seeing this issue with Unity 4.5.4. It happens after the workspace was wiped and Unity reimports all assets, which takes up to 2 hours on a larger project.

            jspohr Johannes Spohr added a comment - I`m still seeing this issue with Unity 4.5.4. It happens after the workspace was wiped and Unity reimports all assets, which takes up to 2 hours on a larger project.
            lacostej lacostej added a comment -

            Johannes, is this 100% reproducible ? I don't yet know why the pipe breaks. Have you tried looking at specific errors in the Editor.log file? Are they displayed under the job console ?

            lacostej lacostej added a comment - Johannes, is this 100% reproducible ? I don't yet know why the pipe breaks. Have you tried looking at specific errors in the Editor.log file? Are they displayed under the job console ?

            Jenkins build log

            jspohr Johannes Spohr added a comment - Jenkins build log

            We do a clean build every night, and on average, the error happens on 1 out of 2 builds. I attached the build log, it ends with

            Hashing assets ... Failure on remote 
            java.io.IOException: Pipe broken
            	at java.io.PipedInputStream.read(PipedInputStream.java:322)
            	at java.io.PipedInputStream.read(PipedInputStream.java:378)
            	at java.io.InputStream.read(InputStream.java:101)
            	at org.jenkinsci.plugins.unity3d.io.StreamCopyThread.run(StreamCopyThread.java:64)
            

            In the Editor.log, it continues at this point:

            Hashing assets ... 110.089302 seconds
            ----- Compute hash(es) for 5009 asset(s).
            
            Non-Standard formatted meta data file, slow loading path 'Assets/_Textures/_Effects/Eff_MagneticField.png'
            Unable to parse YAML file: [Invalid leading UTF-8 octet] at line 0
            
             
            (Filename: C:/BuildAgent/work/d63dfc6385190b60/Runtime/Serialize/TransferFunctions/YAMLRead.cpp Line: 118)
            

            followed by a lot of lines with hashes, like this one:

            *** ee955d0a587174adf938293a0ac5040e replaces a9e85bc0884436a4996fd99cb5e885e2 at path assets/editor/addparent.cs 
            
            jspohr Johannes Spohr added a comment - We do a clean build every night, and on average, the error happens on 1 out of 2 builds. I attached the build log, it ends with Hashing assets ... Failure on remote java.io.IOException: Pipe broken at java.io.PipedInputStream.read(PipedInputStream.java:322) at java.io.PipedInputStream.read(PipedInputStream.java:378) at java.io.InputStream.read(InputStream.java:101) at org.jenkinsci.plugins.unity3d.io.StreamCopyThread.run(StreamCopyThread.java:64) In the Editor.log, it continues at this point: Hashing assets ... 110.089302 seconds ----- Compute hash(es) for 5009 asset(s). Non-Standard formatted meta data file, slow loading path 'Assets/_Textures/_Effects/Eff_MagneticField.png' Unable to parse YAML file: [Invalid leading UTF-8 octet] at line 0 (Filename: C:/BuildAgent/work/d63dfc6385190b60/Runtime/Serialize/TransferFunctions/YAMLRead.cpp Line: 118) followed by a lot of lines with hashes, like this one: *** ee955d0a587174adf938293a0ac5040e replaces a9e85bc0884436a4996fd99cb5e885e2 at path assets/editor/addparent.cs
            lacostej lacostej added a comment -

            Still having this in my head. Question: do you have regular click drifts on your server or slave ? Are you using NTP with a cron job or similar service to restore the clock ?

            lacostej lacostej added a comment - Still having this in my head. Question: do you have regular click drifts on your server or slave ? Are you using NTP with a cron job or similar service to restore the clock ?

            NTP is used on both server and slaves. Clock difference is about 6 seconds between server and Unity slaves.

            jspohr Johannes Spohr added a comment - NTP is used on both server and slaves. Clock difference is about 6 seconds between server and Unity slaves.
            lacostej lacostej added a comment -

            I think this is just a duplicate of JENKINS-23958

            lacostej lacostej added a comment - I think this is just a duplicate of JENKINS-23958
            lacostej lacostej added a comment - - edited

            I'll assume this is a duplicate of JENKINS-23958.

            I need more information from you guys. Please look at the information I added to JENKINS-23958 to help you create those debugging logs. I'll be happy to take a look anytime. This issue bothers me a lot Thanks!

            See https://issues.jenkins-ci.org/browse/JENKINS-23958?focusedCommentId=226618&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-226618

            lacostej lacostej added a comment - - edited I'll assume this is a duplicate of JENKINS-23958 . I need more information from you guys. Please look at the information I added to JENKINS-23958 to help you create those debugging logs. I'll be happy to take a look anytime. This issue bothers me a lot Thanks! See https://issues.jenkins-ci.org/browse/JENKINS-23958?focusedCommentId=226618&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-226618
            lacostej lacostej added a comment -

            I have a fix for JENKINS-23958. Will be in v0.8.

            lacostej lacostej added a comment - I have a fix for JENKINS-23958 . Will be in v0.8.
            lacostej lacostej added a comment -

            Fixed in v0.8

            lacostej lacostej added a comment - Fixed in v0.8

            People

              lacostej lacostej
              juan_batovi Juan Fornos
              Votes:
              3 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: