Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22961

JClouds Plugin: Server slave get deleted if they aren't build before 10 minutes

      I got a custom image for a server in Rackspace, it takes more than 10 minutes (600 seconds) to build because it's compression (I guess), so JClouds Plugin deletes it after 10 minutes and tries to build another server, making that the job never completes properly (it just sits on the queue), here is what I get from the Jenkins' logs

      may 09, 2014 11:43:40 PM Grave org.jclouds.logging.jdk.JDKLogger logError
      << problem customizing node(DFW/4acc0220-af9e-4e23-bd16-5cafcc669ed8):
      java.lang.IllegalStateException: node(DFW/4acc0220-af9e-4e23-bd16-5cafcc669ed8) didn't achieve the status running; aborting after 600 seconds with final status: PENDING
      at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:72)
      at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:45)
      at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:121)
      at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.apply(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:146)
      at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.apply(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:53)
      at shaded.com.google.common.util.concurrent.Futures$1.apply(Futures.java:711)
      at shaded.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:849)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:744)

      (please see the attached file for details)

      I suggest adding a button in the configuration that starts building a server with the objective of making an approximate timing of how much time it takes to build a server slave before using it for the image being added, to avoid hard coding the timeout times and making it more flexible.

          [JENKINS-22961] JClouds Plugin: Server slave get deleted if they aren't build before 10 minutes

          Jorge Araya Navarro created issue -
          Jorge Araya Navarro made changes -
          Attachment New: jenkins-jclouds-error.log [ 25787 ]
          Fix Version/s New: current [ 10162 ]
          Description Original: I configured jclouds as it says on its wiki. I then try these latest steps:

          * Goto Jenkins' home page, click on `Build Executor Status` link on the sidebar.

          * Verify that you have a button with `Provision via JClouds - (YOUR PROFILE NAME) drop down with the slave template name you configured.

          * Click on the slave and see if your slave launched succesfully (please wait until the operation completes).

          Then I waited, and waited, and waited, and the browser was saying "loading...", I then pressed F5 to refresh the page, the "loading..." message of the web browser was gone. I then went into my Rackspace control panel and indeed a new server was being created by jclouds with a custom Windows image I built (and tested before doing this test with jclouds). like 27 or 30 minutes later the building process of the server was at 80% (according to the last update) and, I don't know the exact moment, but then the server was marked as "deleted". I then start reading the jenkins' logs and I found this:

          may 09, 2014 11:43:40 PM Grave org.jclouds.logging.jdk.JDKLogger logError
          << problem customizing node(DFW/4acc0220-af9e-4e23-bd16-5cafcc669ed8):
          java.lang.IllegalStateException: node(DFW/4acc0220-af9e-4e23-bd16-5cafcc669ed8) didn't achieve the status running; aborting after 600 seconds with final status: PENDING
          at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:72)
          at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:45)
          at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:121)
          at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.apply(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:146)
          at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.apply(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:53)
          at shaded.com.google.common.util.concurrent.Futures$1.apply(Futures.java:711)
          at shaded.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:849)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:744)

          (please see the attached file for details)
          New: I got a custom image for a server in Rackspace, it takes more than 10 minutes (600 seconds) to build because it's compression (I guess), so JClouds Plugin deletes it after 10 minutes and tries to build another server, making that the job never completes properly (it just sits on the queue), here is what I get from the Jenkins' logs

          may 09, 2014 11:43:40 PM Grave org.jclouds.logging.jdk.JDKLogger logError
          << problem customizing node(DFW/4acc0220-af9e-4e23-bd16-5cafcc669ed8):
          java.lang.IllegalStateException: node(DFW/4acc0220-af9e-4e23-bd16-5cafcc669ed8) didn't achieve the status running; aborting after 600 seconds with final status: PENDING
          at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:72)
          at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:45)
          at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:121)
          at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.apply(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:146)
          at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.apply(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:53)
          at shaded.com.google.common.util.concurrent.Futures$1.apply(Futures.java:711)
          at shaded.com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:849)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:744)

          (please see the attached file for details)

          I suggest adding a button in the configuration that starts building a server with the objective of making an approximate timing of how much time it takes to build a server slave before using it for the image being added, to avoid hard coding the timeout times and making it more flexible.
          Priority Original: Critical [ 2 ] New: Blocker [ 1 ]
          Summary Original: Rackspace Cloud Server gets deleted by the plugin if it does not build before 10 minutes New: Server slave get deleted if they aren't build before 10 minutes
          Jorge Araya Navarro made changes -
          Component/s New: jclouds [ 16529 ]
          Summary Original: Server slave get deleted if they aren't build before 10 minutes New: JClouds Plugin: Server slave get deleted if they aren't build before 10 minutes

          Daniel Beck added a comment -

          User configuration issue: Timeout is configurable as Instance Startup Timeout.

          Daniel Beck added a comment - User configuration issue: Timeout is configurable as Instance Startup Timeout .
          Daniel Beck made changes -
          Resolution New: Not A Defect [ 7 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]
          Jenkins IRC Bot made changes -
          Component/s Original: jclouds-jenkins [ 17483 ]
          R. Tyler Croy made changes -
          Workflow Original: JNJira [ 155114 ] New: JNJira + In-Review [ 195139 ]
          Fritz Elfert made changes -
          Status Original: Resolved [ 5 ] New: Closed [ 6 ]

            Unassigned Unassigned
            shackra Jorge Araya Navarro
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: