Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-8606

ThinBackup only backs one job out of 10, and continually runs in the background

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Major
    • Resolution: Fixed
    • thinbackup-plugin
    • None
    • Linux, OpenSUSE 11.2, Hudson running inside JBOSS 5.1, 2GB ram, Dual core processor.

    Description

      When thinBackup runs it only performs the backup on one job (under the jobs folder). It seems like it is continually running throughout the day...when I have the backup scheduled at 2:00AM.
      From the log, it runs all day and every minute or so it shows this in the log:
      2011-01-24 09:22:14,093 INFO [org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork] (ThinBackup Worker Thread thread) Started ThinBackup Worker Thread
      2011-01-24 09:22:14,094 INFO [org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork] (ThinBackup Worker Thread thread) Finished ThinBackup Worker Thread. 1 ms

      There is also this exception in the log, but I am having a hard time figuring out if it is related:

      2011-01-24 09:22:44,120 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/hudson].[Stapler]] (http-10.7.0.113-8080-3) Servlet.service() for servlet Stapler threw exception
      java.lang.IllegalStateException: getWriter() has already been called for this response
      at org.apache.catalina.connector.Response.getOutputStream(Response.java:588)
      at org.apache.catalina.connector.ResponseFacade.getOutputStream(ResponseFacade.java:183)
      at javax.servlet.ServletResponseWrapper.getOutputStream(ServletResponseWrapper.java:102)
      at javax.servlet.ServletResponseWrapper.getOutputStream(ServletResponseWrapper.java:102)
      at org.kohsuke.stapler.ResponseImpl.getOutputStream(ResponseImpl.java:82)
      at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:405)
      at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:242)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:150)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:51)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
      at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
      at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
      at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)
      at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
      at java.lang.Thread.run(Thread.java:636)

      Attachments

        Activity

          keperry keperry created issue -
          keperry keperry added a comment -

          Note. This was on Hudson 1.389. I haven't tried it on 1.395.

          keperry keperry added a comment - Note. This was on Hudson 1.389. I haven't tried it on 1.395.
          dhaun Dirk Haun added a comment -

          FWIW, I'm seeing the same thing on Jenkins 1.396:

          Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Started ThinBackup Worker Thread
          Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Finished ThinBackup Worker Thread. 2 ms
          Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Started ThinBackup Worker Thread
          Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Finished ThinBackup Worker Thread. 3 ms
          etc.

          And here I was wondering why it wouldn't backup our extensive list of jobs ...

          dhaun Dirk Haun added a comment - FWIW, I'm seeing the same thing on Jenkins 1.396: Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Started ThinBackup Worker Thread Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Finished ThinBackup Worker Thread. 2 ms Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Started ThinBackup Worker Thread Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Finished ThinBackup Worker Thread. 3 ms etc. And here I was wondering why it wouldn't backup our extensive list of jobs ...
          tofuatjava Thomas Fürer made changes -
          Field Original Value New Value
          Assignee tofuatjava [ tofuatjava ]
          tofuatjava Thomas Fürer made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          tofuatjava Thomas Fürer added a comment -

          will be available in version 1.2

          tofuatjava Thomas Fürer added a comment - will be available in version 1.2
          tofuatjava Thomas Fürer made changes -
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Resolved [ 5 ]
          benpoole Ben Poole added a comment -

          Hi Thomas,

          I'm still experiencing this issue with Jenkins 1.511 and ThinBackup 1.7.2

          logs:

          ==> /var/log/jenkins/jenkins.log <==
          Jul 17, 2013 12:42:02 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink doBackupManual
          INFO: Starting manual backup.
          Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup <init>
          INFO: No previous full backup found, thus creating one.
          Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup backupJobsDirectory
          INFO: Found 53 jobs to back up.
          Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork backupNow
          SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/var/lib/jenkins/backups'.
          java.io.FileNotFoundException: Source '/var/lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds/lastUnstableBuild' does not exist
                  at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
                  at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:814)
                  at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildFiles(HudsonBackup.java:302)
                  at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildsFor(HudsonBackup.java:270)
                  at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJob(HudsonBackup.java:208)
                  at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobsDirectory(HudsonBackup.java:196)
                  at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobs(HudsonBackup.java:175)
                  at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:142)
                  at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:87)
                  at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76)
                  at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:54)
                  at java.lang.Thread.run(Thread.java:722)
          

          I think the write privileges error is a red herring because jenkins does have the correct privs access to the dir.

          Cheers
          Ben

          benpoole Ben Poole added a comment - Hi Thomas, I'm still experiencing this issue with Jenkins 1.511 and ThinBackup 1.7.2 logs: ==> / var /log/jenkins/jenkins.log <== Jul 17, 2013 12:42:02 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink doBackupManual INFO: Starting manual backup. Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup <init> INFO: No previous full backup found, thus creating one. Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup backupJobsDirectory INFO: Found 53 jobs to back up. Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork backupNow SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/ var /lib/jenkins/backups' . java.io.FileNotFoundException: Source '/ var /lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds/lastUnstableBuild' does not exist at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866) at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:814) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildFiles(HudsonBackup.java:302) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildsFor(HudsonBackup.java:270) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJob(HudsonBackup.java:208) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobsDirectory(HudsonBackup.java:196) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobs(HudsonBackup.java:175) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:142) at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:87) at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76) at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:54) at java.lang. Thread .run( Thread .java:722) I think the write privileges error is a red herring because jenkins does have the correct privs access to the dir. Cheers Ben
          benpoole Ben Poole made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          benpoole Ben Poole added a comment -
          root@lamorak:/var/lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds# ls -lh
          total 4.0K
          lrwxrwxrwx 1 root root   19 Jul 11 17:03 1 -> 2013-07-11_17-03-59
          drwxr-xr-x 2 root root 4.0K Jul 11 17:09 2013-07-11_17-03-59
          lrwxrwxrwx 1 root root    2 Jul 11 15:01 lastFailedBuild -> -1
          lrwxrwxrwx 1 root root    1 Jul 11 17:09 lastStableBuild -> 1
          lrwxrwxrwx 1 root root    1 Jul 11 17:09 lastSuccessfulBuild -> 1
          lrwxrwxrwx 1 root root    2 Jul 11 15:01 lastUnstableBuild -> -1
          lrwxrwxrwx 1 root root    2 Jul 11 15:01 lastUnsuccessfulBuild -> -1
          
          benpoole Ben Poole added a comment - root@lamorak:/ var /lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds# ls -lh total 4.0K lrwxrwxrwx 1 root root 19 Jul 11 17:03 1 -> 2013-07-11_17-03-59 drwxr-xr-x 2 root root 4.0K Jul 11 17:09 2013-07-11_17-03-59 lrwxrwxrwx 1 root root 2 Jul 11 15:01 lastFailedBuild -> -1 lrwxrwxrwx 1 root root 1 Jul 11 17:09 lastStableBuild -> 1 lrwxrwxrwx 1 root root 1 Jul 11 17:09 lastSuccessfulBuild -> 1 lrwxrwxrwx 1 root root 2 Jul 11 15:01 lastUnstableBuild -> -1 lrwxrwxrwx 1 root root 2 Jul 11 15:01 lastUnsuccessfulBuild -> -1
          tofuatjava Thomas Fürer added a comment -

          Hi bob,
          this is not related to the original issue, which is fixed.
          the newer problem is related to https://issues.jenkins-ci.org/browse/JENKINS-17475, which I'm currently working on. So please vote for JENKINS-17475 where I close this issue.

          tofuatjava Thomas Fürer added a comment - Hi bob, this is not related to the original issue, which is fixed. the newer problem is related to https://issues.jenkins-ci.org/browse/JENKINS-17475 , which I'm currently working on. So please vote for JENKINS-17475 where I close this issue.
          tofuatjava Thomas Fürer made changes -
          Resolution Fixed [ 1 ]
          Status Reopened [ 4 ] Resolved [ 5 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 138717 ] JNJira + In-Review [ 188125 ]

          People

            tofuatjava Thomas Fürer
            keperry keperry
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: