Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-8606

ThinBackup only backs one job out of 10, and continually runs in the background

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • thinbackup-plugin
    • None
    • Linux, OpenSUSE 11.2, Hudson running inside JBOSS 5.1, 2GB ram, Dual core processor.

      When thinBackup runs it only performs the backup on one job (under the jobs folder). It seems like it is continually running throughout the day...when I have the backup scheduled at 2:00AM.
      From the log, it runs all day and every minute or so it shows this in the log:
      2011-01-24 09:22:14,093 INFO [org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork] (ThinBackup Worker Thread thread) Started ThinBackup Worker Thread
      2011-01-24 09:22:14,094 INFO [org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork] (ThinBackup Worker Thread thread) Finished ThinBackup Worker Thread. 1 ms

      There is also this exception in the log, but I am having a hard time figuring out if it is related:

      2011-01-24 09:22:44,120 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/hudson].[Stapler]] (http-10.7.0.113-8080-3) Servlet.service() for servlet Stapler threw exception
      java.lang.IllegalStateException: getWriter() has already been called for this response
      at org.apache.catalina.connector.Response.getOutputStream(Response.java:588)
      at org.apache.catalina.connector.ResponseFacade.getOutputStream(ResponseFacade.java:183)
      at javax.servlet.ServletResponseWrapper.getOutputStream(ServletResponseWrapper.java:102)
      at javax.servlet.ServletResponseWrapper.getOutputStream(ServletResponseWrapper.java:102)
      at org.kohsuke.stapler.ResponseImpl.getOutputStream(ResponseImpl.java:82)
      at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:405)
      at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:242)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:150)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:51)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
      at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
      at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
      at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)
      at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
      at java.lang.Thread.run(Thread.java:636)

          [JENKINS-8606] ThinBackup only backs one job out of 10, and continually runs in the background

          keperry added a comment -

          Note. This was on Hudson 1.389. I haven't tried it on 1.395.

          keperry added a comment - Note. This was on Hudson 1.389. I haven't tried it on 1.395.

          Dirk Haun added a comment -

          FWIW, I'm seeing the same thing on Jenkins 1.396:

          Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Started ThinBackup Worker Thread
          Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Finished ThinBackup Worker Thread. 2 ms
          Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Started ThinBackup Worker Thread
          Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Finished ThinBackup Worker Thread. 3 ms
          etc.

          And here I was wondering why it wouldn't backup our extensive list of jobs ...

          Dirk Haun added a comment - FWIW, I'm seeing the same thing on Jenkins 1.396: Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Started ThinBackup Worker Thread Feb 10, 2011 3:34:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Finished ThinBackup Worker Thread. 2 ms Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Started ThinBackup Worker Thread Feb 10, 2011 3:35:54 PM hudson.model.AsyncPeriodicWork$1 run INFO: Finished ThinBackup Worker Thread. 3 ms etc. And here I was wondering why it wouldn't backup our extensive list of jobs ...

          Thomas Fürer added a comment -

          will be available in version 1.2

          Thomas Fürer added a comment - will be available in version 1.2

          Ben Poole added a comment -

          Hi Thomas,

          I'm still experiencing this issue with Jenkins 1.511 and ThinBackup 1.7.2

          logs:

          ==> /var/log/jenkins/jenkins.log <==
Jul 17, 2013 12:42:02 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink doBackupManual
INFO: Starting manual backup.
Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup <init>
INFO: No previous full backup found, thus creating one.
Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup backupJobsDirectory
INFO: Found 53 jobs to back up.
Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork backupNow
SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/var/lib/jenkins/backups'.
java.io.FileNotFoundException: Source '/var/lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds/lastUnstableBuild' does not exist
        at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
        at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:814)
        at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildFiles(HudsonBackup.java:302)
        at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildsFor(HudsonBackup.java:270)
        at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJob(HudsonBackup.java:208)
        at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobsDirectory(HudsonBackup.java:196)
        at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobs(HudsonBackup.java:175)
        at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:142)
        at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:87)
        at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76)
        at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:54)
        at java.lang.Thread.run(Thread.java:722)

          I think the write privileges error is a red herring because jenkins does have the correct privs access to the dir.

          Cheers
          Ben

          Ben Poole added a comment - Hi Thomas, I'm still experiencing this issue with Jenkins 1.511 and ThinBackup 1.7.2 logs: ==> / var /log/jenkins/jenkins.log <== Jul 17, 2013 12:42:02 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink doBackupManual INFO: Starting manual backup. Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup <init> INFO: No previous full backup found, thus creating one. Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup backupJobsDirectory INFO: Found 53 jobs to back up. Jul 17, 2013 12:42:03 PM org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork backupNow SEVERE: Cannot perform a backup. Please be sure jenkins/hudson has write privileges in the configured backup path '/ var /lib/jenkins/backups' . java.io.FileNotFoundException: Source '/ var /lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds/lastUnstableBuild' does not exist at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866) at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:814) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildFiles(HudsonBackup.java:302) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupBuildsFor(HudsonBackup.java:270) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJob(HudsonBackup.java:208) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobsDirectory(HudsonBackup.java:196) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backupJobs(HudsonBackup.java:175) at org.jvnet.hudson.plugins.thinbackup.backup.HudsonBackup.backup(HudsonBackup.java:142) at org.jvnet.hudson.plugins.thinbackup.ThinBackupPeriodicWork.backupNow(ThinBackupPeriodicWork.java:87) at org.jvnet.hudson.plugins.thinbackup.ThinBackupMgmtLink$1.execute(ThinBackupMgmtLink.java:76) at org.jvnet.hudson.plugins.thinbackup.hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:54) at java.lang. Thread .run( Thread .java:722) I think the write privileges error is a red herring because jenkins does have the correct privs access to the dir. Cheers Ben

          Ben Poole added a comment - 
          root@lamorak:/var/lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds# ls -lh
total 4.0K
lrwxrwxrwx 1 root root   19 Jul 11 17:03 1 -> 2013-07-11_17-03-59
drwxr-xr-x 2 root root 4.0K Jul 11 17:09 2013-07-11_17-03-59
lrwxrwxrwx 1 root root    2 Jul 11 15:01 lastFailedBuild -> -1
lrwxrwxrwx 1 root root    1 Jul 11 17:09 lastStableBuild -> 1
lrwxrwxrwx 1 root root    1 Jul 11 17:09 lastSuccessfulBuild -> 1
lrwxrwxrwx 1 root root    2 Jul 11 15:01 lastUnstableBuild -> -1
lrwxrwxrwx 1 root root    2 Jul 11 15:01 lastUnsuccessfulBuild -> -1

          Ben Poole added a comment - root@lamorak:/ var /lib/jenkins/jobs/nationalvoices-INTEGRATION_LAMORAK/builds# ls -lh total 4.0K lrwxrwxrwx 1 root root 19 Jul 11 17:03 1 -> 2013-07-11_17-03-59 drwxr-xr-x 2 root root 4.0K Jul 11 17:09 2013-07-11_17-03-59 lrwxrwxrwx 1 root root 2 Jul 11 15:01 lastFailedBuild -> -1 lrwxrwxrwx 1 root root 1 Jul 11 17:09 lastStableBuild -> 1 lrwxrwxrwx 1 root root 1 Jul 11 17:09 lastSuccessfulBuild -> 1 lrwxrwxrwx 1 root root 2 Jul 11 15:01 lastUnstableBuild -> -1 lrwxrwxrwx 1 root root 2 Jul 11 15:01 lastUnsuccessfulBuild -> -1

          Thomas Fürer added a comment -

          Hi bob,
          this is not related to the original issue, which is fixed.
          the newer problem is related to https://issues.jenkins-ci.org/browse/JENKINS-17475, which I'm currently working on. So please vote for JENKINS-17475 where I close this issue.

          Thomas Fürer added a comment - Hi bob, this is not related to the original issue, which is fixed. the newer problem is related to https://issues.jenkins-ci.org/browse/JENKINS-17475 , which I'm currently working on. So please vote for JENKINS-17475 where I close this issue.

            tofuatjava Thomas Fürer
            keperry keperry
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: