Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33255

Largefiles do not work with distributed repository caching


    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • mercurial-plugin
    • Jenkins 1.649, Mercurial plugin 1.54, Mercurial 3.7.1, Centos 7 on the master, Ubuntu 14.04LTS Server on the slave

      When repository caching is enabled for distributed builds, the repository that eventually ends up in the workspace on the slave has lost upstream path information such that it is no longer able to pull largefiles.

      For example, given a slave with:
      /var/build/hgcache/SOMEREPO - this was created by bundling changes from the master
      /var/build/workspace/MyJob - this was cloned/shared from SOMEREPO

      SOMEREPO does not contain any upstream information at all (because it was populated entirely using bundling) and MyJob only contains SOMEREPO as upstream, so when MyJob is updated to the target revision prior to building, it complains that the largefiles "are not available" from SOMEREPO.

      I think the fix for this is to detect whether largefiles are enabled, and if so, for the master to issue an explicit 'lfpull' command, specifying the original upstream URL. This should be done after changes are unbundled into SOMEREPO but before they are cloned/shared/pulled into MyJob, allowing the clone/share/pull to retrieve the largefiles 'naturally' and avoiding any warnings about an inability to find them during the initial update.

      In the long run it might be good to retrieve the largefiles from the master instead of upstream, but getting it working at all is the first step.

            superpig Richard Fine
            superpig Richard Fine
            0 Vote for this issue
            1 Start watching this issue