Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15829

hg share always recreating working copy on slave (on master it's working fine)

XMLWordPrintable

      We recently enabled Use Repository Sharing option in mercurial because we need all branches and there is no such option - we used fake working copy (to have changes in build) before and were manually getting data from repository, also it faster and saves space as we have a lot of jobs for the same repository.
      As a result we have full build on slaves instead of incremental build.
      Jenkins ver. 1.490 Jenkins Mercurial plugin 1.42

      First run on master
      Started by user anonymous
      Building on master in workspace /app/agora/hudson/jobs/ShareTest/workspace
      Acquired master cache lock.
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg pull
      pulling from http://url/hg/AgoraLight
      searching for changes
      no changes found
      Master cache lock released.
      $ hg --config extensions.share= share --noupdate /app/agora/hudson/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight /app/agora/hudson/jobs/ShareTest/workspace/AgoraLight
      [AgoraLight] $ hg update --rev default
      3843 files updated, 0 files merged, 0 files removed, 0 files unresolved
      [AgoraLight] $ hg log --rev . --template

      {node}
      [AgoraLight] $ hg log --rev . --template {rev}
      [AgoraLight] $ hg log --rev 8ff039e2eee43cd13859f650c500ac9f67235d5c
      [AgoraLight] $ hg log --template "<changeset node='{node}

      ' author='

      {author|xmlescape}' rev='{rev}' date='{date}'><msg>{desc|xmlescape}</msg><added>{file_adds|stringify|xmlescape}</added><deleted>{file_dels|stringify|xmlescape}</deleted><files>{files|stringify|xmlescape}</files><parents>{parents}</parents></changeset>\n" --rev default:0 --follow --prune 8ff039e2eee43cd13859f650c500ac9f67235d5c
      Notifying upstream projects of job completion
      Finished: SUCCESS

      Second run on master
      Started by user anonymous
      Building on master in workspace /app/agora/hudson/jobs/ShareTest/workspace
      [AgoraLight] $ hg showconfig paths.default
      Acquired master cache lock.
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg pull
      pulling from http://url/hg/AgoraLight
      searching for changes
      no changes found
      Master cache lock released.
      [AgoraLight] $ hg pull --rev default /app/agora/hudson/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight
      [AgoraLight] $ hg update --clean --rev default
      0 files updated, 0 files merged, 0 files removed, 0 files unresolved
      [AgoraLight] $ hg log --rev . --template {node}
      [AgoraLight] $ hg log --rev . --template {rev}
      [AgoraLight] $ hg log --rev 8ff039e2eee43cd13859f650c500ac9f67235d5c
      [AgoraLight] $ hg log --template "<changeset node='{node}' author='{author|xmlescape}

      ' rev='

      {rev}' date='{date}'><msg>{desc|xmlescape}</msg><added>{file_adds|stringify|xmlescape}</added><deleted>{file_dels|stringify|xmlescape}</deleted><files>{files|stringify|xmlescape}</files><parents>{parents}</parents></changeset>\n" --rev default:0 --follow --prune 8ff039e2eee43cd13859f650c500ac9f67235d5c
      Notifying upstream projects of job completion
      Finished: SUCCESS

      First run on slave
      Started by user anonymous
      Building remotely on cll36a-9516 in workspace /app/agora/hudsonslave/workspace/ShareTest
      Acquired master cache lock.
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg pull
      pulling from http://url/hg/AgoraLight
      searching for changes
      no changes found
      Master cache lock released.
      Acquired slave node cache lock for node cll36a-9516.
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg heads --template {node}\n --topo --closed
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg heads --template {node}\n --topo --closed
      Local cache is up to date.
      Slave node cache lock released for node cll36a-9516.
      $ hg --config extensions.share= share --noupdate /app/agora/hudsonslave/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight /app/agora/hudsonslave/workspace/ShareTest/AgoraLight
      [AgoraLight] $ hg update --rev default
      3843 files updated, 0 files merged, 0 files removed, 0 files unresolved
      [AgoraLight] $ hg log --rev . --template {node}
      [AgoraLight] $ hg log --rev . --template {rev}

      [AgoraLight] $ hg log --rev 8ff039e2eee43cd13859f650c500ac9f67235d5c
      [AgoraLight] $ hg log --template "<changeset node='

      {node}' author='{author|xmlescape}' rev='{rev}' date='{date}'><msg>{desc|xmlescape}</msg><added>{file_adds|stringify|xmlescape}</added><deleted>{file_dels|stringify|xmlescape}</deleted><files>{files|stringify|xmlescape}</files><parents>{parents}</parents></changeset>\n" --rev default:0 --follow --prune 8ff039e2eee43cd13859f650c500ac9f67235d5c
      Notifying upstream projects of job completion
      Finished: SUCCESS

      Second run on slave
      Started by user anonymous
      Building remotely on cll36a-9516 in workspace /app/agora/hudsonslave/workspace/ShareTest
      Acquired master cache lock.
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg pull
      pulling from http://url/hg/AgoraLight
      searching for changes
      no changes found
      Master cache lock released.
      Acquired slave node cache lock for node cll36a-9516.
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg heads --template {node}

      \n --topo --closed
      [D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight] $ hg heads --template

      {node}\n --topo --closed
      Local cache is up to date.
      Slave node cache lock released for node cll36a-9516.
      $ hg --config extensions.share= share --noupdate /app/agora/hudsonslave/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight /app/agora/hudsonslave/workspace/ShareTest/AgoraLight
      [AgoraLight] $ hg update --rev default
      3843 files updated, 0 files merged, 0 files removed, 0 files unresolved
      [AgoraLight] $ hg log --rev . --template {node}

      [AgoraLight] $ hg log --rev . --template

      {rev}
      [AgoraLight] $ hg log --rev 8ff039e2eee43cd13859f650c500ac9f67235d5c
      [AgoraLight] $ hg log --template "<changeset node='{node}' author='{author|xmlescape}' rev='{rev}

      ' date='

      {date}

      '><msg>

      {desc|xmlescape}

      </msg><added>

      {file_adds|stringify|xmlescape}

      </added><deleted>

      {file_dels|stringify|xmlescape}

      </deleted><files>

      {files|stringify|xmlescape}

      </files><parents>

      {parents}

      </parents></changeset>\n" --rev default:0 --follow --prune 8ff039e2eee43cd13859f650c500ac9f67235d5c
      Notifying upstream projects of job completion
      Finished: SUCCESS

          [JENKINS-15829] hg share always recreating working copy on slave (on master it's working fine)

          Willem Verstraeten added a comment -

          Igor: what do you mean with the 'fake working copy' option? I can find a 'custom workspace' option, is that it ?

          Also, are there any other builds (of other jobs) running between the first build on the slave and the second build ?

          Willem Verstraeten added a comment - Igor: what do you mean with the 'fake working copy' option? I can find a 'custom workspace' option, is that it ? Also, are there any other builds (of other jobs) running between the first build on the slave and the second build ?

          Igor Kostenko added a comment - - edited

          > what do you mean with the 'fake working copy' option
          It's not an option - we just specifying Directory parameter of mercurial plugin as fake and not using this directory - we did it for having changes information on build's page.
          It's not related to current bug, and we are using directory created by plugin now, since enabling Use Repository Sharing option.

          > Also, are there any other builds (of other jobs) running between the first build on the slave and the second build ?
          Nothing is running between first and second builds and I see the same behavior on linux, solaris and windows slaves - only master (on linux) is fine.

          Igor Kostenko added a comment - - edited > what do you mean with the 'fake working copy' option It's not an option - we just specifying Directory parameter of mercurial plugin as fake and not using this directory - we did it for having changes information on build's page. It's not related to current bug, and we are using directory created by plugin now, since enabling Use Repository Sharing option. > Also, are there any other builds (of other jobs) running between the first build on the slave and the second build ? Nothing is running between first and second builds and I see the same behavior on linux, solaris and windows slaves - only master (on linux) is fine.

          Igor Kostenko added a comment -

          If somebody else have the same issue, here the workaround which I'm using on slaves:
          In Source Code Management / Mercurial / Advanced / Subdirectory I'm specifying directory fake and adding at the beginning of build
          Unix:

          if [ -d MyRepo ]; then
    hg -R MyRepo update
else
    hg --config extensions.share= share fake MyRepo
fi

          Windows:

          if exist MyRepo\nul (hg -R MyRepo update) else (hg --config extensions.share= share fake MyRepo)

          Igor Kostenko added a comment - If somebody else have the same issue, here the workaround which I'm using on slaves: In Source Code Management / Mercurial / Advanced / Subdirectory I'm specifying directory fake and adding at the beginning of build Unix: if [ -d MyRepo ]; then hg -R MyRepo update else hg --config extensions.share= share fake MyRepo fi Windows: if exist MyRepo\nul (hg -R MyRepo update) else (hg --config extensions.share= share fake MyRepo)

          Willem Verstraeten added a comment -

          I think this is because the mercurial caches on the slave don't have a default path to push and pull from.

          Igor: could you, on your slave, create a .hg/hgrc file in the /app/agora/hudsonslave/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight that contains the following information:

          [paths]
default=__URL_OF_YOUR_MASTER_REPO__

          Does the working copy get recreated every time then, after you remove your workaround?

          Willem Verstraeten added a comment - I think this is because the mercurial caches on the slave don't have a default path to push and pull from. Igor: could you, on your slave, create a .hg/hgrc file in the /app/agora/hudsonslave/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight that contains the following information: [paths] default=__URL_OF_YOUR_MASTER_REPO__ Does the working copy get recreated every time then, after you remove your workaround?

          Jesse Glick added a comment -

          It is true that when caching is enabled, slave repos have no paths.default: changeset bundles are transferred over the Jenkins remoting channel instead.

          Jesse Glick added a comment - It is true that when caching is enabled, slave repos have no paths.default : changeset bundles are transferred over the Jenkins remoting channel instead.

          Igor Kostenko added a comment -

          Sorry for long pause, I've changed a job and only now responsible for jenkins again.

          Igor: could you, on your slave, create a .hg/hgrc file in the /app/agora/hudsonslave/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight that contains the following information:

          It helped: 

          [QAF_Test2] $ hg pull --rev default /home/jenkins_local/hgcache/82556880B412DFAE704E615CE9ABB4DA5A38C2EF-doc_source
[QAF_Test2] $ hg update --clean --rev default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

          Igor Kostenko added a comment - Sorry for long pause, I've changed a job and only now responsible for jenkins again. Igor: could you, on your slave, create a .hg/hgrc file in the /app/agora/hudsonslave/hgcache/D38534FF4534AE3232A1AB292D5BA54825AE752B-AgoraLight that contains the following information: It helped: [QAF_Test2] $ hg pull --rev default /home/jenkins_local/hgcache/82556880B412DFAE704E615CE9ABB4DA5A38C2EF-doc_source [QAF_Test2] $ hg update --clean --rev default 0 files updated, 0 files merged, 0 files removed, 0 files unresolved

          Brandon Turner added a comment -

          Here's a potential patch that works for me: https://github.com/jenkinsci/mercurial-plugin/pull/50

          Brandon Turner added a comment - Here's a potential patch that works for me: https://github.com/jenkinsci/mercurial-plugin/pull/50

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Brandon Turner
          Path:
          src/main/java/hudson/plugins/mercurial/MercurialSCM.java
          http://jenkins-ci.org/commit/mercurial-plugin/3173ad4643f4be0aaa43f2bae3aabd93ba852e2f
          Log:
          Don't recreate workspace when using Repository Sharing

          When using Repository Sharing, the working copy is always recloned (from
          the slave cache), which can take a long time on large repos.

          This fixes the logic used to detect whether a workspace copy can be
          reused to support Repository Sharing.

          JENKINS-15829

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Brandon Turner Path: src/main/java/hudson/plugins/mercurial/MercurialSCM.java http://jenkins-ci.org/commit/mercurial-plugin/3173ad4643f4be0aaa43f2bae3aabd93ba852e2f Log: Don't recreate workspace when using Repository Sharing When using Repository Sharing, the working copy is always recloned (from the slave cache), which can take a long time on large repos. This fixes the logic used to detect whether a workspace copy can be reused to support Repository Sharing. JENKINS-15829

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/hudson/plugins/mercurial/MercurialSCM.java
          http://jenkins-ci.org/commit/mercurial-plugin/a1839d86a7c60105c5a03f48978f39b82bc01eb0
          Log:
          Merge pull request #50 from blt04/fix-shared-working-copy-update

          [FIXED JENKINS-15829] Don't recreate workspace when using Repository Sharing

          Compare: https://github.com/jenkinsci/mercurial-plugin/compare/182c4952b574...a1839d86a7c6

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/plugins/mercurial/MercurialSCM.java http://jenkins-ci.org/commit/mercurial-plugin/a1839d86a7c60105c5a03f48978f39b82bc01eb0 Log: Merge pull request #50 from blt04/fix-shared-working-copy-update [FIXED JENKINS-15829] Don't recreate workspace when using Repository Sharing Compare: https://github.com/jenkinsci/mercurial-plugin/compare/182c4952b574...a1839d86a7c6

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/test/java/hudson/plugins/mercurial/SCMTestBase.java
          src/test/java/hudson/plugins/mercurial/SharingSCMTest.java
          http://jenkins-ci.org/commit/mercurial-plugin/36ac77a536be921a1367488318331308f0d1cf55
          Log:
          JENKINS-15829 Verified fix with test (see #50).

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/test/java/hudson/plugins/mercurial/SCMTestBase.java src/test/java/hudson/plugins/mercurial/SharingSCMTest.java http://jenkins-ci.org/commit/mercurial-plugin/36ac77a536be921a1367488318331308f0d1cf55 Log: JENKINS-15829 Verified fix with test (see #50).

            willemv Willem Verstraeten
            igorkostenko Igor Kostenko
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: