Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-73241

Jenkins fails checkout of git commit that adds submodules where files were previously present

      There is a bug (???) in git where the checkout of a previous commit that had files in a path where the a submodule of the current commit also has files, the checkout will fail. This is because git will cowardly refuse to overwrite the unindexed (in the parent repo) files before doing anything. Running checkout with the -f option solves the problem.

      #!/bin/sh
      git init aaa
      git init bbb
      
      pushd bbb
      touch b1
      git add b1 && git commit -m b1
      popd bbb
      
      pushd aaa
      mkdir b && touch b/b1
      git add b/b1 && git commit -m a1
      git rm -rf b && git submodule add ../b b
      git commit -m a2
      popd aaa
      
      git clone aaa aaa.test
      pushd aaa.test
      git submodule update --recursive --init
      # This will fail because b/b1 already exists and is outside of aaa.test's index
      git checkout HEAD~1
      # This works
      git checkout -f HEAD~1

      However, this breaks the normal operation of the Jenkins Git plugin, as it doesn't use "-f" when updating submodules.

      The only workaround I can come up with right now is, when using Groovy pipelines, wrap the checkout command in a Shared Library where the SubmoduleOption is passed to the original checkout as disabled, then running "git submodule --update --recursive --force" instead if the option was enabled by the arguments.

      // checkout.groovy
      def call(args) {
          def submoduleOptionEnabled = false
          // TODO: Set submoduleOptionEnabled to true if SubmoduleOption was enabled
          // TODO: Force disable SubmoduleOption
          binding.steps.checkout(args)
          if(submoduleOptionEnabled) {
              binding.steps.sh('git submodule update --recursive --update --force')
          }
      }

          [JENKINS-73241] Jenkins fails checkout of git commit that adds submodules where files were previously present

          Danolo created issue -
          Danolo made changes -
          Description Original: There is a bug (???) in git where the checkout of a previous commit that had files in a path where the a submodule of the current commit also has files, the checkout will fail. This is because git will cowardly refuse to overwrite the unindexed (in the parent repo) files before doing anything. Running checkout with the -f option solves the problem.
          {code:java}
          #!/bin/sh
          git init aaa
          git init bbb

          pushd bbb
          touch b1
          git add b1 && git commit -m b1
          popd bbb

          pushd aaa
          mkdir b && touch b/b1
          git add b/b1 && git commit -m a1
          git rm -rf b && git submodule add ../b b
          git commit -m a2
          popd aaa

          git clone aaa aaa.test
          pushd aaa.test
          git submodule update --recursive --init
          # This command will fail because b/b1 already exists and is outside of aaa.test's index
          git checkout HEAD~1
          # This works
          git checkout -f HEAD~1{code}
          However, this breaks the normal operation of the Jenkins Git plugin, as it doesn't use "-f" when updating submodules as configured by the SubmoduleOption option.

          The only workaround I can come up with right now is, when using Groovy pipelines, wrap the checkout command in a Shared Library where the SubmoduleOption is passed to the original checkout as disabled, then running "git submodule --update --recursive --force" instead if the option was enabled by the arguments.
          {code:java}
          // checkout.groovy
          def call(args) {
              def submoduleOptionEnabled = false
              // TODO: Set submoduleOptionenabled if SubmoduleOption was enabled
              // TODO: Force disable SubmoduleOption
              binding.steps.checkout(args)
              if(submoduleOptionEnabled) {
                  binding.steps.sh('git submodule update --recursive --update --force')
              }
          }{code}
          New: There is a bug (???) in git where the checkout of a previous commit that had files in a path where the a submodule of the current commit also has files, the checkout will fail. This is because git will cowardly refuse to overwrite the unindexed (in the parent repo) files before doing anything. Running checkout with the -f option solves the problem.
          {code:java}
          #!/bin/sh
          git init aaa
          git init bbb

          pushd bbb
          touch b1
          git add b1 && git commit -m b1
          popd bbb

          pushd aaa
          mkdir b && touch b/b1
          git add b/b1 && git commit -m a1
          git rm -rf b && git submodule add ../b b
          git commit -m a2
          popd aaa

          git clone aaa aaa.test
          pushd aaa.test
          git submodule update --recursive --init
          # This command will fail because b/b1 already exists and is outside of aaa.test's index
          git checkout HEAD~1
          # This works
          git checkout -f HEAD~1{code}
          However, this breaks the normal operation of the Jenkins Git plugin, as it doesn't use "-f" when updating submodules as configured by the SubmoduleOption option.

          The only workaround I can come up with right now is, when using Groovy pipelines, wrap the checkout command in a Shared Library where the SubmoduleOption is passed to the original checkout as disabled, then running "git submodule --update --recursive --force" instead if the option was enabled by the arguments.
          {code:java}
          // checkout.groovy
          def call(args) {
              def submoduleOptionEnabled = false
              // TODO: Set submoduleOptionEnabled to true if SubmoduleOption was enabled
              // TODO: Force disable SubmoduleOption
              binding.steps.checkout(args)
              if(submoduleOptionEnabled) {
                  binding.steps.sh('git submodule update --recursive --update --force')
              }
          }{code}
          Danolo made changes -
          Description Original: There is a bug (???) in git where the checkout of a previous commit that had files in a path where the a submodule of the current commit also has files, the checkout will fail. This is because git will cowardly refuse to overwrite the unindexed (in the parent repo) files before doing anything. Running checkout with the -f option solves the problem.
          {code:java}
          #!/bin/sh
          git init aaa
          git init bbb

          pushd bbb
          touch b1
          git add b1 && git commit -m b1
          popd bbb

          pushd aaa
          mkdir b && touch b/b1
          git add b/b1 && git commit -m a1
          git rm -rf b && git submodule add ../b b
          git commit -m a2
          popd aaa

          git clone aaa aaa.test
          pushd aaa.test
          git submodule update --recursive --init
          # This command will fail because b/b1 already exists and is outside of aaa.test's index
          git checkout HEAD~1
          # This works
          git checkout -f HEAD~1{code}
          However, this breaks the normal operation of the Jenkins Git plugin, as it doesn't use "-f" when updating submodules as configured by the SubmoduleOption option.

          The only workaround I can come up with right now is, when using Groovy pipelines, wrap the checkout command in a Shared Library where the SubmoduleOption is passed to the original checkout as disabled, then running "git submodule --update --recursive --force" instead if the option was enabled by the arguments.
          {code:java}
          // checkout.groovy
          def call(args) {
              def submoduleOptionEnabled = false
              // TODO: Set submoduleOptionEnabled to true if SubmoduleOption was enabled
              // TODO: Force disable SubmoduleOption
              binding.steps.checkout(args)
              if(submoduleOptionEnabled) {
                  binding.steps.sh('git submodule update --recursive --update --force')
              }
          }{code}
          New: There is a bug (???) in git where the checkout of a previous commit that had files in a path where the a submodule of the current commit also has files, the checkout will fail. This is because git will cowardly refuse to overwrite the unindexed (in the parent repo) files before doing anything. Running checkout with the -f option solves the problem.
          {code:java}
          #!/bin/sh
          git init aaa
          git init bbb

          pushd bbb
          touch b1
          git add b1 && git commit -m b1
          popd bbb

          pushd aaa
          mkdir b && touch b/b1
          git add b/b1 && git commit -m a1
          git rm -rf b && git submodule add ../b b
          git commit -m a2
          popd aaa

          git clone aaa aaa.test
          pushd aaa.test
          git submodule update --recursive --init
          # This will fail because b/b1 already exists and is outside of aaa.test's index
          git checkout HEAD~1
          # This works
          git checkout -f HEAD~1{code}
          However, this breaks the normal operation of the Jenkins Git plugin, as it doesn't use "-f" when updating submodules as configured by the SubmoduleOption option.

          The only workaround I can come up with right now is, when using Groovy pipelines, wrap the checkout command in a Shared Library where the SubmoduleOption is passed to the original checkout as disabled, then running "git submodule --update --recursive --force" instead if the option was enabled by the arguments.
          {code:java}
          // checkout.groovy
          def call(args) {
              def submoduleOptionEnabled = false
              // TODO: Set submoduleOptionEnabled to true if SubmoduleOption was enabled
              // TODO: Force disable SubmoduleOption
              binding.steps.checkout(args)
              if(submoduleOptionEnabled) {
                  binding.steps.sh('git submodule update --recursive --update --force')
              }
          }{code}
          Danolo made changes -
          Description Original: There is a bug (???) in git where the checkout of a previous commit that had files in a path where the a submodule of the current commit also has files, the checkout will fail. This is because git will cowardly refuse to overwrite the unindexed (in the parent repo) files before doing anything. Running checkout with the -f option solves the problem.
          {code:java}
          #!/bin/sh
          git init aaa
          git init bbb

          pushd bbb
          touch b1
          git add b1 && git commit -m b1
          popd bbb

          pushd aaa
          mkdir b && touch b/b1
          git add b/b1 && git commit -m a1
          git rm -rf b && git submodule add ../b b
          git commit -m a2
          popd aaa

          git clone aaa aaa.test
          pushd aaa.test
          git submodule update --recursive --init
          # This will fail because b/b1 already exists and is outside of aaa.test's index
          git checkout HEAD~1
          # This works
          git checkout -f HEAD~1{code}
          However, this breaks the normal operation of the Jenkins Git plugin, as it doesn't use "-f" when updating submodules as configured by the SubmoduleOption option.

          The only workaround I can come up with right now is, when using Groovy pipelines, wrap the checkout command in a Shared Library where the SubmoduleOption is passed to the original checkout as disabled, then running "git submodule --update --recursive --force" instead if the option was enabled by the arguments.
          {code:java}
          // checkout.groovy
          def call(args) {
              def submoduleOptionEnabled = false
              // TODO: Set submoduleOptionEnabled to true if SubmoduleOption was enabled
              // TODO: Force disable SubmoduleOption
              binding.steps.checkout(args)
              if(submoduleOptionEnabled) {
                  binding.steps.sh('git submodule update --recursive --update --force')
              }
          }{code}
          New: There is a bug (???) in git where the checkout of a previous commit that had files in a path where the a submodule of the current commit also has files, the checkout will fail. This is because git will cowardly refuse to overwrite the unindexed (in the parent repo) files before doing anything. Running checkout with the -f option solves the problem.
          {code:java}
          #!/bin/sh
          git init aaa
          git init bbb

          pushd bbb
          touch b1
          git add b1 && git commit -m b1
          popd bbb

          pushd aaa
          mkdir b && touch b/b1
          git add b/b1 && git commit -m a1
          git rm -rf b && git submodule add ../b b
          git commit -m a2
          popd aaa

          git clone aaa aaa.test
          pushd aaa.test
          git submodule update --recursive --init
          # This will fail because b/b1 already exists and is outside of aaa.test's index
          git checkout HEAD~1
          # This works
          git checkout -f HEAD~1{code}
          However, this breaks the normal operation of the Jenkins Git plugin, as it doesn't use "-f" when updating submodules.

          The only workaround I can come up with right now is, when using Groovy pipelines, wrap the checkout command in a Shared Library where the SubmoduleOption is passed to the original checkout as disabled, then running "git submodule --update --recursive --force" instead if the option was enabled by the arguments.
          {code:java}
          // checkout.groovy
          def call(args) {
              def submoduleOptionEnabled = false
              // TODO: Set submoduleOptionEnabled to true if SubmoduleOption was enabled
              // TODO: Force disable SubmoduleOption
              binding.steps.checkout(args)
              if(submoduleOptionEnabled) {
                  binding.steps.sh('git submodule update --recursive --update --force')
              }
          }{code}
          Danolo made changes -
          Link New: This issue is related to JENKINS-55876 [ JENKINS-55876 ]
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]

            Unassigned Unassigned
            danoloan Danolo
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: