[JENKINS-43693] multibranchPipelineJob overrides old branch indexing sources

Type: Bug
Resolution: Fixed
Priority: Major
Component/s: job-dsl-plugin
Labels:
None
Environment:
Debian 8
Jenkins 2.46.1 (as service)

Plugins (all newest stable):
Job DSL 1.61
Pipeline: Multibranch 2.14

Similar Issues:
Powered by SuggestiMate

Show
Released As:
job-dsl 1.75

When running the multibranchPipelineJob step it seems like it overrides the sources of a Multibranch Pipeline Job instead of just updating it. After running the step, even though nothing has changed the output of the branch indexing will become:

Checking branch master
      ‘Jenkinsfile’ found
Met criteria
Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists
Branch reopened: master (b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
Scheduled build for branch: master

To test this, I have installed a brand new Jenkins with the Pipeline plugins and Job DSL plugin (all as the stable version) and made a freestyle job, with a Job DSL step that has the following script:

multibranchPipelineJob('job-dsl-plugin_multibranch') {
  branchSources {
    git { 
      remote('https://github.com/jenkinsci/job-dsl-plugin.git')
    }
  }
}

After running the Job DSL step and afterwards running branch indexing a "master" job is created with the following log message from branch indexing:

Checking branch master
      ‘Jenkinsfile’ found
Met criteria
Scheduled build for branch: master

When running branch indexing again, following log message are shown as expected:

Checking branch master
      ‘Jenkinsfile’ found
Met criteria
No changes detected: master (still at b409f3996299dd12244d6cf7f8b6d8df3aa674ca)

However, if I run the Job DSL again and run branch indexing, the first log message (with takeover and so on) are shown. This is not what I would have expected, I would have expected something like a "no changes detected" message.

is duplicated by

JENKINS-57235 Seed job (job-dsl) runs trigger a rebuild of all multibranch pipelines branches

Closed

Anders Kielsholm created issue - 2017-04-19 15:17

Anders Kielsholm made changes - 2017-04-19 15:18

Description

Original: When running the multibranchPipelineJob step it seems like it overrides the sources of a Multibranch Pipeline Job instead of just updating it. After running the step, even though nothing has changed the output of the branch indexing will become:

{code:java}
Checking branch master
  ‘Jenkinsfile’ found
Met criteria
Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists
Branch reopened: master (b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
Scheduled build for branch: master
{code}

To test this, I have installed a brand new Jenkins with the Pipeline plugins and Job DSL plugin (all as the stable version) and made a freestyle job, with a Job DSL step that has the following script:

{code:java}
multibranchPipelineJob('job-dsl-plugin_multibranch') {
  branchSources {
    git {
      remote('https://github.com/jenkinsci/job-dsl-plugin.git')
    }
  }
}
{code}

After running the Job DSL step and afterwards running branch indexing a "master" job is created with the following log message from branch indexing:
{code:java}
Checking branch master
      ‘Jenkinsfile’ found
Met criteria
Scheduled build for branch: master
{code}
When running branch indexing again, following log message are shown as expected:

{code:java}
Checking branch master
      ‘Jenkinsfile’ found
Met criteria
No changes detected: master (still at b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
{code}
However, if I run the Job DSL again and run branch indexing, the first log message (with takeover and so on) are shown. This is not what I would have expected, I would have expected something like a "no changes detected" message.

New: When running the multibranchPipelineJob step it seems like it overrides the sources of a Multibranch Pipeline Job instead of just updating it. After running the step, even though nothing has changed the output of the branch indexing will become:
{code:java}
Checking branch master
  ‘Jenkinsfile’ found
Met criteria
Takeover for job-dsl-plugin_multibranch » master by source #1 from source that no longer exists
Branch reopened: master (b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
Scheduled build for branch: master
{code}

To test this, I have installed a brand new Jenkins with the Pipeline plugins and Job DSL plugin (all as the stable version) and made a freestyle job, with a Job DSL step that has the following script:
{code:java}
multibranchPipelineJob('job-dsl-plugin_multibranch') {
  branchSources {
    git {
      remote('https://github.com/jenkinsci/job-dsl-plugin.git')
    }
  }
}
{code}

After running the Job DSL step and afterwards running branch indexing a "master" job is created with the following log message from branch indexing:
{code:java}
Checking branch master
      ‘Jenkinsfile’ found
Met criteria
Scheduled build for branch: master
{code}

When running branch indexing again, following log message are shown as expected:
{code:java}
Checking branch master
      ‘Jenkinsfile’ found
Met criteria
No changes detected: master (still at b409f3996299dd12244d6cf7f8b6d8df3aa674ca)
{code}

However, if I run the Job DSL again and run branch indexing, the first log message (with takeover and so on) are shown. This is not what I would have expected, I would have expected something like a "no changes detected" message.

Daniel Spilker added a comment - 2017-04-24 20:40 - edited

The generated config XML for the branch source contains an <id> element. Job DSL generates a random ID each time the seed job runs. I think the Multibranch plugin uses this ID to manage the repository state. So it's not OK to generate a random ID with each run of the seed job.

You can use the Automatically Generated DSL to set the ID:

multibranchPipelineJob('job-dsl-plugin_multibranch') {
  branchSources {
    branchSource {
      source {
        git {
          id('must-be-unique')
          remote('https://github.com/jenkinsci/job-dsl-plugin.git')
          credentialsId(null)
          remoteName('origin')
          rawRefSpecs('+refs/heads/*:refs/remotes/origin/*')
          includes('*')
          excludes('')
          ignoreOnPushNotifications(false)
        }
      }
    }
  }
}

Daniel Spilker added a comment - 2017-04-24 20:40 - edited The generated config XML for the branch source contains an <id> element. Job DSL generates a random ID each time the seed job runs. I think the Multibranch plugin uses this ID to manage the repository state. So it's not OK to generate a random ID with each run of the seed job. You can use the Automatically Generated DSL to set the ID: multibranchPipelineJob( 'job-dsl-plugin_multibranch' ) { branchSources { branchSource { source { git { id( 'must-be-unique' ) remote( 'https: //github.com/jenkinsci/job-dsl-plugin.git' ) credentialsId( null ) remoteName( 'origin' ) rawRefSpecs( '+refs/heads/*:refs/remotes/origin/*' ) includes( '*' ) excludes('') ignoreOnPushNotifications( false ) } } } } }

kyle rockman added a comment - 2017-04-25 13:44

HUZZAH! I've been fighting with this bug for a month or two now at work. daspilker Thank you that fixed the issue!

kyle rockman added a comment - 2017-04-25 13:44 HUZZAH! I've been fighting with this bug for a month or two now at work. daspilker Thank you that fixed the issue!

SCM/JIRA link daemon added a comment - 2017-04-30 11:12

Code changed in jenkins
User: Daniel Spilker
Path:
docs/Home.md
job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy
job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy
job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy
job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy
http://jenkins-ci.org/commit/job-dsl-plugin/1bd2d95eaae1356de6b3f92613766527280f41ca
Log:
enhanced support for the Pipeline Multibranch Plugin

[FIXES JENKINS-43693]

SCM/JIRA link daemon added a comment - 2017-04-30 11:12 Code changed in jenkins User: Daniel Spilker Path: docs/Home.md job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy http://jenkins-ci.org/commit/job-dsl-plugin/1bd2d95eaae1356de6b3f92613766527280f41ca Log: enhanced support for the Pipeline Multibranch Plugin [FIXES JENKINS-43693]

SCM/JIRA link daemon made changes - 2017-04-30 11:12

Resolution		New: Fixed [ 1 ]
Status	Original: Open [ 1 ]	New: Resolved [ 5 ]

SCM/JIRA link daemon added a comment - 2017-04-30 11:12

Code changed in jenkins
User: Daniel Spilker
Path:
docs/Home.md
job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy
job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy
job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy
job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy
http://jenkins-ci.org/commit/job-dsl-plugin/f8b51912215c545d702b6e8c8fbe6fc129fb3130
Log:
Merge branch '~~JENKINS-43693~~'

Compare: https://github.com/jenkinsci/job-dsl-plugin/compare/c722b8bb538c...f8b51912215c

SCM/JIRA link daemon added a comment - 2017-04-30 11:12 Code changed in jenkins User: Daniel Spilker Path: docs/Home.md job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitBranchSourceContext.groovy job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/helpers/workflow/GitHubBranchSourceContext.groovy job-dsl-core/src/test/groovy/javaposse/jobdsl/dsl/helpers/workflow/BranchSourcesContextsSpec.groovy http://jenkins-ci.org/commit/job-dsl-plugin/f8b51912215c545d702b6e8c8fbe6fc129fb3130 Log: Merge branch ' JENKINS-43693 ' Compare: https://github.com/jenkinsci/job-dsl-plugin/compare/c722b8bb538c...f8b51912215c

Daniel Spilker made changes - 2017-05-09 06:49

Status

Original: Resolved [ 5 ]

New: Closed [ 6 ]

Chance Zibolski added a comment - 2017-11-10 17:42

I believe I am still hitting this issue. After updating to jobDSL 1.66, after running my jobDSL job, if I try to build one of the updated multibranch projects it fails with `ERROR: Could not determine exact tip revision of master`. Is it possible this is still unfixed?

Chance Zibolski added a comment - 2017-11-10 17:42 I believe I am still hitting this issue. After updating to jobDSL 1.66, after running my jobDSL job, if I try to build one of the updated multibranch projects it fails with `ERROR: Could not determine exact tip revision of master`. Is it possible this is still unfixed?

Mike Dziedziela added a comment - 2017-12-08 13:17 - edited

I face the same issue.

Jenkins - v2.60.1

Job DSL - v1.64

Pipeline Multibranch - v2.16

I have just commited to test branch but scanning has triggered build for all branches which had Jenkinsfile in it.

Checking branch develop
      ‘Jenkinsfile’ found
    Met criteria
Takeover for example_project » develop by source #1 from source that no longer exists
Branch reopened: develop (11617f79c6408e1d5fd54ea60a64d31b787a9471)
Scheduled build for branch: develop

Checking branch test
      ‘Jenkinsfile’ found
    Met criteria
Takeover for example_project » test by source #1 from source that no longer exists
Branch reopened: test (f84bcda107da005f9363759a68af2008a135a84e)
Scheduled build for branch: test

Checking branch master
      ‘Jenkinsfile’ found
    Met criteria
Takeover for example_project » master by source #1 from source that no longer exists
Branch reopened: master (2a2521fe4ea2ab71edb51b3279b5e49c4c8e77d5)
Scheduled build for branch: master

let me know if you need more information.

EDIT:

Jenkins is integrated with GitLab v. 10.0.3 CE

Mike Dziedziela added a comment - 2017-12-08 13:17 - edited I face the same issue. Jenkins - v2.60.1 Job DSL - v1.64 Pipeline Multibranch - v2.16 I have just commited to test branch but scanning has triggered build for all branches which had Jenkinsfile in it. Checking branch develop ‘Jenkinsfile’ found Met criteria Takeover for example_project » develop by source #1 from source that no longer exists Branch reopened: develop (11617f79c6408e1d5fd54ea60a64d31b787a9471) Scheduled build for branch: develop Checking branch test ‘Jenkinsfile’ found Met criteria Takeover for example_project » test by source #1 from source that no longer exists Branch reopened: test (f84bcda107da005f9363759a68af2008a135a84e) Scheduled build for branch: test Checking branch master ‘Jenkinsfile’ found Met criteria Takeover for example_project » master by source #1 from source that no longer exists Branch reopened: master (2a2521fe4ea2ab71edb51b3279b5e49c4c8e77d5) Scheduled build for branch: master let me know if you need more information. EDIT: Jenkins is integrated with GitLab v. 10.0.3 CE

Assignee:: Daniel Spilker

Reporter:: Anders Kielsholm

Votes:: 1 Vote for this issue

Watchers:: 19 Start watching this issue

Created:: 2017-04-19 15:17

Updated:: 2020-06-01 14:29

Resolved:: 2019-08-26 11:17

Jenkins

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Daniel Spilker added a comment - 2017-04-24 20:40, Edited by Daniel Spilker - 2017-04-24 20:40

Expand comment: Daniel Spilker added a comment - 2017-04-24 20:40, Edited by Daniel Spilker - 2017-04-24 20:40

Collapse comment: kyle rockman added a comment - 2017-04-25 13:44

Expand comment: kyle rockman added a comment - 2017-04-25 13:44

Collapse comment: SCM/JIRA link daemon added a comment - 2017-04-30 11:12

Expand comment: SCM/JIRA link daemon added a comment - 2017-04-30 11:12

Collapse comment: SCM/JIRA link daemon added a comment - 2017-04-30 11:12

Expand comment: SCM/JIRA link daemon added a comment - 2017-04-30 11:12

Collapse comment: Chance Zibolski added a comment - 2017-11-10 17:42

Expand comment: Chance Zibolski added a comment - 2017-11-10 17:42

Collapse comment: Mike Dziedziela added a comment - 2017-12-08 13:17, Edited by Mike Dziedziela - 2017-12-08 13:22

Expand comment: Mike Dziedziela added a comment - 2017-12-08 13:17, Edited by Mike Dziedziela - 2017-12-08 13:22

People

Dates