• Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • p4-plugin
    • None

      When scanning multi-branch projects in Jenkins, sparse streams are not detected when the Jenkinsfile was not changed within this stream.

      'Jenkinsfile' not found

      After changing the Jenkinsfile in the sparse stream, the branch is recognized by the multi-branch scan but only changed files are checked out during the build

          [JENKINS-73981] Support perforce/helix sparse streams

          Dhaval added a comment - - edited

          Hi,
          In sparse streams files are not synced by default. Only meta data is available. Please sync JenkinsFile manually and then try, it should work.
          In 2024.2 we have also introduced the feature where some files in sparse streams can be downloaded by default. This should be available in later weeks of Nov 2024.

          P4 plugin does checkout of files present in local path. If files are not present in local path (only meta data is present as in case of sparse streams), they will not be checkout.

          p4karl 

          Dhaval added a comment - - edited Hi, In sparse streams files are not synced by default. Only meta data is available. Please sync JenkinsFile manually and then try, it should work. In 2024.2 we have also introduced the feature where some files in sparse streams can be downloaded by default. This should be available in later weeks of Nov 2024. P4 plugin does checkout of files present in local path. If files are not present in local path (only meta data is present as in case of sparse streams), they will not be checkout. p4karl  

          Daniel H added a comment - - edited

          What do you mean with "sync Jenkinsfile manually"?

          I use multibranch-projects and what this plugin does is scanning all branches/streams for a Jenkinsfile and if found list the stream as a job in Jenkins.

          For sparse streams, the stream is only found if the Jenkinsfile was changed at the sparse stream what is the first bug/issue.

           

          Second issue is, that when a sparse stream was recognized as as a job in jenkins, I just call "checkout scm" in my Jenkinsfiles, but only files which were changed at the sparse stream are checked out. There should be no special handling needed for sparse streams.

           

          Daniel H added a comment - - edited What do you mean with "sync Jenkinsfile manually"? I use multibranch-projects and what this plugin does is scanning all branches/streams for a Jenkinsfile and if found list the stream as a job in Jenkins. For sparse streams, the stream is only found if the Jenkinsfile was changed at the sparse stream what is the first bug/issue.   Second issue is, that when a sparse stream was recognized as as a job in jenkins, I just call "checkout scm" in my Jenkinsfiles, but only files which were changed at the sparse stream are checked out. There should be no special handling needed for sparse streams.  

          Dhaval added a comment -

          We will look into it. Meanwhile please do an additional sync before starting the build while using sparse streams.

          Dhaval added a comment - We will look into it. Meanwhile please do an additional sync before starting the build while using sparse streams.

          Daniel H added a comment -

          patdhaval30 what do you mean with "do an additional sync"?

          I have no influence on how the multi-branch plugin scans for Jenkinsfiles.

           

          During the build, I already do a "checkout scm" command which should sync the code from the stream at the build-note, but this is not working with sparse streams

          Daniel H added a comment - patdhaval30 what do you mean with "do an additional sync"? I have no influence on how the multi-branch plugin scans for Jenkinsfiles.   During the build, I already do a "checkout scm" command which should sync the code from the stream at the build-note, but this is not working with sparse streams

          Dhaval added a comment -

          danielh  We are analysing this issue and will get back you.

          Dhaval added a comment - danielh   We are analysing this issue and will get back you.

          Dhaval added a comment - - edited

          danielh We are not able to reproduce the issue.
          Can you help us understand this previous comment with more details?
          "For sparse streams, the stream is only found if the Jenkinsfile was changed at the sparse stream what is the first bug/issue."

          We would like to know what you mean by "jenkins file was changed at sparse stream".

          Dhaval added a comment - - edited danielh We are not able to reproduce the issue. Can you help us understand this previous comment with more details? "For sparse streams, the stream is only found if the Jenkinsfile was changed at the sparse stream what is the first bug/issue." We would like to know what you mean by "jenkins file was changed at sparse stream".

          Daniel H added a comment -

          patdhaval30 these are the steps to reproduce:

          1. Create a mainline Stream (no sparse) //Depot/myStream
          2. Create anew MultiBranch-Pipeline job in Jenkins
            1. Use Helix Streams as source
            2. Include Streams: //Depot/...
            3. Mode: By Jenkinsfile
            4. Script Path: Jenkinsfile
          3. Upload a "Jenkinsfile" into Depot/myStream/Jenkinsfile
          4. Use Jenkins to scan the MultiBranch pipelines: -> myStream found as project
          5. Create a development sparse stream from myStream
          6. Use Jenkins to scan the MultiBranch pipelines: -> Sparse stream NOT found as additional job in jenkins
          7. Change the Jenkinsfile in the spase stream (i.e. just add an empty line)
          8. Use Jenkins to scan the MultiBranch pipelines: -> Sparse stream found as additional job in jenkins

          Daniel H added a comment - patdhaval30 these are the steps to reproduce: Create a mainline Stream (no sparse) //Depot/myStream Create anew MultiBranch-Pipeline job in Jenkins Use Helix Streams as source Include Streams: //Depot/... Mode: By Jenkinsfile Script Path: Jenkinsfile Upload a "Jenkinsfile" into Depot/myStream/Jenkinsfile Use Jenkins to scan the MultiBranch pipelines: -> myStream found as project Create a development sparse stream from myStream Use Jenkins to scan the MultiBranch pipelines: -> Sparse stream NOT found as additional job in jenkins Change the Jenkinsfile in the spase stream (i.e. just add an empty line) Use Jenkins to scan the MultiBranch pipelines: -> Sparse stream found as additional job in jenkins

          Dhaval added a comment - - edited

          danielh I am able to find JenkinsFile as well as sparse streams as a project.
          Can you share your jenkins pipeline logs, from jobs folder please share config.xml and build.xml so that we can understand your problem better?
          (In the below images please see the results I got while reproducing your issue.)

          In the image spaseJenkins is a sparse stream created from streams/main.


          Dhaval added a comment - - edited danielh I am able to find JenkinsFile as well as sparse streams as a project. Can you share your jenkins pipeline logs, from jobs folder please share config.xml and build.xml so that we can understand your problem better? (In the below images please see the results I got while reproducing your issue.) In the image spaseJenkins is a sparse stream created from streams/main.

            Unassigned Unassigned
            danielh Daniel H
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: