Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39107

expose "Stream Codeline" as an environment variable

      I retrieve the pipeline script from SCM. See attached screenshot for context.
      Now it would be helpful to be able to use the value of (at least) "Stream Codeline" inside my pipeline script.

      As far as I understand this is not possible so far?

        1. Stream_Codeline.png
          73 kB
          Heiko Nardmann
        2. image-2024-10-11-21-15-55-796.png
          42 kB
          Heiko Nardmann
        3. image-2024-10-11-21-15-20-655.png
          20 kB
          Heiko Nardmann
        4. image-2024-10-11-21-14-48-936.png
          24 kB
          Heiko Nardmann
        5. image-2024-10-11-21-14-31-437.png
          32 kB
          Heiko Nardmann
        6. image-2024-10-11-21-14-13-607.png
          46 kB
          Heiko Nardmann
        7. image-2024-10-11-21-13-52-223.png
          90 kB
          Heiko Nardmann
        8. image-2024-10-11-21-13-03-272.png
          64 kB
          Heiko Nardmann
        9. image-2024-10-11-21-12-06-805.png
          60 kB
          Heiko Nardmann
        10. image-2024-08-28-14-47-57-540.png
          6 kB
          Karl Wirth
        11. image-2024-03-22-12-04-43-305.png
          25 kB
          Karl Wirth
        12. image-2024-03-22-12-03-59-215.png
          31 kB
          Karl Wirth
        13. image-2024-03-22-12-03-27-381.png
          28 kB
          Karl Wirth
        14. image-2024-03-22-12-02-48-513.png
          39 kB
          Karl Wirth
        15. image-2024-03-22-12-01-18-424.png
          27 kB
          Karl Wirth
        16. image-2024-03-22-11-58-34-962.png
          24 kB
          Karl Wirth
        17. image-2024-01-11-19-06-37-867.png
          14 kB
          Heiko Nardmann
        18. image-2024-01-03-11-27-03-765.png
          12 kB
          Heiko Nardmann

          [JENKINS-39107] expose "Stream Codeline" as an environment variable

          Could someone review the following code? It seems to work fine for me:

          import org.jenkinsci.plugins.workflow.job.WorkflowJob
          import org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition
          import org.jenkinsci.plugins.p4.PerforceScm
          import org.jenkinsci.plugins.p4.workspace.StreamWorkspaceImpl
          import org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl
          
              /**
               * @brief for the current build get the stream path that is configured to access the Jenkinsfile
               *        inside the job
               *
               * @return stream path or null if it could not be determined
               */
              String getStreamForJenkinsfile() {
                  def rawBuildObj = currentBuild.getRawBuild()
                  def currentJobObj = rawBuildObj.project
                  if ( !currentJobObj ) {
                      echo "ERROR: no job object found!"
                      return null
                  }
          
                  if ( !(currentJobObj instanceof WorkflowJob) ) {
                      echo "ERROR: job is not an instance of WorkflowJob!"
                      return null
                  }
          
                  def definition = currentJobObj.getDefinition()
                  if ( !(definition instanceof CpsScmFlowDefinition) ) {
                      return null
                  }
          
                  SCM scm = definition.getScm()
                  if ( !(scm instanceof org.jenkinsci.plugins.p4.PerforceScm) ) {
                      return null
                  }
          
                  def workspace = scm.getWorkspace()
                  if ( !(workspace instanceof StreamWorkspaceImpl) && !(workspace instanceof ManualWorkspaceImpl) ) {
                      return null
                  }
          
                  def streamName
                  if ( workspace instanceof StreamWorkspaceImpl ) {
                      streamName = workspace.getStreamName()
                  } else if ( workspace instanceof ManualWorkspaceImpl ) {
                      streamName = workspace.getSpec().getStreamName()
                  } else {
                      return null
                  }
          
                  return streamName
              } // end of getStreamForJenkinsfile()
           

          Based on what I've provided in my previous post there might be a better/cleaner way to get hands on the stream name for the Jenkinsfile. But for now this workaround seems to be okay.

           

          Heiko Nardmann added a comment - Could someone review the following code? It seems to work fine for me: import org.jenkinsci.plugins.workflow.job.WorkflowJob import org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition import org.jenkinsci.plugins.p4.PerforceScm import org.jenkinsci.plugins.p4.workspace.StreamWorkspaceImpl import org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl   /**      * @brief for the current build get the stream path that is configured to access the Jenkinsfile      *        inside the job      *      * @ return stream path or null if it could not be determined      */   String getStreamForJenkinsfile() {         def rawBuildObj = currentBuild.getRawBuild()         def currentJobObj = rawBuildObj.project         if ( !currentJobObj ) {             echo "ERROR: no job object found!"             return null         }       if ( !(currentJobObj instanceof WorkflowJob) ) {             echo "ERROR: job is not an instance of WorkflowJob!"             return null         }       def definition = currentJobObj.getDefinition()         if ( !(definition instanceof CpsScmFlowDefinition) ) {             return null         }       SCM scm = definition.getScm()         if ( !(scm instanceof org.jenkinsci.plugins.p4.PerforceScm) ) {             return null         }       def workspace = scm.getWorkspace()         if ( !(workspace instanceof StreamWorkspaceImpl) && !(workspace instanceof ManualWorkspaceImpl) ) {             return null         }       def streamName         if ( workspace instanceof StreamWorkspaceImpl ) {             streamName = workspace.getStreamName()         } else if ( workspace instanceof ManualWorkspaceImpl ) {             streamName = workspace.getSpec().getStreamName()         } else {             return null         }         return streamName     } // end of getStreamForJenkinsfile() Based on what I've provided in my previous post there might be a better/cleaner way to get hands on the stream name for the Jenkinsfile. But for now this workaround seems to be okay.  

          Dhaval added a comment -

          Hi,
          heiko_nardmann While STRAM_AT_CHANGE can be accessed via variable. The code written is a good way to access STREAM name.

          Dhaval added a comment - Hi, heiko_nardmann While STRAM_AT_CHANGE can be accessed via variable. The code written is a good way to access STREAM name.

          Could you consider adding corr. code to the plugin so that my request as defined by this ticket finally gets resolved?

          Heiko Nardmann added a comment - Could you consider adding corr. code to the plugin so that my request as defined by this ticket finally gets resolved?

          Dhaval added a comment -

          heiko_nardmann  You would like to see stream name in which the jenkinsfile is present in a variable?
          If there are multiple jenkinsfile then will the expectation be to show stream name of latest jenkins file in the variable and overwrite the  earlier file stream name? 

          Dhaval added a comment - heiko_nardmann   You would like to see stream name in which the jenkinsfile is present in a variable? If there are multiple jenkinsfile then will the expectation be to show stream name of latest jenkins file in the variable and overwrite the  earlier file stream name? 

          Multiple Jenkinsfiles? How is this possible with a pipeline job? Could you provide an example? Maybe a screenshot of job configuration page?

          So far I thought that one can only specify a single Jenkinsfile. How would the pipeline know otherwise which one to use?

          Heiko Nardmann added a comment - Multiple Jenkinsfiles? How is this possible with a pipeline job? Could you provide an example? Maybe a screenshot of job configuration page? So far I thought that one can only specify a single Jenkinsfile. How would the pipeline know otherwise which one to use?

          Dhaval added a comment -

          heiko_nardmann There are multiple jenkinsfile in multi branch project.

          Dhaval added a comment - heiko_nardmann There are multiple jenkinsfile in multi branch project.

          Heiko Nardmann added a comment - - edited

          I've to admit we have never used this. But from https://help.perforce.com/helix-core/integrations-plugins/p4jenkins/current/Content/P4Jenkins/multibranch-pipeline-setup.html ...

          Jenkins will now probe each child stream associated with the Include streams path //streams/... (effectively all the streams returned by the command p4 streams //streams/...), if the probe finds a Jenkinsfile in the stream then it creates a Pipeline Job and schedules a build.

          This sounds like each build has its own single Jenkinsfile?

          To my understanding every build can only be based on a single Jenkinsfile.

          And looking at this multi branch thing: this seems to be a perfect situation where one wants to know which stream one operates on in a single build.

          Heiko Nardmann added a comment - - edited I've to admit we have never used this. But from https://help.perforce.com/helix-core/integrations-plugins/p4jenkins/current/Content/P4Jenkins/multibranch-pipeline-setup.html ... Jen kins will now probe each child stream associated with the Include streams path //streams/... (effectively all the streams returned by the command p4 streams //streams/...), if the probe finds a Jenkinsfile in the stream then it creates a Pipeline Job and schedules a buil d. This sounds like each build has its own single Jenkinsfile? To my understanding every build can only be based on a single Jenkinsfile. And looking at this multi branch thing: this seems to be a perfect situation where one wants to know which stream one operates on in a single build.

          Dhaval added a comment -

          In this situation, would you expect the variable to contain stream name of latest stream from which jenkinsfile is run?

          Dhaval added a comment - In this situation, would you expect the variable to contain stream name of latest stream from which jenkinsfile is run?

          Heiko Nardmann added a comment - - edited

          Hmmm ... your answer tells me that we don't have a common understanding of the situation. Probably me because I haven't used multi branch so far. I try to depict my understanding based on the documentation:

          1. given is a multi branch job specified to look for streams
          2. let us assume that there are e.g. three streams //streams/A, //streams/B, and //streams/C
          3. each stream has its own Jenkinsfile
            1. //streams/A/Jenkinsfile
            2. //streams/B/Jenkinsfile
            3. //streams/C/Jenkinsfile

          Now from structure to behaviour:

          1. as part of the multi branch job Jenkins probes for streams and find the streams given above
          2. three pipeline jobs are created:
            1. for //streams/A based on //streams/A/Jenkinsfile
            2. for //streams/B based on //streams/B/Jenkinsfile
            3. for //streams/C based on //streams/C/Jenkinsfile
          3. based on these jobs three builds are triggered
            1. a build A based on //streams/A/Jenkinsfile
            2. a build B based on //streams/B/Jenkinsfile
            3. a build C based on //streams/C/Jenkinsfile

          Now my assumption is that each of these builds only knows about its own Jenkinsfile. So ... with these builds running in parallel gives us

          • build A returns //streams/A/Jenkinsfile when echoing environment variable JENKINSFILE_PATH
          • build B returns //streams/B/Jenkinsfile when echoing environment variable JENKINSFILE_PATH
          • build "C" returns //streams/C/Jenkinsfile when echoing environment variable JENKINSFILE_PATH

          Same for some variable P4_STREAM (which needs to be introduced):

          • build A returns //streams/A when echoing environment variable P4_STREAM
          • build B returns //streams/B when echoing environment variable P4_STREAM
          • build C returns //streams/C when echoing environment variable P4_STREAM

          In this scenario there is nothing which I would call a "latest stream".

          Could you please check where my understanding is wrong?

          Okay, maybe ... my example code uses the configuration of a non multi branch job. I see that there might be some race condition if a user changes configuration right after a build has been triggered. I ignore this race condition as this case is far too rare for me. Maybe I have another look at APIs to see whether I can access the necessary information in a more dynamic way - via build object.

          And yes, a multi branch job always dynamically determines all of this. So accessing the (static) configuration does not make sense at all. This also needs to be handled dynamically.

          Or does

          Jenkins will now probe each child stream associated with the Include streams path //streams/... (effectively all the streams returned by the command p4 streams //streams/...), if the probe finds a Jenkinsfile in the stream then it creates a Pipeline Job and schedules a build.

          mean that really a new job is created persistently? With its own configuration? As already stated ... no experience with multi branch setups.

          Multi branch seems to make sense if you have a single component built by multiple branches/streams. In our case with a quite large application consisting out of many components we have a lot of Jenkinsfiles inside a single branch/stream which all need to be built. So ...

          • multi branch:
            component to branch: 1:n
          • my scenario:
            component to branch: n:1

          So whenever we branch from mainline/trunk we also "branch" the corresponding Jenkins pipeline jobs.

          Maybe the reason for this different approach is that we host multiple large applications with shared components inside a large mono repo/mainline?

          Heiko Nardmann added a comment - - edited Hmmm ... your answer tells me that we don't have a common understanding of the situation. Probably me because I haven't used multi branch so far. I try to depict my understanding based on the documentation: given is a multi branch job specified to look for streams let us assume that there are e.g. three streams //streams/A , //streams/B , and //streams/C each stream has its own Jenkinsfile //streams/A/Jenkinsfile //streams/B/Jenkinsfile //streams/C/Jenkinsfile Now from structure to behaviour: as part of the multi branch job Jenkins probes for streams and find the streams given above three pipeline jobs are created: for //streams/A based on //streams/A/Jenkinsfile for //streams/B based on //streams/B/Jenkinsfile for //streams/C based on //streams/C/Jenkinsfile based on these jobs three builds are triggered a build A based on //streams/A/Jenkinsfile a build B based on //streams/B/Jenkinsfile a build C based on //streams/C/Jenkinsfile Now my assumption is that each of these builds only knows about its own Jenkinsfile. So ... with these builds running in parallel gives us build A returns //streams/A/Jenkinsfile when echoing environment variable JENKINSFILE_PATH build B returns //streams/B/Jenkinsfile when echoing environment variable JENKINSFILE_PATH build "C" returns //streams/C/Jenkinsfile when echoing environment variable JENKINSFILE_PATH Same for some variable P4_STREAM (which needs to be introduced): build A returns //streams/A when echoing environment variable P4_STREAM build B returns //streams/B when echoing environment variable P4_STREAM build C returns //streams/C when echoing environment variable P4_STREAM In this scenario there is nothing which I would call a "latest stream". Could you please check where my understanding is wrong? Okay, maybe ... my example code uses the configuration of a non multi branch job. I see that there might be some race condition if a user changes configuration right after a build has been triggered. I ignore this race condition as this case is far too rare for me. Maybe I have another look at APIs to see whether I can access the necessary information in a more dynamic way - via build object. And yes, a multi branch job always dynamically determines all of this. So accessing the (static) configuration does not make sense at all. This also needs to be handled dynamically. Or does Jen kins will now probe each child stream associated with the Include streams path //streams/... (effectively all the streams returned by the command p4 streams //streams/...), if the probe finds a Jenkinsfile in the stream then it creates a Pipeline Job and schedules a buil d. mean that really a new job is created persistently? With its own configuration? As already stated ... no experience with multi branch setups. Multi branch seems to make sense if you have a single component built by multiple branches/streams. In our case with a quite large application consisting out of many components we have a lot of Jenkinsfiles inside a single branch/stream which all need to be built. So ... multi branch: component to branch: 1:n my scenario: component to branch: n:1 So whenever we branch from mainline/trunk we also "branch" the corresponding Jenkins pipeline jobs. Maybe the reason for this different approach is that we host multiple large applications with shared components inside a large mono repo/mainline?

          Dhaval added a comment -

          heiko_nardmann Apologies for the confusion. Your understadning is correct. We will add it in p4plugin 1.17.2

          Dhaval added a comment - heiko_nardmann Apologies for the confusion. Your understadning is correct. We will add it in p4plugin 1.17.2

            naveenboni Naveen Boni
            heiko_nardmann Heiko Nardmann
            Votes:
            9 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated: