Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39107

expose "Stream Codeline" as an environment variable

      I retrieve the pipeline script from SCM. See attached screenshot for context.
      Now it would be helpful to be able to use the value of (at least) "Stream Codeline" inside my pipeline script.

      As far as I understand this is not possible so far?

          [JENKINS-39107] expose "Stream Codeline" as an environment variable

          I just want to - by default - use the same stream for later on synching the source code as the one that the Jenkinsfile originates from.

          In my case I even analyze the stream spec to avoid synching of binary assets which are not relevant for the current target platform. So yes, I create a manual workspace like

                      ws = [$class: 'ManualWorkspaceImpl', 
                            charset: 'none',
                            cleanup: false,
                            name: clientSpecName,
                            pinHost: false,
                            spec: clientSpec(
                                allwrite: false,
                                backup: false,
                                changeView: changeView,
                                clobber: true,
                                compress: false,
                                line: 'LOCAL',
                                locked: false,
                                modtime: false,
                                rmdir: true,
                                serverID: '',
                                streamName: streamNameAttribute,
                                type: workspaceType,
                                view: manualView)]
                      def scm = perforce(
                                            browser: swarm('<some Helix Swarm url>'),
                                            credential: p4CredentialId,
                                            filter: [latestWithPin(true)],
                                            populate: forceClean(
                                                have: true,
                                                parallel: [enable: true, threads: '4'],
                                                pin: changelistToBuildWith,
                                                quiet: true
                                            ),
                                            workspace: ws
                                        )
                      scmVars = checkout changelog: true,
                                         scm: scm
           

          in my pipeline code. So far I need to explicitly define the streamName inside my Jenkinsfiles which is in most cases redundant.

          Heiko Nardmann added a comment - I just want to - by default - use the same stream for later on synching the source code as the one that the Jenkinsfile originates from. In my case I even analyze the stream spec to avoid synching of binary assets which are not relevant for the current target platform. So yes, I create a manual workspace like           ws = [$class: 'ManualWorkspaceImpl' ,                    charset: 'none' ,                   cleanup: false ,                   name: clientSpecName,                   pinHost: false ,                   spec: clientSpec(                       allwrite: false ,                       backup: false ,                       changeView: changeView,                       clobber: true ,                       compress: false ,                       line: 'LOCAL' ,                       locked: false ,                       modtime: false ,                       rmdir: true ,                       serverID: '',                       streamName: streamNameAttribute,                       type: workspaceType,                       view: manualView)]             def scm = perforce(                                   browser: swarm( '<some Helix Swarm url>' ),                                   credential: p4CredentialId,                                   filter: [latestWithPin( true )],                                   populate: forceClean(                                       have: true ,                                       parallel: [enable: true , threads: '4' ],                                       pin: changelistToBuildWith,                                       quiet: true                                   ),                                   workspace: ws                               )             scmVars = checkout changelog: true ,                                scm: scm in my pipeline code. So far I need to explicitly define the streamName inside my Jenkinsfiles which is in most cases redundant.

          By today I've updated the P4 plugin to the new 1.17.0.

          Bad luck ... I don't see any P4_STREAM_AT_CHANGE environment variable. My code inside Jenkinsfile looks like

          @Library('Nav-CI-Standard') _
          
          echo "Outside pipeline in Jenkinsfile:"
          
          echo "Variable is:"
          echo "env.JENKINSFILE_PATH=${env.JENKINSFILE_PATH}"
          echo "env=${env.getEnvironment()}"
          
          import com.stryker.BuildConfiguration
          
          ... some additional code ...
           

          As part of the third echo I have expected to see some P4_STREAM_AT_CHANGE environment variable. Missing ...

          Heiko Nardmann added a comment - By today I've updated the P4 plugin to the new 1.17.0. Bad luck ... I don't see any P4_STREAM_AT_CHANGE environment variable. My code inside Jenkinsfile looks like @Library( 'Nav-CI-Standard' ) _ echo "Outside pipeline in Jenkinsfile:" echo "Variable is:" echo "env.JENKINSFILE_PATH=${env.JENKINSFILE_PATH}" echo "env=${env.getEnvironment()}" import com.stryker.BuildConfiguration ... some additional code ... As part of the third echo I have expected to see some P4_STREAM_AT_CHANGE environment variable. Missing ...

          What worries me even more: after doing a git pull and then

          grep -rw P4_STREAM_AT_CHANGE

          I don't see any reference to P4_STREAM_AT_CHANGE beside the implementation inside

          ./src/main/java/org/jenkinsci/plugins/p4/build/P4StreamEnvironmentContributionAction.java

          ? And of course inside the RELEASE.md ...

          But ... where do I find the corr. unit test?

          Heiko Nardmann added a comment - What worries me even more: after doing a git pull and then grep -rw P4_STREAM_AT_CHANGE I don't see any reference to P4_STREAM_AT_CHANGE beside the implementation inside ./src/main/java/org/jenkinsci/plugins/p4/build/P4StreamEnvironmentContributionAction.java ? And of course inside the RELEASE.md ... But ... where do I find the corr. unit test?

          Dhaval added a comment -

          Stream_At_Change is available inside the pipeline tag similar to other variables like P4_PORT. This is done as these variables are populated only after check-out and sync step. To access these variables for further use 

          pipeline {

          agent any stages {

          stage("Repro") {

          steps {

          script {

          echo "Inside pipeline in Jenkinsfile:"

          echo "Variable is:"

          echo "env.JENKINSFILE_PATH=${env.JENKINSFILE_PATH}" }

          } } } }

          To see how P4_STREAM_AT_CHANGE is getting updated, please refer following:
          Line number 584 in PerforceSCM.java is used to set this variable. Please look into this method of PerforceSCM.java.
          private void setStreamEnvVariables(Run<?, ?> run, Workspace ws)

          This function was tested with our system test suite. We can add a unit test case in next release from our test suite.

          Dhaval added a comment - Stream_At_Change is available inside the pipeline tag similar to other variables like P4_PORT. This is done as these variables are populated only after check-out and sync step. To access these variables for further use  pipeline { agent any stages { stage("Repro") { steps { script { echo "Inside pipeline in Jenkinsfile:" echo "Variable is:" echo "env.JENKINSFILE_PATH=${env.JENKINSFILE_PATH}" } } } } } To see how P4_STREAM_AT_CHANGE is getting updated, please refer following: Line number 584 in PerforceSCM.java is used to set this variable. Please look into this method of PerforceSCM.java. private void setStreamEnvVariables(Run<?, ?> run, Workspace ws) This function was tested with our system test suite. We can add a unit test case in next release from our test suite.

          Heiko Nardmann added a comment - - edited

          Pity ... I've expected that the checkout of the library would set this now as well.

          Can I maybe perform some kind of dummy checkout to reach my goal? As a workaround until this problem is really solved?

          Okay, probably not a good idea ... this might confuse Jenkins wrt. determining changes even more.

          Yes, please add the corr. unit tests soon. System tests typically miss a number of corner cases.

          Heiko Nardmann added a comment - - edited Pity ... I've expected that the checkout of the library would set this now as well. Can I maybe perform some kind of dummy checkout to reach my goal? As a workaround until this problem is really solved? Okay, probably not a good idea ... this might confuse Jenkins wrt. determining changes even more. Yes, please add the corr. unit tests soon. System tests typically miss a number of corner cases.

          Dhaval added a comment -

          Fixed in 1.17.0

          Dhaval added a comment - Fixed in 1.17.0

          Hi patdhaval30 ,

          based on what Karl has reproduced - see https://issues.jenkins.io/browse/JENKINS-39107?focusedId=446317&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-446317 - I'm not sure whether I would call this problem fixed now.

          It might be that things have been approved in a number of usecases; but for my usecase - the reason for creating this issue - the problem still exists.

          From time to time I also wonder whether I should try to fix this on my own but for that the missing unit tests need to be provided so that coverage is improved when doing local tests. Which is another reason why the ticket should not be considered to be fixed.

          Okay, maybe the philosophy among developers varies here: at least in our companys teams a feature is considered to be implemented if at least unit tests are available and the corr. code coverage is sufficient. Automated system tests is of course a different thing ...

          Do you agree with me that we should reopen this ticket?

           

          Heiko Nardmann added a comment - Hi patdhaval30 , based on what Karl has reproduced - see https://issues.jenkins.io/browse/JENKINS-39107?focusedId=446317&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-446317 - I'm not sure whether I would call this problem fixed now. It might be that things have been approved in a number of usecases; but for my usecase - the reason for creating this issue - the problem still exists. From time to time I also wonder whether I should try to fix this on my own but for that the missing unit tests need to be provided so that coverage is improved when doing local tests. Which is another reason why the ticket should not be considered to be fixed. Okay, maybe the philosophy among developers varies here: at least in our companys teams a feature is considered to be implemented if at least unit tests are available and the corr. code coverage is sufficient. Automated system tests is of course a different thing ... Do you agree with me that we should reopen this ticket?  

          Dhaval added a comment -

          Hi heiko_nardmann ,

          Agreed, we have internally created a new ticket to add test cases. We will close this issue when test cases are added.

          Thanks.

          Dhaval added a comment - Hi heiko_nardmann , Agreed, we have internally created a new ticket to add test cases. We will close this issue when test cases are added. Thanks.

          Could someone review the following code? It seems to work fine for me:

          import org.jenkinsci.plugins.workflow.job.WorkflowJob
          import org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition
          import org.jenkinsci.plugins.p4.PerforceScm
          import org.jenkinsci.plugins.p4.workspace.StreamWorkspaceImpl
          import org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl
          
              /**
               * @brief for the current build get the stream path that is configured to access the Jenkinsfile
               *        inside the job
               *
               * @return stream path or null if it could not be determined
               */
              String getStreamForJenkinsfile() {
                  def rawBuildObj = currentBuild.getRawBuild()
                  def currentJobObj = rawBuildObj.project
                  if ( !currentJobObj ) {
                      echo "ERROR: no job object found!"
                      return null
                  }
          
                  if ( !(currentJobObj instanceof WorkflowJob) ) {
                      echo "ERROR: job is not an instance of WorkflowJob!"
                      return null
                  }
          
                  def definition = currentJobObj.getDefinition()
                  if ( !(definition instanceof CpsScmFlowDefinition) ) {
                      return null
                  }
          
                  SCM scm = definition.getScm()
                  if ( !(scm instanceof org.jenkinsci.plugins.p4.PerforceScm) ) {
                      return null
                  }
          
                  def workspace = scm.getWorkspace()
                  if ( !(workspace instanceof StreamWorkspaceImpl) && !(workspace instanceof ManualWorkspaceImpl) ) {
                      return null
                  }
          
                  def streamName
                  if ( workspace instanceof StreamWorkspaceImpl ) {
                      streamName = workspace.getStreamName()
                  } else if ( workspace instanceof ManualWorkspaceImpl ) {
                      streamName = workspace.getSpec().getStreamName()
                  } else {
                      return null
                  }
          
                  return streamName
              } // end of getStreamForJenkinsfile()
           

          Based on what I've provided in my previous post there might be a better/cleaner way to get hands on the stream name for the Jenkinsfile. But for now this workaround seems to be okay.

           

          Heiko Nardmann added a comment - Could someone review the following code? It seems to work fine for me: import org.jenkinsci.plugins.workflow.job.WorkflowJob import org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition import org.jenkinsci.plugins.p4.PerforceScm import org.jenkinsci.plugins.p4.workspace.StreamWorkspaceImpl import org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl   /**      * @brief for the current build get the stream path that is configured to access the Jenkinsfile      *        inside the job      *      * @ return stream path or null if it could not be determined      */   String getStreamForJenkinsfile() {         def rawBuildObj = currentBuild.getRawBuild()         def currentJobObj = rawBuildObj.project         if ( !currentJobObj ) {             echo "ERROR: no job object found!"             return null         }       if ( !(currentJobObj instanceof WorkflowJob) ) {             echo "ERROR: job is not an instance of WorkflowJob!"             return null         }       def definition = currentJobObj.getDefinition()         if ( !(definition instanceof CpsScmFlowDefinition) ) {             return null         }       SCM scm = definition.getScm()         if ( !(scm instanceof org.jenkinsci.plugins.p4.PerforceScm) ) {             return null         }       def workspace = scm.getWorkspace()         if ( !(workspace instanceof StreamWorkspaceImpl) && !(workspace instanceof ManualWorkspaceImpl) ) {             return null         }       def streamName         if ( workspace instanceof StreamWorkspaceImpl ) {             streamName = workspace.getStreamName()         } else if ( workspace instanceof ManualWorkspaceImpl ) {             streamName = workspace.getSpec().getStreamName()         } else {             return null         }         return streamName     } // end of getStreamForJenkinsfile() Based on what I've provided in my previous post there might be a better/cleaner way to get hands on the stream name for the Jenkinsfile. But for now this workaround seems to be okay.  

          Dhaval added a comment -

          Hi,
          heiko_nardmann While STRAM_AT_CHANGE can be accessed via variable. The code written is a good way to access STREAM name.

          Dhaval added a comment - Hi, heiko_nardmann While STRAM_AT_CHANGE can be accessed via variable. The code written is a good way to access STREAM name.

            naveenboni Naveen Boni
            heiko_nardmann Heiko Nardmann
            Votes:
            9 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated: