Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-73732

Task Stream support with Modern SCM global libraries

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • p4-plugin
    • p4-plugin 1.16.1
      Jenkins 2.462.1

      Can use normal streams but not task streams with a "Modern SCM". 

      This is because it uses a manual workspace and really needs to use a streams workspace when using task streams.

      Reproduction steps:

      (1) Add library to a main line stream:

       

      $ p4 print //streams_libs/main/subdirectory/src/org/bar/lib.groovy
      //streams_libs/main/subdirectory/src/org/bar/lib.groovy#1 - add change 3 (text)
      package org.bar;
      def testFunc ()
      {
        echo "HELLO FROM testFunc version 3"  dispEnv()
      }
      return this;

      (2) Create a task stream called from the mainline called '//streams_libs/task'.

       

      (3) Create the following Modern SCM library definition

      (4) Create a pipeline job with the folowing groovy:

       

      @Library('task-stream-lib')
      import org.bar.lib.* 
      pipeline
       { 
      	agent any
      	stages {
       		stage("Repro") {
        		steps {
       				script {
       					def z = new org.bar.lib()
       					z.testFunc()
                                              z.dispEnv()
      				}
      			}
       		}
       	}
       } 
       

      (5) Run the job. The following error is seen:

      P4 Task: establishing connection.
      ... server: localhost:1666
      ... node: vm-kwirth-jenkins24621
      P4 Task: cleanup client: jenkins-lib-845375b0-c92a-4a45-9445-d47878b27034
      ... p4 client -o jenkins-lib-845375b0-c92a-4a45-9445-d47878b27034
       +... p4 revert /var/lib/jenkins/workspace/UseTaskStreamLibModern%40libs/3203c928762ddc75___
       +P4 Task: remove client: jenkins-lib-845375b0-c92a-4a45-9445-d47878b27034
      ... p4 client -d jenkins-lib-845375b0-c92a-4a45-9445-d47878b27034
       +ERROR: Library task-stream-lib-modern expected to contain at least one of src or vars directories
      org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
      WorkflowScript: Loading libraries failed
      
      1 error
      
      	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1107)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
      	at PluginClassLoader for script-security//org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
      	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:190)
      	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:175)
      	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:652)
      	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:598)
      	at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:335)
      	at hudson.model.ResourceController.execute(ResourceController.java:101)
      	at hudson.model.Execu 

       

       

          [JENKINS-73732] Task Stream support with Modern SCM global libraries

          Heiko Nardmann added a comment - - edited

          To automate the creation of a library entry - in case a branch of a library is needed - inside Jenkins I'm currently working on sth like

           

          // the following two need to be arguments to some script
          def libraryToLookFor = args[0]
          def agileJiraTicketNo = args[1]
          
          import org.jenkinsci.plugins.workflow.libs.GlobalLibraries
          
          def globalLibraries = GlobalLibraries.get()
          def existingLibraries = globalLibraries.getLibraries()
          
          import org.jenkinsci.plugins.workflow.libs.LibraryConfiguration
          
          def libraryConfigurationToClone
          for (LibraryConfiguration eachLib : existingLibraries) {
              if ( eachLib.getName() == libraryToLookFor ) {
                  libraryConfigurationToClone = eachLib
                  break
              }
          }
               
          import org.jenkinsci.plugins.p4.PerforceScm
          import org.jenkinsci.plugins.p4.populate.ForceCleanImpl
          import org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl
          import org.jenkinsci.plugins.p4.workspace.WorkspaceSpec
          import java.text.SimpleDateFormat
          
          def date = new Date()
          def sdf = new SimpleDateFormat("yyyy/MMdd")
          //println sdf.format(date)
          
          // the following creates a pipeline library based on LegacySCM as ModernSCM does not work with task streams;
          // see JENKINS-73732
          
          def taskStreamsName = "//task_streams_" + sdf.format(date) + "_" + libraryToLookFor + "." + agileJiraTicketNo
          
          def p4CredentialId = "fri4proxy01_site_proxy"
          def p4WorkspaceSpec = new WorkspaceSpec(/*allwrite=*/ false,
                                                  /*clobber=*/ true,
                                                  /*compress=*/ false,
                                                  /*locked=*/ false,
                                                  /*modtime=*/ false,
                                                  /*rmdir=*/ true,
                                                  /*streamName=*/ taskStreamsName,
                                                  /*line=*/ "LOCAL",
                                                  /*view=*/ null,
                                                  /*changeView=*/ null,
                                                  /*type=*/ "READONLY",
                                                  /*serverID=*/ null,
                                                  /*backup=*/ false)
          def p4Workspace = new ManualWorkspaceImpl(/*charset=*/ null, /*pinHost=*/ false, /*name=*/ "jenkins-\${NODE_NAME}-\${JOB_NAME}-\${EXECUTOR_NUMBER}-library", /*spec=*/ p4WorkspaceSpec, /*cleanup=*/ false)
          def p4Populate = new ForceCleanImpl(/*have=*/ false,
                                              /*quiet=*/ false,
                                              /*pin=*/ null,
                                              /*parallel=*/ null)
          def p4Browser = null
          def p4Filter = []
          def p4Scm = new PerforceScm(p4CredentialId, p4Workspace, p4Filter, p4Populate, p4Browser)
          //println("p4Scm=" + p4Scm)
          
          import org.jenkinsci.plugins.workflow.libs.SCMRetriever
          
          def scmRetriever = new SCMRetriever(p4Scm)
          scmRetriever.setLibraryPath("./")
          //println("scmRetriever=" + scmRetriever)
          
          def nameOfNewLibrary = libToLookFor + "." + agileJiraTicketNo
          
          LibraryConfiguration config = new LibraryConfiguration(nameOfNewLibrary, scmRetriever);
          config.setImplicit(false);
          config.setDefaultVersion("now");
          
          GlobalLibraries globalLibs = (GlobalLibraries) Jenkins.getInstance().getDescriptor(GlobalLibraries.class);
          
          def oldLibraries = globalLibs.getLibraries()
          def newLibraries = oldLibraries + config
          
          globalLibs.setLibraries(newLibraries)

           

           

          Maybe this is helpful for someone else ...

          Heiko Nardmann added a comment - - edited To automate the creation of a library entry - in case a branch of a library is needed - inside Jenkins I'm currently working on sth like   // the following two need to be arguments to some script def libraryToLookFor = args[0] def agileJiraTicketNo = args[1] import org.jenkinsci.plugins.workflow.libs.GlobalLibraries def globalLibraries = GlobalLibraries.get() def existingLibraries = globalLibraries.getLibraries() import org.jenkinsci.plugins.workflow.libs.LibraryConfiguration def libraryConfigurationToClone for (LibraryConfiguration eachLib : existingLibraries) {     if ( eachLib.getName() == libraryToLookFor ) {         libraryConfigurationToClone = eachLib         break     } }       import org.jenkinsci.plugins.p4.PerforceScm import org.jenkinsci.plugins.p4.populate.ForceCleanImpl import org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl import org.jenkinsci.plugins.p4.workspace.WorkspaceSpec import java.text.SimpleDateFormat def date = new Date() def sdf = new SimpleDateFormat("yyyy/MMdd") //println sdf.format(date) // the following creates a pipeline library based on LegacySCM as ModernSCM does not work with task streams; // see JENKINS-73732 def taskStreamsName = "//task_streams_" + sdf.format(date) + "_" + libraryToLookFor + "." + agileJiraTicketNo def p4CredentialId = "fri4proxy01_site_proxy" def p4WorkspaceSpec = new WorkspaceSpec(/*allwrite=*/ false,                                         /*clobber=*/ true,                                         /*compress=*/ false,                                         /*locked=*/ false,                                         /*modtime=*/ false,                                         /*rmdir=*/ true,                                         /*streamName=*/ taskStreamsName,                                         /*line=*/ "LOCAL",                                         /*view=*/ null,                                         /*changeView=*/ null,                                         /*type=*/ "READONLY",                                         /*serverID=*/ null,                                         /*backup=*/ false) def p4Workspace = new ManualWorkspaceImpl(/*charset=*/ null, /*pinHost=*/ false, /*name=*/ "jenkins-\${NODE_NAME}-\${JOB_NAME}-\${EXECUTOR_NUMBER}-library", /*spec=*/ p4WorkspaceSpec, /*cleanup=*/ false) def p4Populate = new ForceCleanImpl(/*have=*/ false,                                     /*quiet=*/ false,                                     /*pin=*/ null,                                     /*parallel=*/ null) def p4Browser = null def p4Filter = [] def p4Scm = new PerforceScm(p4CredentialId, p4Workspace, p4Filter, p4Populate, p4Browser) //println("p4Scm=" + p4Scm) import org.jenkinsci.plugins.workflow.libs.SCMRetriever def scmRetriever = new SCMRetriever(p4Scm) scmRetriever.setLibraryPath("./") //println("scmRetriever=" + scmRetriever) def nameOfNewLibrary = libToLookFor + "." + agileJiraTicketNo LibraryConfiguration config = new LibraryConfiguration(nameOfNewLibrary, scmRetriever); config.setImplicit(false); config.setDefaultVersion("now"); GlobalLibraries globalLibs = (GlobalLibraries) Jenkins.getInstance().getDescriptor(GlobalLibraries.class); def oldLibraries = globalLibs.getLibraries() def newLibraries = oldLibraries + config globalLibs.setLibraries(newLibraries)     Maybe this is helpful for someone else ...

            Unassigned Unassigned
            p4karl Karl Wirth
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: