Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26101

Load entire script from SCM

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: pipeline
    • Labels:
      None
    • Similar Issues:

      Description

      Allow the workflow to (visually) configure an SCM, check it out into a workspace on the Jenkins master, and load a sandboxed Groovy script from a defined location in that workspace.

      Perhaps allow the workspace to be treated as a sourcepath so libraries can be imported.

      Optionally include the flow SCM in polling and/or changelog generation.

      Abandoned ideas:

      1. Construct URL with an instance of URLStreamHandler we can create a URL space that can access files over FilePath, and somehow prevent the class loader from continuing to use the workspace after it goes out of scope.
      2. Similar but the URLStreamHandler uses SCMSource to access files without ever checking out anything.

        Attachments

          Issue Links

            Activity

            jglick Jesse Glick created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-26129 [ JENKINS-26129 ]
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Description h3. Adding classpath from workspace?
            By constructing {{URL}} with an instance of {{URLStreamHandler}} we can create a URL space that can access files over {{FilePath}}. This would enable primitives like this:

            {code}
            node {
              git url:'my-shared-script.git', credentialId: '1234-5678-deadbeefdeadbeef'
              
              addClassPath "."

              ...
            }
            {code}

            There's some details that need to be figured out how to prevent classloader from continuing to use workspace after the workspace goes out of scope.


            h3. "Add classpath from SCMSource" primitive
            The idea is similar here except the custom {{URLStreamHandler}} uses {{SCMSource}} to access files without ever checking out anything.

            {code}
            library type:'git', remote:'my-shared-script.git', credentialId: '1234-5678-deadbeefdeadbeef'
              
            ...
            {code}


            h3. Primitive that uses {{SCM}} to checkout stuff into the master
            Instead of messing around with {{URLStreamHandler}}, just use ordinary {{SCM}} and checkout code into some hidden locations on the master and adds that local directory as classpath.

            Needs to be seen if {{SCM}} checkout have any unwanted side-effect to the project or to the master.

            h3. Security Considerations
            In all the cases, do not allow class files, jar files to be added directly to the classpath as it'd bypass the sandbox.
            Allow the workflow to (visually) configure an {{SCM}}, check it out into a workspace on the Jenkins master, and load a sandboxed Groovy script from a defined location in that workspace.

            _Perhaps_ allow the workspace to be treated as a sourcepath so libraries can be imported.

            Optionally include the flow SCM in polling and/or changelog generation.

            Abandoned ideas:

            # Construct {{URL}} with an instance of {{URLStreamHandler}} we can create a URL space that can access files over {{FilePath}}, and somehow prevent the class loader from continuing to use the workspace after it goes out of scope.
            # Similar but the {{URLStreamHandler}} uses {{SCMSource}} to access files without ever checking out anything.
            Summary Load Script from SCM (wave #2) Load entire script from SCM
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction2.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/config.jelly
            http://jenkins-ci.org/commit/workflow-plugin/50be69d7db0fff857f044bf725bcb0f1d8d0e7a2
            Log:
            JENKINS-26101 Proof of concept of flow definition loaded entirely from a local SCM checkout.

            Compare: https://github.com/jenkinsci/workflow-plugin/compare/0d2226288236^...50be69d7db0f

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction2.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/config.jelly http://jenkins-ci.org/commit/workflow-plugin/50be69d7db0fff857f044bf725bcb0f1d8d0e7a2 Log: JENKINS-26101 Proof of concept of flow definition loaded entirely from a local SCM checkout. Compare: https://github.com/jenkinsci/workflow-plugin/compare/0d2226288236 ^...50be69d7db0f
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 40 (Web Link)" [ 12030 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            flow.groovy
            http://jenkins-ci.org/commit/workflow-plugin-pipeline-demo/485c0ec6837c82795b0d438809ba286718c7784d
            Log:
            JENKINS-26101 Should be able to load complete script from SCM now.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: flow.groovy http://jenkins-ci.org/commit/workflow-plugin-pipeline-demo/485c0ec6837c82795b0d438809ba286718c7784d Log: JENKINS-26101 Should be able to load complete script from SCM now.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            CHANGES.md
            TUTORIAL.md
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/CpsScmFlowDefinitionTest.java
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/steps/scm/SubversionStepTest.java
            api/src/main/java/org/jenkinsci/plugins/workflow/flow/FlowDefinition.java
            basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/PushdStep.java
            basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ReadFileStep.java
            basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/RelativePathValidator.java
            basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/WriteFileStep.java
            basic-steps/src/test/java/org/jenkinsci/plugins/workflow/steps/RelativePathValidatorTest.java
            cps/pom.xml
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction2.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/FlowHead.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/Snippetizer.java
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition/config.jelly
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition/help-snippetizer.html
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/config.jelly
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/help-scm.html
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/help-scriptPath.html
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/Snippetizer/block.jelly
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/Snippetizer/help.html
            cps/src/test/groovy/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinitionTest.groovy
            cps/src/test/java/org/jenkinsci/plugins/workflow/cps/SnippetizerTest.java
            demo/Dockerfile
            demo/JENKINS_HOME/jobs/cd/config.xml
            demo/JENKINS_HOME/scriptApproval.xml
            demo/README.md
            job/src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java
            scm-step/src/main/java/org/jenkinsci/plugins/workflow/steps/scm/GenericSCMStep.java
            scm-step/src/main/java/org/jenkinsci/plugins/workflow/steps/scm/GitStep.java
            scm-step/src/main/java/org/jenkinsci/plugins/workflow/steps/scm/SCMStep.java
            http://jenkins-ci.org/commit/workflow-plugin/f1e8e16057819582ba376226a1af47e45b34798d
            Log:
            Merge pull request #40 from jenkinsci/script-from-scm-JENKINS-26101

            [FIXED JENKINS-26101] Load script from SCM

            Compare: https://github.com/jenkinsci/workflow-plugin/compare/e2947e92a2d0...f1e8e1605781

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: CHANGES.md TUTORIAL.md aggregator/src/test/java/org/jenkinsci/plugins/workflow/CpsScmFlowDefinitionTest.java aggregator/src/test/java/org/jenkinsci/plugins/workflow/steps/scm/SubversionStepTest.java api/src/main/java/org/jenkinsci/plugins/workflow/flow/FlowDefinition.java basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/PushdStep.java basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ReadFileStep.java basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/RelativePathValidator.java basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/WriteFileStep.java basic-steps/src/test/java/org/jenkinsci/plugins/workflow/steps/RelativePathValidatorTest.java cps/pom.xml cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction2.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/FlowHead.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/Snippetizer.java cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition/config.jelly cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition/help-snippetizer.html cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/config.jelly cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/help-scm.html cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/help-scriptPath.html cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/Snippetizer/block.jelly cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/Snippetizer/help.html cps/src/test/groovy/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinitionTest.groovy cps/src/test/java/org/jenkinsci/plugins/workflow/cps/SnippetizerTest.java demo/Dockerfile demo/JENKINS_HOME/jobs/cd/config.xml demo/JENKINS_HOME/scriptApproval.xml demo/README.md job/src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java scm-step/src/main/java/org/jenkinsci/plugins/workflow/steps/scm/GenericSCMStep.java scm-step/src/main/java/org/jenkinsci/plugins/workflow/steps/scm/GitStep.java scm-step/src/main/java/org/jenkinsci/plugins/workflow/steps/scm/SCMStep.java http://jenkins-ci.org/commit/workflow-plugin/f1e8e16057819582ba376226a1af47e45b34798d Log: Merge pull request #40 from jenkinsci/script-from-scm- JENKINS-26101 [FIXED JENKINS-26101] Load script from SCM Compare: https://github.com/jenkinsci/workflow-plugin/compare/e2947e92a2d0...f1e8e1605781
            jglick Jesse Glick made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction2.java
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java
            cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/config.jelly
            http://jenkins-ci.org/commit/workflow-cps-plugin/c4204850101b4f232ea6c164f66f6ec36cc076a4
            Log:
            JENKINS-26101 Proof of concept of flow definition loaded entirely from a local SCM checkout.
            Originally-Committed-As: 50be69d7db0fff857f044bf725bcb0f1d8d0e7a2

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowDefinition.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowFactoryAction2.java cps/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java cps/src/main/resources/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition/config.jelly http://jenkins-ci.org/commit/workflow-cps-plugin/c4204850101b4f232ea6c164f66f6ec36cc076a4 Log: JENKINS-26101 Proof of concept of flow definition loaded entirely from a local SCM checkout. Originally-Committed-As: 50be69d7db0fff857f044bf725bcb0f1d8d0e7a2
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 160091 ] JNJira + In-Review [ 196313 ]
            abayer Andrew Bayer made changes -
            Component/s pipeline-general [ 21692 ]
            abayer Andrew Bayer made changes -
            Component/s workflow-plugin [ 18820 ]

              People

              Assignee:
              jglick Jesse Glick
              Reporter:
              jglick Jesse Glick
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: