Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37573

Create way for pipeline graph analysis to return full nested structures

      As a consumer of pipeline graph analysis it would be helpful to have a way to get the full block chunk structure (with nesting). This is useful for cases where we need to find the properties of a FlowNode based on block contents (example its label, or the node used to run it).

      In cases where we need scope, we may have nested blocks with following nodes belonging to the enclosing parent block.

      This needs:

      • A SimpleChunkVisitor implementation using an ArrayDeque to store the context (open and nested chunks). If finished with a current chunk (popping out of the scope & removing from ArrayDeque), it is added as a nested chunk in the parent scope, when hitting the ChunkStart
      • A NestableMemoryFlowChunk that can have other chunks added within it or contain lists of nodes nodes.
      • NestableMemoryFlowChunk needs something like "addChunk(NestableMemoryFlowChunk)" and "addNode(FlowNode)".
      • Two container types, RecursiveNestableMemoryFlowChunk has a list of FlowChunk, ContainerNestableMemoryFlowChunk has an ArrayList of flownodes, and is converted to recursive when you add the first nested chunk
      • I've described this as the DOMLike model – this is because you might provide tree-traversal APIs
      • AdvancedVisitor is one way to collect these data models (and couple to datastructures for custom storage). it would invoke a factory to create chunks implementing these interfaces and store data.

          [JENKINS-37573] Create way for pipeline graph analysis to return full nested structures

          Andrew Bayer added a comment -

          svanoort - this is related to the stuff you're working on for my junit work, right?

          Andrew Bayer added a comment - svanoort - this is related to the stuff you're working on for my junit work, right?

          Sam Van Oort added a comment -

          abayer kinda – I'll link it in the PR but it's intended to be more full-featured.

          Sam Van Oort added a comment - abayer kinda – I'll link it in the PR but it's intended to be more full-featured.

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/4ba6b42b651d9a5853a32681e9582f9649ad8fa2
          Log:
          Merge pull request #50 from svanoort/jenkins-27395-block-structure-lookup

          JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph

          Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/63e8ad0c2715...4ba6b42b651d

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-api-plugin/4ba6b42b651d9a5853a32681e9582f9649ad8fa2 Log: Merge pull request #50 from svanoort/jenkins-27395-block-structure-lookup JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/63e8ad0c2715...4ba6b42b651d

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/70a05dc4d4a330b797ce370fd587fccfe74355d9
          Log:
          Revert "Revert "JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph""

          This reverts commit 88ffdfc69c43bd4dde21a6578b5ac466999b4fd4.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-api-plugin/70a05dc4d4a330b797ce370fd587fccfe74355d9 Log: Revert "Revert " JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"" This reverts commit 88ffdfc69c43bd4dde21a6578b5ac466999b4fd4.

          Sam Van Oort added a comment -

          Closing this out because we have a bit chunk of the tree traversal APIs available now (at least getting parents – getting children can be done via the AbstractFlowScanner classes using bounds set from the previous). 

          Sam Van Oort added a comment - Closing this out because we have a bit chunk of the tree traversal APIs available now (at least getting parents – getting children can be done via the AbstractFlowScanner classes using bounds set from the previous). 

          Sam Van Oort added a comment - - edited

          Close enough, or at least many of the useful bits – workflow-api 2.22

          Sam Van Oort added a comment - - edited Close enough, or at least many of the useful bits – workflow-api 2.22

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-cps-plugin/88ffdfc69c43bd4dde21a6578b5ac466999b4fd4
          Log:
          Revert "JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/88ffdfc69c43bd4dde21a6578b5ac466999b4fd4 Log: Revert " JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java
          src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java
          http://jenkins-ci.org/commit/workflow-cps-plugin/c0daeb5ce9ba55e6f51cb6c8db903cc5fbba324b
          Log:
          Merge pull request #52 from jenkinsci/revert-50-jenkins-27395-block-structure-lookup

          Revert "JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecution.java src/main/java/org/jenkinsci/plugins/workflow/graph/BlockStartNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/FlowNode.java src/main/java/org/jenkinsci/plugins/workflow/graph/GraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graph/StandardGraphLookupView.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScanningUtils.java src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/NodeStepNamePredicate.java src/test/java/org/jenkinsci/plugins/workflow/graph/FlowNodeTest.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/FlowScannerTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/c0daeb5ce9ba55e6f51cb6c8db903cc5fbba324b Log: Merge pull request #52 from jenkinsci/revert-50-jenkins-27395-block-structure-lookup Revert " JENKINS-37573 / JENKINS-45553 Provide a fast view of block structures in the flow graph"

            svanoort Sam Van Oort
            svanoort Sam Van Oort
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: