Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34038

New flow analyzer algorithm in pipeline stage view based on a visitor pattern




      Implementation of the fixes for https://issues.jenkins-ci.org/browse/JENKINS-33624 greatly increases scalability of stage view with large flow graphs, but it revealed some major underlying issues with the existing algorithms, in correctness, scalability, and extensibility. They're O(n log n) at least, and in practice something like O(m*n + n log n) with m being stage count.

      • This needs to track FlowSegments, which are linear sections of the flow graph with a defined start and end. This is needed because you need to handle and work with the structure of the flow (stages, labels, parallel blocks) without having to be aware of every node within, since there may be thousands.
      • This needs to support parallel flowsegments correctly, enabling JENKINS-33185
      • It needs to be aware of block scoping to correctly assign executor node labels to FlowNodes - Root cause for JENKINS-33290 and eventually for block labelling.
      • It needs to offer an (eventually pluggable) filter option to allow searching for specific nodes.
      • It needs to be largely O( n ), and where possible only do one pass through flownodes.
      • It should offer an API that is SAX-like, which tracks segment start/end, block start/end, and node visiting (in stage) with stages being mapped as a segment type.
      • It should be extensible for plugin authors to define their own analysis of flowgraphs (probably for visualization)

      If it's considered high enough quality, this can be added to the main pipeline plugin, as an available API.

      I have a PR (linked) that is addressing this need, which is currently a work in progress but close to covering this.




            svanoort Sam Van Oort
            svanoort Sam Van Oort
            2 Vote for this issue
            6 Start watching this issue