Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39770

Pipeline visualization not rendered when there is more that 100 nodes

    XMLWordPrintable

    Details

    • Similar Issues:
    • Sprint:
      Blue Ocean 1.0-rc3

      Description

      Using the Blue Ocean plugin, I see the following traceback:

      FIREFOX:

      error rendering PipelineRunGraph TypeError: child is undefined
      Stack trace:
      render/selectedStage</childMatches<@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/org/jenkins/ui/jsmodules/blueocean-dashboard/jenkins-js-extension.js:10705:25
      render/selectedStage<@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/org/jenkins/ui/jsmodules/blueocean-dashboard/jenkins-js-extension.js:10704:40
      render@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/org/jenkins/ui/jsmodules/blueocean-dashboard/jenkins-js-extension.js:10701:33
      [394]</ReactCompositeComponentMixin._renderValidatedComponentWithoutOwnerOrContext@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:68260:27
      [394]</ReactCompositeComponentMixin._renderValidatedComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:68283:29
      [394]</ReactCompositeComponentMixin.performInitialMount@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67825:25
      [394]</ReactCompositeComponentMixin.mountComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67721:16
      [445]</ReactReconciler.mountComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:74734:18
      [394]</ReactCompositeComponentMixin.performInitialMount@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67834:18
      [394]</ReactCompositeComponentMixin.mountComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67721:16
      [445]</ReactReconciler.mountComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:74734:18
      [394]</ReactCompositeComponentMixin.performInitialMount@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67834:18
      [394]</ReactCompositeComponentMixin.mountComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67721:16
      [445]</ReactReconciler.mountComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:74734:18
      mountComponentIntoNode@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:72953:16
      [471]</Mixin.perform@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:77932:13
      batchedMountComponentIntoNode@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:72975:3
      [416]</ReactDefaultBatchingStrategy.batchedUpdates@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:71613:7
      batchedUpdates@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:75831:3
      [433]</ReactMount._renderNewRootComponent@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:73169:5
      [433]</ReactMount._renderSubtreeIntoContainer@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:73250:21
      [433]</ReactMount.render@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:73271:12
      _renderExtension@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:7547:17
      _renderAllExtensions@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:7529:25
      componentDidUpdate@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:7447:13
      [361]</<.notifyAll@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:62506:9
      [444]</ON_DOM_READY_QUEUEING.close@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:74600:5
      [471]</Mixin.closeAll@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:77998:11
      [471]</Mixin.perform@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:77945:11
      [471]</Mixin.perform@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:77932:13
      [453]</<.perform@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:75823:12
      [453]</flushBatchedUpdates@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:75906:7
      [471]</Mixin.closeAll@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:77998:11
      [471]</Mixin.perform@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:77945:11
      [416]</ReactDefaultBatchingStrategy.batchedUpdates@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:71615:7
      enqueueUpdate@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:75934:5
      enqueueUpdate@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:75542:3
      [452]</ReactUpdateQueue.enqueueSetState@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:75727:5
      [390]</ReactComponent.prototype.setState@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67033:3
      _onScrollHandler@http://dev018.ant.nmhn.lab:8080/adjuncts/5b1d2830/org/jenkins/ui/jsmodules/blueocean-dashboard/jenkins-js-extension.js:12224:17
      

      CHROMIUM:

      error rendering PipelineRunGraph TypeError: Cannot read property 'id' of undefined
          at http://127.0.0.1:15000/adjuncts/5b1d2830/org/jenkins/ui/jsmodules/blueocean-dashboard/jenkins-js-extension.js:10705:37
          at Array.filter (native)
          at http://127.0.0.1:15000/adjuncts/5b1d2830/org/jenkins/ui/jsmodules/blueocean-dashboard/jenkins-js-extension.js:10704:54
          at Array.filter (native)
          at PipelineRunGraph.render (http://127.0.0.1:15000/adjuncts/5b1d2830/org/jenkins/ui/jsmodules/blueocean-dashboard/jenkins-js-extension.js:10701:44)
          at ReactCompositeComponentWrapper._renderValidatedComponentWithoutOwnerOrContext (http://127.0.0.1:15000/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:68260:32)
          at ReactCompositeComponentWrapper._renderValidatedComponent (http://127.0.0.1:15000/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:68283:34)
          at ReactCompositeComponentWrapper.performInitialMount (http://127.0.0.1:15000/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67825:30)
          at ReactCompositeComponentWrapper.mountComponent (http://127.0.0.1:15000/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:67721:21)
          at Object.mountComponent (http://127.0.0.1:15000/adjuncts/5b1d2830/io/jenkins/blueocean/blueocean.js:74734:35)
      

      Using Jenkins 2.30 ; Blue Ocean beta 12.

        Attachments

        1. undef.png
          undef.png
          139 kB
        2. undef3.png
          undef3.png
          43 kB
        3. undef6.png
          undef6.png
          70 kB
        4. undef7.png
          undef7.png
          58 kB

          Issue Links

            Activity

            Hide
            roidelapluie Julien Pivotto added a comment -

            Suggestion is that pipeline does not work when there are more than 100 steps because /run/x/nodes only returns 100 entries.

            Show
            roidelapluie Julien Pivotto added a comment - Suggestion is that pipeline does not work when there are more than 100 steps because /run/x/nodes only returns 100 entries.
            Hide
            roidelapluie Julien Pivotto added a comment - - edited

            HOW TO REPREDUCE

            Pipeline:

            def max=100
            stage('test'){
                node {
                    checkouts = [:]
                    for (i=0; i<max; i++) {
                        def x = i
                        checkouts["foo${x}"] = {
                            checkout([$class: 'GitSCM', branches: [[name: '*/master']],
                            doGenerateSubmoduleConfigurations: false, extensions:
                            [[$class: 'RelativeTargetDirectory', relativeTargetDir: "foo${x}"],
                            [$class: 'ScmName', name: "foo${x}"]],
                            submoduleCfg: [], userRemoteConfigs: [[url: "/tmp/foo${x}"]]])
                       }
                    }
                    parallel checkouts 
                }
            }
            

            Shell:

            cd /tmp; git init foo;(cd foo; touch a; git add a; git commit -m a); for i in $(seq 0 150); do cp -r foo foo$i;(cd foo$i; git commit --amend -m a); done
            

            Works with def max=98

            Show
            roidelapluie Julien Pivotto added a comment - - edited HOW TO REPREDUCE Pipeline: def max=100 stage( 'test' ){ node { checkouts = [:] for (i=0; i<max; i++) { def x = i checkouts[ "foo${x}" ] = { checkout([$class: 'GitSCM' , branches: [[name: '*/master' ]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'RelativeTargetDirectory' , relativeTargetDir: "foo${x}" ], [$class: 'ScmName' , name: "foo${x}" ]], submoduleCfg: [], userRemoteConfigs: [[url: "/tmp/foo${x}" ]]]) } } parallel checkouts } } Shell: cd /tmp; git init foo;(cd foo; touch a; git add a; git commit -m a); for i in $(seq 0 150); do cp -r foo foo$i;(cd foo$i; git commit --amend -m a); done Works with def max=98
            Hide
            roidelapluie Julien Pivotto added a comment -

            Works without parallel.

            Show
            roidelapluie Julien Pivotto added a comment - Works without parallel.
            Hide
            roidelapluie Julien Pivotto added a comment -

            I think the scope of this ticket should be: raise a correct error messages if there are more than 100 nodes to display

            Show
            roidelapluie Julien Pivotto added a comment - I think the scope of this ticket should be: raise a correct error messages if there are more than 100 nodes to display
            Hide
            jamesdumay James Dumay added a comment -

            Thanks for the report Julien - ill have someone take a peek at this early next week

            Show
            jamesdumay James Dumay added a comment - Thanks for the report Julien - ill have someone take a peek at this early next week
            Hide
            michaelneale Michael Neale added a comment -

            So it works without parallel - ie lots of stages?

            Not sure how that many parallel can practically be displayed either

            Show
            michaelneale Michael Neale added a comment - So it works without parallel - ie lots of stages? Not sure how that many parallel can practically be displayed either
            Hide
            michaelneale Michael Neale added a comment -

            is 100 the exact limit?

            Show
            michaelneale Michael Neale added a comment - is 100 the exact limit?
            Hide
            jamesdumay James Dumay added a comment -

            Thats 100 parallels inside a stage?

            Show
            jamesdumay James Dumay added a comment - Thats 100 parallels inside a stage?
            Hide
            roidelapluie Julien Pivotto added a comment - - edited

            We have 33 repositories

            We run three operations on each of them:

            checkout – fetch tags – push new tag

            Plus other stages (build, publish, doc, release).

            The total is greater that 100.

            I think this is because the API /run/x/nodes is paginated ( https://github.com/jenkinsci/blueocean-plugin/blob/f1475e883adbfa8d77777aeafc6a4e6aff216cfc/blueocean-rest/src/main/java/io/jenkins/blueocean/rest/pageable/PagedResponse.java#L31 )

            I do not CARE about SEEING the 100 bullets. Actually that would be horrible. But having that parallelism decreases the duration from 60s to 5s.

            Show
            roidelapluie Julien Pivotto added a comment - - edited We have 33 repositories We run three operations on each of them: checkout – fetch tags – push new tag Plus other stages (build, publish, doc, release). The total is greater that 100. I think this is because the API /run/x/nodes is paginated ( https://github.com/jenkinsci/blueocean-plugin/blob/f1475e883adbfa8d77777aeafc6a4e6aff216cfc/blueocean-rest/src/main/java/io/jenkins/blueocean/rest/pageable/PagedResponse.java#L31 ) I do not CARE about SEEING the 100 bullets. Actually that would be horrible. But having that parallelism decreases the duration from 60s to 5s.
            Hide
            roidelapluie Julien Pivotto added a comment -
            lock('checkout') {
                stage('checkout') {
                    checkout([$class: 'GitSCM',
                            branches: [[name: branch]],
                            doGenerateSubmoduleConfigurations: false,
                            extensions: [[$class: 'CleanCheckout'], [$class: 'ScmName', name: 'super']],
                            submoduleCfg: [],
                            userRemoteConfigs: [[url: "${gitBaseUrl}/${superRepo}"]]])
            
                    def repos = readFile('.gitslave')
                    def reposLines = repos.readLines()
                    def checkouts = [:]
                    for (line in reposLines) {
                        def repoInfo = line.split(' ')
                        def repoUrl = repoInfo[0]
                        def repoPath = repoInfo[1]
                        def curatedRepoUrl = repoUrl.substring(4, repoUrl.length()-1)
                        def curatedRepoPath = repoPath.substring(1, repoPath.length()-1)
                        def thisCheckout = {
                            retry(3) {
                                checkout([$class: 'GitSCM',
                                        branches: [[name: branch]],
                                        doGenerateSubmoduleConfigurations: false,
                                        extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: curatedRepoPath], [$class: 'CleanCheckout'], [$class: 'ScmName', name: curatedRepoPath]],
                                        submoduleCfg: [],
                                        userRemoteConfigs: [[url: "${gitBaseUrl}/${curatedRepoUrl}"]]])
                            }
                        
                        checkouts[curatedRepoPath] = thisCheckout
                    }
                    parallel checkouts
                }
            }
            

            That piece of code does a SCM checkout of multiple repos in parallel. That creates a lot of nodes in blue ocean but those nodes are not necessary. Maybe a noNodes {} step around them could do the trick.

            Show
            roidelapluie Julien Pivotto added a comment - lock( 'checkout' ) { stage( 'checkout' ) { checkout([$class: 'GitSCM' , branches: [[name: branch]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'CleanCheckout' ], [$class: 'ScmName' , name: ' super ' ]], submoduleCfg: [], userRemoteConfigs: [[url: "${gitBaseUrl}/${superRepo}" ]]]) def repos = readFile( '.gitslave' ) def reposLines = repos.readLines() def checkouts = [:] for (line in reposLines) { def repoInfo = line.split( ' ' ) def repoUrl = repoInfo[0] def repoPath = repoInfo[1] def curatedRepoUrl = repoUrl.substring(4, repoUrl.length()-1) def curatedRepoPath = repoPath.substring(1, repoPath.length()-1) def thisCheckout = { retry(3) { checkout([$class: 'GitSCM' , branches: [[name: branch]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'RelativeTargetDirectory' , relativeTargetDir: curatedRepoPath], [$class: 'CleanCheckout' ], [$class: 'ScmName' , name: curatedRepoPath]], submoduleCfg: [], userRemoteConfigs: [[url: "${gitBaseUrl}/${curatedRepoUrl}" ]]]) } checkouts[curatedRepoPath] = thisCheckout } parallel checkouts } } That piece of code does a SCM checkout of multiple repos in parallel. That creates a lot of nodes in blue ocean but those nodes are not necessary. Maybe a noNodes {} step around them could do the trick.
            Hide
            michaelneale Michael Neale added a comment -

            hrm... so the problem is that we actively do not want to show that many things in parallel. there isn't a way to do that now that I can see though... so "fixing" it to show 100+ in parallel is not right though...

            Show
            michaelneale Michael Neale added a comment - hrm... so the problem is that we actively do not want to show that many things in parallel. there isn't a way to do that now that I can see though... so "fixing" it to show 100+ in parallel is not right though...
            Hide
            roidelapluie Julien Pivotto added a comment -

            Michael Neale We should at least handle that case and print a message that we can not display more than X nodes (and X can even be smaller, e.g 42!)

            Show
            roidelapluie Julien Pivotto added a comment - Michael Neale We should at least handle that case and print a message that we can not display more than X nodes (and X can even be smaller, e.g 42!)
            Hide
            michaelneale Michael Neale added a comment -

            Have started some discussion here: https://groups.google.com/forum/#!topic/jenkinsci-ux/5wHIChTF2VQ which may be better than in this ticket... work out what consensus is.

            Show
            michaelneale Michael Neale added a comment - Have started some discussion here: https://groups.google.com/forum/#!topic/jenkinsci-ux/5wHIChTF2VQ which may be better than in this ticket... work out what consensus is.
            Hide
            arronmabrey Arron Mabrey added a comment -

            I just ran into this 100 node limit as well. We have one mono-repo with >30 projects inside it.

            We have 4 stages `deps`, `pre-test`, `test`, `post-test`. Within each stage we run each project in parallel. Bringing the node count to > 120, thus causing the error.

            I'm trying to understand if this is a hard technical limit, or something else... like an undesirable user experience?

            Thanks,
            – Arron

            Show
            arronmabrey Arron Mabrey added a comment - I just ran into this 100 node limit as well. We have one mono-repo with >30 projects inside it. We have 4 stages `deps`, `pre-test`, `test`, `post-test`. Within each stage we run each project in parallel. Bringing the node count to > 120, thus causing the error. I'm trying to understand if this is a hard technical limit, or something else... like an undesirable user experience? Thanks, – Arron
            Hide
            jamesdumay James Dumay added a comment -

            Arron Mabrey at the moment this is a hard technical limit but it does raise questions to how big a Pipeline can get before the UI becomes somewhat useless. This is something we want to solve in the near future.

            Show
            jamesdumay James Dumay added a comment - Arron Mabrey at the moment this is a hard technical limit but it does raise questions to how big a Pipeline can get before the UI becomes somewhat useless. This is something we want to solve in the near future.
            Hide
            arronmabrey Arron Mabrey added a comment -

            James Dumay Thanks for the feedback. I also agree that the ui could get a little crazy with many nodes. If possible can you expand somewhat on what the particular hard limit is? The roundness of the number 100 seems somewhat arbitrary (although I realize it's probably not), what hard limit does this number represent?

            Show
            arronmabrey Arron Mabrey added a comment - James Dumay Thanks for the feedback. I also agree that the ui could get a little crazy with many nodes. If possible can you expand somewhat on what the particular hard limit is? The roundness of the number 100 seems somewhat arbitrary (although I realize it's probably not), what hard limit does this number represent?
            Hide
            jamesdumay James Dumay added a comment -

            Arron Mabrey 100 is the default limit for pagination in our rest APIs

            Show
            jamesdumay James Dumay added a comment - Arron Mabrey 100 is the default limit for pagination in our rest APIs
            Hide
            arronmabrey Arron Mabrey added a comment -

            Ahh okay.

            Show
            arronmabrey Arron Mabrey added a comment - Ahh okay.
            Hide
            jamesdumay James Dumay added a comment -

            Arron Mabrey for context, when the visualization was written it we did not have pagination for our REST API enabled by default.

            Show
            jamesdumay James Dumay added a comment - Arron Mabrey for context, when the visualization was written it we did not have pagination for our REST API enabled by default.
            Hide
            evildeece Alastair D'Silva added a comment -

            Vivek's comments here suggest that the limit can be altered via a parameter on the REST request:
            https://github.com/jenkinsci/blueocean-plugin/pull/486/files#r77956164

            Would it be possible to do that, rather than using the default of 100?

            Show
            evildeece Alastair D'Silva added a comment - Vivek's comments here suggest that the limit can be altered via a parameter on the REST request: https://github.com/jenkinsci/blueocean-plugin/pull/486/files#r77956164 Would it be possible to do that, rather than using the default of 100?
            Hide
            jamesdumay James Dumay added a comment -

            A bit more complicated than that but effectively yes we are looking into it

            Show
            jamesdumay James Dumay added a comment - A bit more complicated than that but effectively yes we are looking into it
            Hide
            jamesdumay James Dumay added a comment -

            This will be released in rc2 this week.

            Show
            jamesdumay James Dumay added a comment - This will be released in rc2 this week.

              People

              Assignee:
              tscherler Thorsten Scherler
              Reporter:
              roidelapluie Julien Pivotto
              Votes:
              3 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: