Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39646

Properly generate bundles for packages that have multiple top-level modules

    XMLWordPrintable

Details

    • Blue Ocean 1.2-beta2

    Description

      Bundled are generated in a sub optimal way. One side effect is there could be multiple submodules of popular libraries appearing in different bundles. 


      See this doc for a description of the problem(s).

      Attachments

        Issue Links

          Activity

            tfennelly Tom FENNELLY created issue -
            tfennelly Tom FENNELLY made changes -
            Field Original Value New Value
            Epic Link JENKINS-37957 [ 174099 ]
            tfennelly Tom FENNELLY made changes -
            Link This issue is related to JENKINS-38013 [ JENKINS-38013 ]
            tfennelly Tom FENNELLY made changes -
            Link This issue depends on JENKINS-39657 [ JENKINS-39657 ]
            tfennelly Tom FENNELLY made changes -
            Description We fixed some bundling issues last week ... non bootstrap bundles (dashboard etc) are now back down to somewhat reasonable sizes (they were at ~ 2Mb in some cases), but the blueocean.js bootstrp bundle is still at about 3Mb because we have the kitchen sink thrown in there. If it's going to be that huge (something we did not know earlier in the game), then we need to tweak how we are doing the bundling.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue).

            So, parking this issue for now and we will come back to it later. Relevant WIP:

            * https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles
            * https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles
            We fixed some bundling issues last week ... non bootstrap bundles (dashboard etc) are now back down to somewhat reasonable sizes (they were at ~ 2Mb in some cases), but the blueocean.js bootstrp bundle is still at about 3Mb because we have the kitchen sink thrown in there. If it's going to be that huge (something we did not know earlier in the game), then we need to tweak how we are doing the bundling.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see JENKINS-39657.

            So, parking this issue for now and we will come back to it later. Relevant WIP:

            * https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles
            * https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles
            tfennelly Tom FENNELLY made changes -
            Component/s js-builder [ 21132 ]
            Component/s js-modules [ 21133 ]
            tfennelly Tom FENNELLY made changes -
            Summary Extract more dependencies out into their own JS bundles Properly generate bundles for packages that have multiple top-level modules
            tfennelly Tom FENNELLY made changes -
            Issue Type Task [ 3 ] Bug [ 1 ]
            tfennelly Tom FENNELLY made changes -
            Epic Link JENKINS-37957 [ 174099 ] JENKINS-35749 [ 171790 ]
            tfennelly Tom FENNELLY made changes -
            Priority Major [ 3 ] Blocker [ 1 ]
            tfennelly Tom FENNELLY made changes -
            Description We fixed some bundling issues last week ... non bootstrap bundles (dashboard etc) are now back down to somewhat reasonable sizes (they were at ~ 2Mb in some cases), but the blueocean.js bootstrp bundle is still at about 3Mb because we have the kitchen sink thrown in there. If it's going to be that huge (something we did not know earlier in the game), then we need to tweak how we are doing the bundling.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see JENKINS-39657.

            So, parking this issue for now and we will come back to it later. Relevant WIP:

            * https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles
            * https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles
            js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see JENKINS-39657.

            So, parking this issue for now and we will come back to it later. Relevant WIP:

            * https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles
            * https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles
            tfennelly Tom FENNELLY made changes -
            Description js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see JENKINS-39657.

            So, parking this issue for now and we will come back to it later. Relevant WIP:

            * https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles
            * https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles
            js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles that are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see JENKINS-39657.

            So, parking this issue for now and we will come back to it later. Relevant WIP:

            * https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles
            * https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles
            jamesdumay James Dumay made changes -
            Priority Blocker [ 1 ] Major [ 3 ]
            jamesdumay James Dumay made changes -
            Labels technical-debt
            jamesdumay James Dumay made changes -
            Epic Link JENKINS-35749 [ 171790 ] JENKINS-35756 [ 171764 ]
            michaelneale Michael Neale made changes -
            Description js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles that are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see JENKINS-39657.

            So, parking this issue for now and we will come back to it later. Relevant WIP:

            * https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles
            * https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles
            js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles that are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see -JENKINS-39657-.

            So, parking this issue for now and we will come back to it later. Relevant WIP:
             * [https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles]
             * [https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles]

             

            Those branches are way out of date now, but from memory, one of the main issues that needs to be solved (and which I take a stab at in those PRs) is the idea of generating a "complete" bundles (deterministically including all modules from a package e.g. react Vs just the modules that are on the entry module's graph) and changing the module loading such that it can load any module from a bundle (and not just the entry module).
            tfennelly Tom FENNELLY made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            tfennelly Tom FENNELLY made changes -
            Attachment screenshot-1.png [ 38658 ]
            michaelneale Michael Neale made changes -
            Sprint Blue Ocean 1.2-beta2 [ 341 ]
            michaelneale Michael Neale made changes -
            Assignee Tom FENNELLY [ tfennelly ]
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.2-beta2 [ 341 ] Blue Ocean 1.2-beta2, Blue Ocean 1.2-beta3 [ 341, 346 ]
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.2-beta2, Blue Ocean 1.2-beta3 [ 341, 346 ] Blue Ocean 1.2-beta2 [ 341 ]
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.2-beta2 [ 341 ] Blue Ocean 1.2-beta2, Blue Ocean 1.2-beta4 [ 341, 361 ]
            jamesdumay James Dumay made changes -
            Rank Ranked higher
            jamesdumay James Dumay made changes -
            Priority Critical [ 2 ] Major [ 3 ]
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.2-beta2, Blue Ocean 1.2-beta4 [ 341, 361 ] Blue Ocean 1.3, Blue Ocean 1.2-beta2 [ 296, 341 ]
            jamesdumay James Dumay made changes -
            Rank Ranked lower
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.3-beta-2, Blue Ocean 1.2-beta2 [ 296, 341 ] Blue Ocean 1.4, Blue Ocean 1.2-beta2 [ 311, 341 ]
            jamesdumay James Dumay made changes -
            Rank Ranked higher
            michaelneale Michael Neale made changes -
            Description js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles that are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see -JENKINS-39657-.

            So, parking this issue for now and we will come back to it later. Relevant WIP:
             * [https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles]
             * [https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles]

             

            Those branches are way out of date now, but from memory, one of the main issues that needs to be solved (and which I take a stab at in those PRs) is the idea of generating a "complete" bundles (deterministically including all modules from a package e.g. react Vs just the modules that are on the entry module's graph) and changing the module loading such that it can load any module from a bundle (and not just the entry module).
            Bundled are generated in a sub optimal way. One side effect is there could be multiple submodules of popular libraries appearing in different bundles. 

            ----

            js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles that are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see --JENKINS-39657--.

            So, parking this issue for now and we will come back to it later. Relevant WIP:
             * [https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles]
             * [https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles]

             

            Those branches are way out of date now, but from memory, one of the main issues that needs to be solved (and which I take a stab at in those PRs) is the idea of generating a "complete" bundles (deterministically including all modules from a package e.g. react Vs just the modules that are on the entry module's graph) and changing the module loading such that it can load any module from a bundle (and not just the entry module).
            tfennelly Tom FENNELLY made changes -
            Description Bundled are generated in a sub optimal way. One side effect is there could be multiple submodules of popular libraries appearing in different bundles. 

            ----

            js-modules, js-builder etc were written based on some assumptions (pre Blue Ocean). The main one that's relevant to this task was the assumption that dependency packages are always accessed via a single top-level/entry/main module and that that modules would be the bundle entry-point module.

            Of course in reality that's not how some NPM packages are created. Some of them contain multiple top-level/entry modules. React is the one most relevant to Blue Ocean ... it has lots of top level modules, so much so that lots of secondary/child NPM packages have sprung up in the react eco system that do nothing more than define an NPM "main" module entry point for a module inside the react package e.g. some of the ones we use in Blue Ocean ... "react-dom", "react-addons-css-transition-group", "react-addons-test-utils".

            We need to change how our bundling works so that we can properly accommodate this reality, generating dependency bundles that are guaranteed to contain all the package modules needed so that all entry-modules are covered/supported by the bundle. Likewise, when generating bundles that depend on these packages, we need to make sure that the externalized/shared bundle is always wired in properly and that no external dep modules "leak" in through the back door.

            I've been working on this off-and-on over the last week or two. I've made progress on it (blueocean.js now at abou 0.6Mb), but have run into other react pita issues now (react == "pain" for me :) ). I've not been able to figure that issue out and I think I'm going to need to build some more tooling for myself in order to do that (will also hopefully help when we hit the next react/other bundling issue) - see --JENKINS-39657--.

            So, parking this issue for now and we will come back to it later. Relevant WIP:
             * [https://github.com/tfennelly/jenkins-js-builder/tree/all-extern-bundles]
             * [https://github.com/tfennelly/blueocean-plugin/tree/all-extern-bundles]

             

            Those branches are way out of date now, but from memory, one of the main issues that needs to be solved (and which I take a stab at in those PRs) is the idea of generating a "complete" bundles (deterministically including all modules from a package e.g. react Vs just the modules that are on the entry module's graph) and changing the module loading such that it can load any module from a bundle (and not just the entry module).
            Bundled are generated in a sub optimal way. One side effect is there could be multiple submodules of popular libraries appearing in different bundles. 

            ----

            [See this doc for a description of the problem(s)|https://docs.google.com/document/d/1HPeK1W9UtHAGGmjnrBTIjmb7gs8HEYgJ5KtSxOkc9tY/edit?usp=sharing].
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.5 - candidates, Blue Ocean 1.2-beta2 [ 311, 341 ] Blue Ocean 1.2-beta2 [ 341 ]
            jamesdumay James Dumay made changes -
            Rank Ranked higher
            jamesdumay James Dumay made changes -
            Resolution Won't Fix [ 2 ]
            Status Open [ 1 ] Resolved [ 5 ]
            tscherler Thorsten Scherler made changes -
            Link This issue blocks JENKINS-47673 [ JENKINS-47673 ]
            tscherler Thorsten Scherler made changes -
            Link This issue relates to JENKINS-47673 [ JENKINS-47673 ]
            tscherler Thorsten Scherler made changes -
            Link This issue blocks JENKINS-47673 [ JENKINS-47673 ]
            jbriden Jenn Briden made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

            People

              Unassigned Unassigned
              tfennelly Tom FENNELLY
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: