Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32485

Extension Point for contributing usage statistics

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: core
    • Labels:
    • Similar Issues:

      Description

      Would be interesting to make the usage statistics (See UsageStatistics.java) class could actually retrieve its information in a dynamic way. Instead of explicitly calling things in its code.

      Currently the code explicitly calls the ArchitectureMonitor to retrieve nodes archs.

      So, that new feature has resurfaced recently to facilitate handling JENKINS-26466 (node_monitors package extraction as a new plugin).

      So, adding a new dedicated extension point would enable some things:

      • it would enable ArchitectureMonitor to be an @Extension of that new ExtensionPoint so that UsageStatistics doesn't depend on a NodeMonitor impl anymore (thanks Jesse Glick for the idea)
      • On the feature front, it would also help fulfill a request I've done some years ago, which actually resurfaced here

        Attachments

          Issue Links

            Activity

            Hide
            batmat Baptiste Mathus added a comment -

            Ping Daniel Beck with your security officer's cap on, as this feature might add privacy/security issues along the way if any plugin can easily push any stats he wants here. WDYT?

            Side note: this also might not be a real issue, as any plugin is actually already able to push anything he wants anywhere if he wants to without using the usage statistics infrastructure.

            Show
            batmat Baptiste Mathus added a comment - Ping Daniel Beck with your security officer's cap on, as this feature might add privacy/security issues along the way if any plugin can easily push any stats he wants here. WDYT? Side note: this also might not be a real issue, as any plugin is actually already able to push anything he wants anywhere if he wants to without using the usage statistics infrastructure.
            Hide
            batmat Baptiste Mathus added a comment -

            Some more context :

            [23:07] <+     batmat> | jglick: btw, would be happy to have your opinion on UsageStatistics -> ArchitectureMonitor
            [23:07] <+     batmat> | jglick: my understanding is that it uses it only for ~3 lines of code to retrieve os.name etc. sysprops.
            [23:08] <+     batmat> | jglick: wondering if the solution is to do some kind of duplication (I hate myself saying that). WDYT?
            [23:11] <      jglick> | batmat: well it looks like it is using a *cached* version of that info, so making a blocking call to the slave during page rendering is not OK, regardless of duplication. 
                                  This seems like an appropriate place for an extension point: `UsageStatisticsContributor` or the like. `ArchitectureMonitor` would implement it, and iterate `Computer`s 
                                  injecting `os` keys into the JSON.
            [23:19] <+     batmat> | jglick: I think I got it. And that would make potentially enriching the statistics easier in the go. Thanks for the heads up
            [23:20] <      jglick> | batmat: exactly
            
            Show
            batmat Baptiste Mathus added a comment - Some more context : [23:07] <+ batmat> | jglick: btw, would be happy to have your opinion on UsageStatistics -> ArchitectureMonitor [23:07] <+ batmat> | jglick: my understanding is that it uses it only for ~3 lines of code to retrieve os.name etc. sysprops. [23:08] <+ batmat> | jglick: wondering if the solution is to do some kind of duplication (I hate myself saying that). WDYT? [23:11] < jglick> | batmat: well it looks like it is using a *cached* version of that info, so making a blocking call to the slave during page rendering is not OK, regardless of duplication. This seems like an appropriate place for an extension point: `UsageStatisticsContributor` or the like. `ArchitectureMonitor` would implement it, and iterate `Computer`s injecting `os` keys into the JSON. [23:19] <+ batmat> | jglick: I think I got it. And that would make potentially enriching the statistics easier in the go. Thanks for the heads up [23:20] < jglick> | batmat: exactly
            Show
            batmat Baptiste Mathus added a comment - https://github.com/jenkinsci/jenkins/pull/1985
            Hide
            danielbeck Daniel Beck added a comment -

            A more fine-grained control to opt out of certain kinds of reporting to the anonymous stats may be helpful. Enable/disable status and a UI to control it could be a part of the extension point. While developers may be able to circumvent that in their implementation, I'd consider that acting in bad faith and would not hesitate to remove any such plugin and its devs from the project permanently.

            That said, I see the danger mostly with infra-statistics publishing evil data. Luckily, those that can merge changes there is a tiny set of more trusted people. Otherwise, access to the data that's sent is limited to even fewer people.

            Would be interesting to look into about the effects any such extensibility may have on the server-side processing of the data before offering this extension point to plugins. Worst case, it may actually break stats generation.

            Show
            danielbeck Daniel Beck added a comment - A more fine-grained control to opt out of certain kinds of reporting to the anonymous stats may be helpful. Enable/disable status and a UI to control it could be a part of the extension point. While developers may be able to circumvent that in their implementation, I'd consider that acting in bad faith and would not hesitate to remove any such plugin and its devs from the project permanently. That said, I see the danger mostly with infra-statistics publishing evil data. Luckily, those that can merge changes there is a tiny set of more trusted people. Otherwise, access to the data that's sent is limited to even fewer people. Would be interesting to look into about the effects any such extensibility may have on the server-side processing of the data before offering this extension point to plugins. Worst case, it may actually break stats generation.
            Hide
            batmat Baptiste Mathus added a comment - - edited

            A more fine-grained control to opt out of certain kinds of reporting to the anonymous stats may be helpful. Enable/disable status and a UI to control it could be a part of the extension point.

            Gonna have to think of it. Will take a bit more time as I'm far less fluent with UI.

            While developers may be able to circumvent that in their implementation, I'd consider that acting in bad faith and would not hesitate to remove any such plugin and its devs from the project permanently.

            Sure, makes me think also I should certainly add a big warning in the extension point javadoc about that: be really thoughtful about privacy and anonymization when planning to implement that extension point.
            And in doubt (or always, maybe): make validate the new usage code by the other developers (and/or maybe at least by the Jenkins security officer of the moment).

            Show
            batmat Baptiste Mathus added a comment - - edited A more fine-grained control to opt out of certain kinds of reporting to the anonymous stats may be helpful. Enable/disable status and a UI to control it could be a part of the extension point. Gonna have to think of it. Will take a bit more time as I'm far less fluent with UI. While developers may be able to circumvent that in their implementation, I'd consider that acting in bad faith and would not hesitate to remove any such plugin and its devs from the project permanently. Sure, makes me think also I should certainly add a big warning in the extension point javadoc about that: be really thoughtful about privacy and anonymization when planning to implement that extension point. And in doubt (or always, maybe): make validate the new usage code by the other developers (and/or maybe at least by the Jenkins security officer of the moment).
            Hide
            rtyler R. Tyler Croy added a comment -

            I'm going to leave this labeled evergreen, but I want to make sure it is understood that the existing Usage Statistics architecture is in no way, shape, or form, going to be supported moving forward.

            We will however be supporting different mechanisms for exposing and [sharing analytics](https://issues.jenkins-ci.org/browse/JENKINS-49808), but the "Usage Statistics" approach currently used in Jenkins is a dead end for Evergreen.

            Show
            rtyler R. Tyler Croy added a comment - I'm going to leave this labeled evergreen , but I want to make sure it is understood that the existing Usage Statistics architecture is in no way, shape, or form, going to be supported moving forward. We will however be supporting different mechanisms for exposing and [sharing analytics] ( https://issues.jenkins-ci.org/browse/JENKINS-49808 ), but the "Usage Statistics" approach currently used in Jenkins is a dead end for Evergreen.
            Hide
            jglick Jesse Glick added a comment -

            Since JENKINS-49808 is being tracked on the Evergreen board, if you do not intend to do anything with this issue, you may as well remove the essentials & evergreen labels.

            Show
            jglick Jesse Glick added a comment - Since JENKINS-49808 is being tracked on the Evergreen board, if you do not intend to do anything with this issue, you may as well remove the essentials & evergreen labels.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              batmat Baptiste Mathus
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated: