Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-2941

Document plugin installation manager use for plugin upgrades

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: jenkins.io
    • Labels:
      None
    • Environment:
      - jenkins 2.278
      - k8s 1.17
    • Similar Issues:

      Description

      Document plugin installation manager effective use

      Jenkins is installed successfully the last week.

      The team spent time to stabilize Jenkins + its plugins with version pinning.

      Preprod jenkins was working fine,,, so we promoted to prod.

      Today, while checking the Pod status, it was the surprise : 

       

         Status          RESTART AGE
      Init:CrashLoopBackOff 192 6d11h
      

       

      I checked the log of init container, i got : 

       

      ssh-credentials (1.18.2) requires a greater version of Jenkins (2.282) than 2.278
       
      

      Are Jenkins plugins not immutable artifacts ? 

       

       

        Attachments

          Activity

          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          > Are Jenkins plugins not immutable artifacts ? 

          Depends on how your system is configured. But yes, keeping them immutable is a best practice for containerized deployments

           

          Show
          oleg_nenashev Oleg Nenashev added a comment - > Are Jenkins plugins not immutable artifacts ?  Depends on how your system is configured. But yes, keeping them immutable is a best practice for containerized deployments  
          Hide
          markewaite Mark Waite added a comment -

          If your Jenkins installation is defined to use a plugins.txt file to download the plugins each time it starts, then you have created a Jenkins configuration that is mutable, especially if the version numbers are not specified in your plugins.txt file.

          The "Installing Jenkins on Docker" page illustrates how to create your own Docker image that includes a specific version of a plugin (and all the dependencies of that plugin). A simplified version of that Dockerfile looks like this:

          FROM jenkins/jenkins:2.277.1-lts-jdk11
          RUN jenkins-plugin-cli --plugins blueocean:1.24.4
          

          When that Docker image is created with docker build -t myjenkins-blueocean:1.1 ., then that Docker image is immutable, with exactly the plugin versions that were downloaded at that time. Every time that Docker image is run, it will use those plugins.

          If you're managing your plugin versions some other way, you may get different results.

          For example, during an outage of the Jenkins mirror sites, a number of users discovered that they had made the mistake of downloading the latest Jenkins plugins every time they start their Kubernetes pod. That delays pod startup significantly and makes the pod depend on the Jenkins infrastructure during startup. Many users have since switched to create Docker images that include the Jenkins plugins so that they don't require download of plugins at pod startup.

          Show
          markewaite Mark Waite added a comment - If your Jenkins installation is defined to use a plugins.txt file to download the plugins each time it starts, then you have created a Jenkins configuration that is mutable, especially if the version numbers are not specified in your plugins.txt file. The "Installing Jenkins on Docker" page illustrates how to create your own Docker image that includes a specific version of a plugin (and all the dependencies of that plugin). A simplified version of that Dockerfile looks like this: FROM jenkins/jenkins:2.277.1-lts-jdk11 RUN jenkins-plugin-cli --plugins blueocean:1.24.4 When that Docker image is created with docker build -t myjenkins-blueocean:1.1 . , then that Docker image is immutable, with exactly the plugin versions that were downloaded at that time. Every time that Docker image is run, it will use those plugins. If you're managing your plugin versions some other way, you may get different results. For example, during an outage of the Jenkins mirror sites, a number of users discovered that they had made the mistake of downloading the latest Jenkins plugins every time they start their Kubernetes pod. That delays pod startup significantly and makes the pod depend on the Jenkins infrastructure during startup. Many users have since switched to create Docker images that include the Jenkins plugins so that they don't require download of plugins at pod startup.
          Hide
          abdennour Abdennour Toumi added a comment -

          Thank you Mark Waite , i tried this way, but jenkins never comes up with this way.

          The 2 challenges are with this way : 

          • How to upgrade existing jenkins instance with this method without crashing the app ( conflict between plugins already installed, and plugins coming with the jenkins image)
          • How to give the right values in the helm chart, so it uses plugins inside the image and ignore everything else.

          Actually, we faced these challenges, and the jenkins instance never come up. 

          Again, from release engineering perspective, Mark Waite solution is the best because artifacts must be immutable. 

          However, there are some challenges that i mention some of them above.

           

          Show
          abdennour Abdennour Toumi added a comment - Thank you Mark Waite  , i tried this way, but jenkins never comes up with this way. The 2 challenges are with this way :  How to upgrade existing jenkins instance with this method without crashing the app ( conflict between plugins already installed, and plugins coming with the jenkins image) How to give the right values in the helm chart, so it uses plugins inside the image and ignore everything else. Actually, we faced these challenges, and the jenkins instance never come up.  Again, from release engineering perspective, Mark Waite  solution is the best because artifacts must be immutable.  However, there are some challenges that i mention some of them above.  
          Hide
          markewaite Mark Waite added a comment -

          Abdennour Toumi I think you're making some other mistake if Jenkins never comes up.

          If you define a custom Jenkins image and include the plugins in the Jenkins image, then you can run that image locally (outside Kubernetes) to confirm that it starts successfully.

          If it does not start successfully, then you can fix it, create a new version of the image, and test the new version of the image locally (outside Kubernetes).

          Show
          markewaite Mark Waite added a comment - Abdennour Toumi I think you're making some other mistake if Jenkins never comes up. If you define a custom Jenkins image and include the plugins in the Jenkins image, then you can run that image locally (outside Kubernetes) to confirm that it starts successfully. If it does not start successfully, then you can fix it, create a new version of the image, and test the new version of the image locally (outside Kubernetes).
          Hide
          markewaite Mark Waite added a comment -

          Abdennour Toumi I would like to modify this issue to be a request for a more detailed description of the preferred techniques for managing plugin versions in a Jenkins that is managed as code. In that case, we would use it to describe the plugin installation manager tool on the jenkins.io site. We would describe the conditions that cause the message you encountered and the command line arguments that can be used to avoid that message.

          As the issue is phrased now, I don't know what we can do that would resolve it.

          Would that rephrasing be acceptable for you or would you like to take a different approach?

          Show
          markewaite Mark Waite added a comment - Abdennour Toumi I would like to modify this issue to be a request for a more detailed description of the preferred techniques for managing plugin versions in a Jenkins that is managed as code. In that case, we would use it to describe the plugin installation manager tool on the jenkins.io site. We would describe the conditions that cause the message you encountered and the command line arguments that can be used to avoid that message. As the issue is phrased now, I don't know what we can do that would resolve it. Would that rephrasing be acceptable for you or would you like to take a different approach?
          Hide
          markewaite Mark Waite added a comment -

          Reassigned as a documentation issue.

          Show
          markewaite Mark Waite added a comment - Reassigned as a documentation issue.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            abdennour Abdennour Toumi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: