Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65107

The new LTS release may lock out old users

    XMLWordPrintable

Details

    Description

      Hi,

       

      If a user has a very old passwordhash defined in users/foo_xx/config.xml the new LTS release may lock the user out of jenkins.

       

      This happened with my admin user (setup arund 2012).I spent > 1 day figuring this out and ruling out a security incident and would greatly appreciate that this was mentioned in the releasnotes of 2.277. I upgraded from 2.268.

       

       

      Attachments

        Issue Links

          Activity

            tarjei_asku Tarjei created issue -
            markewaite Mark Waite added a comment - - edited

            I can't duplicate the problem that you've described. Maybe there are details missing from your description?

            Steps I used while failing to duplicate the problem:

            1. Download Jenkins 2.105 (from 2012)
              $ wget https://fallback.get.jenkins.io/war/2.105/jenkins.war
            2. Run Jenkins 2.105
              $ mv jenkins.war jenkins-2.105.war && JENKINS_HOME=$(pwd) java -jar jenkins-2.105.war
            3. Answer the password prompt with the random password written to the Jenkins startup log
            4. Choose to install no plugins (since update center does not support that old a version)
            5. Add myself as a user with the same random password as generated by Jenkins (also tried a more common password with same results)
            6. Stop Jenkins 2.105
            7. Download Jenkins 2.277.1
              $ wget https://get.jenkins.io/war-stable/2.277.1/jenkins.war
            8. Start Jenkins 2.277.1
              $ mv jenkins.war jenkins-2.277.1.war && JENKINS_HOME=$(pwd) java -jar jenkins-2.277.1.war
            9. Login with the username and password created by Jenkins 2.105

            The passwordHash written by Jenkins 2.105 looks like this:

            <passwordHash>#jbcrypt:$2a$10$oaFBLELplTbFIOw7khYbjusCMnXOnaP7vhNpEpBxW7PVeQRwbQvKW</passwordHash>

            The passwordHash written by Jenkins 2.277.1 looks like this (in a file with a version 10 XML tag inside):

            <passwordHash>#jbcrypt:$2a$10$oaFBLELplTbFIOw7khYbjusCMnXOnaP7vhNpEpBxW7PVeQRwbQvKW</passwordHash>

            Maybe the password hash from much earlier than 2.105?

            Did you follow the instructions in the Jenkins 2.277.1 upgrade guide by updating your authentication plugins before the upgrade to 2.277.1?

            markewaite Mark Waite added a comment - - edited I can't duplicate the problem that you've described. Maybe there are details missing from your description? Steps I used while failing to duplicate the problem: Download Jenkins 2.105 (from 2012) $ wget https://fallback.get.jenkins.io/war/2.105/jenkins.war Run Jenkins 2.105 $ mv jenkins.war jenkins-2.105.war && JENKINS_HOME=$(pwd) java -jar jenkins-2.105.war Answer the password prompt with the random password written to the Jenkins startup log Choose to install no plugins (since update center does not support that old a version) Add myself as a user with the same random password as generated by Jenkins (also tried a more common password with same results) Stop Jenkins 2.105 Download Jenkins 2.277.1 $ wget https://get.jenkins.io/war-stable/2.277.1/jenkins.war Start Jenkins 2.277.1 $ mv jenkins.war jenkins-2.277.1.war && JENKINS_HOME=$(pwd) java -jar jenkins-2.277.1.war Login with the username and password created by Jenkins 2.105 The passwordHash written by Jenkins 2.105 looks like this: <passwordHash>#jbcrypt:$2a$10$oaFBLELplTbFIOw7khYbjusCMnXOnaP7vhNpEpBxW7PVeQRwbQvKW</passwordHash> The passwordHash written by Jenkins 2.277.1 looks like this (in a file with a version 10 XML tag inside): <passwordHash>#jbcrypt:$2a$10$oaFBLELplTbFIOw7khYbjusCMnXOnaP7vhNpEpBxW7PVeQRwbQvKW</passwordHash> Maybe the password hash from much earlier than 2.105? Did you follow the instructions in the Jenkins 2.277.1 upgrade guide by updating your authentication plugins before the upgrade to 2.277.1?
            tarjei_asku Tarjei added a comment -

            Hi Mark, thank you for a throughout and quick followup.

            I can confirm that the passwordHash did not contain a hash starting with #jbcrypt, something that suggests that the install was even older than I stated. I think it might be as old as from 2010.

            Another issue that I had was that there were two admin entries in the users directory, but only one in the users.xml file. I do not know if the other entry was seen by jenkins at all or if if just existed in the directory without affecting any code at all.

             

            > Did you follow the instructions in the Jenkins 2.277.1 upgrade guide by updating your authentication plugins before the upgrade to 2.277.1?

            I have made sure that all plugins have been up to date at all times. It was my understanding reading the guide that that was what was needed.

             

             

             

            tarjei_asku Tarjei added a comment - Hi Mark, thank you for a throughout and quick followup. I can confirm that the passwordHash did not contain a hash starting with #jbcrypt, something that suggests that the install was even older than I stated. I think it might be as old as from 2010. Another issue that I had was that there were two admin entries in the users directory, but only one in the users.xml file. I do not know if the other entry was seen by jenkins at all or if if just existed in the directory without affecting any code at all.   > Did you follow the instructions in the Jenkins 2.277.1 upgrade guide by updating your authentication plugins before the upgrade to 2.277.1? I have made sure that all plugins have been up to date at all times. It was my understanding reading the guide that that was what was needed.      
            markewaite Mark Waite added a comment -

            Thanks tarjei_asku. I'm not sure which change would cause very old password hashes to not be interpreted correctly in 2.277.1 when they were correctly interpreted in 2.268, but that is a relatively narrow range of changes that I can search to find what I missed in the changelog and upgrade guide. Thanks again for doing the research to identify the issue. Once I can duplicate the problem I'll include the item in the changelog and in the upgrade guide.

            markewaite Mark Waite added a comment - Thanks tarjei_asku . I'm not sure which change would cause very old password hashes to not be interpreted correctly in 2.277.1 when they were correctly interpreted in 2.268, but that is a relatively narrow range of changes that I can search to find what I missed in the changelog and upgrade guide. Thanks again for doing the research to identify the issue. Once I can duplicate the problem I'll include the item in the changelog and in the upgrade guide.
            danielbeck Daniel Beck made changes -
            Field Original Value New Value
            Labels regression
            danielbeck Daniel Beck made changes -
            Priority Minor [ 4 ] Critical [ 2 ]
            danielbeck Daniel Beck added a comment -

            I upgraded from 2.268.

            To clarify, you used the 2.268 weekly release from November, and updated to the LTS release 2.277.1, switching release lines in the process?

            danielbeck Daniel Beck added a comment - I upgraded from 2.268. To clarify, you used the 2.268 weekly release from November, and updated to the LTS release 2.277.1, switching release lines in the process?
            danielbeck Daniel Beck added a comment -

            I couldn't find a change in the specified version range that looks related. My first guess was my recent(ish) work on removing DES from Protector (/me sighs), but that was back in 2.236.

            danielbeck Daniel Beck added a comment - I couldn't find a change in the specified version range that looks related. My first guess was my recent(ish) work on removing DES from Protector (/me sighs), but that was back in 2.236.
            danielbeck Daniel Beck added a comment -

            https://groups.google.com/g/jenkinsci-users/c/zwm_iGQLf-I/m/d6xl5bszCQAJ indicates a regression in 2.266, but this report states it still worked in 2.268.

            danielbeck Daniel Beck added a comment - https://groups.google.com/g/jenkinsci-users/c/zwm_iGQLf-I/m/d6xl5bszCQAJ indicates a regression in 2.266, but this report states it still worked in 2.268.
            worf_lt Worf added a comment -

            Hi Guys!
            I've tried to update to latest LTS (v2.277.2) yesterday and ran into the same issue as described by the reporter. Our Jenkins instance runs since 2012 and was continuously updated.

            However, all existing users still use SHA password hashes (there wasn't any migration to jbcrypt) and since commit https://github.com/jenkinsci/jenkins/commit/a9ca5ef3d4c97937636bf3c585f4232514279b14 the CLASSIC password encoder has gone from hudson.security.HudsonPrivateSecurityRealm.
            As a consequence, no user could log in anymore and I had to rollback to v2.263.4 where login still works.

            worf_lt Worf added a comment - Hi Guys! I've tried to update to latest LTS (v2.277.2) yesterday and ran into the same issue as described by the reporter. Our Jenkins instance runs since 2012 and was continuously updated. However, all existing users still use SHA password hashes (there wasn't any migration to jbcrypt) and since commit https://github.com/jenkinsci/jenkins/commit/a9ca5ef3d4c97937636bf3c585f4232514279b14 the CLASSIC password encoder has gone from hudson.security.HudsonPrivateSecurityRealm . As a consequence, no user could log in anymore and I had to rollback to v2.263.4 where login still works.
            markewaite Mark Waite made changes -
            Labels regression JEP-227 regression
            markewaite Mark Waite added a comment -

            Thanks for the details worf_lt. The transition from acegi security to Spring Security framework (JEP-227) is intentional in Jenkins 2.277.1. If users in 2.263.4 update their password, does it write a bcrypt based password or does it continue to write the old style password?

            I suspect that the removal of the classic password encoder is intentional. Copying jglick in case he wants to comment on the change.

            markewaite Mark Waite added a comment - Thanks for the details worf_lt . The transition from acegi security to Spring Security framework ( JEP-227 ) is intentional in Jenkins 2.277.1. If users in 2.263.4 update their password, does it write a bcrypt based password or does it continue to write the old style password? I suspect that the removal of the classic password encoder is intentional. Copying jglick in case he wants to comment on the change.
            jglick Jesse Glick added a comment -

            Removal of the very old password encoder was intentional. Should suffice to update your password while running 2.265 or older.

            jglick Jesse Glick added a comment - Removal of the very old password encoder was intentional. Should suffice to update your password while running 2.265 or older.
            markewaite Mark Waite made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            markewaite Mark Waite made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            markewaite Mark Waite made changes -
            Remote Link This issue links to "PR-4242 - Update 2.277.1 upgrade guide to describe the issue (Web Link)" [ 26617 ]
            worf_lt Worf added a comment -

            I can confirm that a change of the user's password in 2.263.4 will have its hash to be written in the new format and thus mitigates the issue.
            Nevertheless, I would suspect such a change to be stated in the Change Log (regardless of whether it was intentionally made). This way, administrators who are in the (hopefully) rare situation of maintaining a Jenkins installation whose users did not change their password since 2012 (horrible security practice) can take measures before users complain that they cannot log in anymore.

            Thanks for your feedback and efforts!

            worf_lt Worf added a comment - I can confirm that a change of the user's password in 2.263.4 will have its hash to be written in the new format and thus mitigates the issue. Nevertheless, I would suspect such a change to be stated in the Change Log (regardless of whether it was intentionally made). This way, administrators who are in the (hopefully) rare situation of maintaining a Jenkins installation whose users did not change their password since 2012 (horrible security practice) can take measures before users complain that they cannot log in anymore. Thanks for your feedback and efforts!
            markewaite Mark Waite added a comment - - edited

            Documented in Jenkins 2.277.1 upgrade guide.

            markewaite Mark Waite added a comment - - edited Documented in Jenkins 2.277.1 upgrade guide .
            markewaite Mark Waite made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Closed [ 6 ]
            danielbeck Daniel Beck made changes -
            Resolution Fixed [ 1 ]
            Status Closed [ 6 ] Reopened [ 4 ]
            danielbeck Daniel Beck made changes -
            Resolution Won't Fix [ 2 ]
            Status Reopened [ 4 ] Closed [ 6 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-64573 [ JENKINS-64573 ]

            People

              Unassigned Unassigned
              tarjei_asku Tarjei
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: