Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22727

AD plugin times out for large user/group membership

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • AD plugin v1.37, Jenkins 1.56, Running as a service, Windows Server 2012 R2, Active Directory with multiple domains

      Logs show that the plugin has correctly matched my username against the right DC and authenticated correctly. All my groups are printed along with some additional ldap content. Then there's a two minute gap in the logs around Stage 2:

      Apr 22, 2014 11:46:27 PM FINE hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider
      Stage 2: looking up via memberOf
      Apr 22, 2014 11:48:27 PM FINE hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider
      CN=Jon Wiswall,OU=<ou>,OU=<ou>,DC=<dc>,DC=<dc>,DC=<dc>,DC=<dc> is a member of cn: <group name>
      

      After the 2-minute break the log prints the first 20 or so of my ~150 group memberships.

      Looks like the LDAP server gives up at this point:

      Failed to retrieve user information for <username>
      javax.naming.TimeLimitExceededException: [LDAP: error code 3 - Timelimit Exceeded]; remaining name 'DC=<dc>,DC=<dc>,DC=<dc>,DC=<dc>'
      	at com.sun.jndi.ldap.LdapCtx.mapErrorCode(Unknown Source)
      	at com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source)
      	at com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source)
      	at com.sun.jndi.ldap.LdapNamingEnumeration.getNextBatch(Unknown Source)
      	at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(Unknown Source)
      	at com.sun.jndi.ldap.LdapNamingEnumeration.hasMore(Unknown Source)
      	at hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider.parseMembers(ActiveDirectoryUnixAuthenticationProvider.java:456)
      

      ... which then fails the Jenkins login with an authentication failed message.

      I'm sure this is to do with our large Active Directory deployment.

      Could the plugin only check the username/pw combo, and then if matrix or project-based security is enabled, check if the named groups are present? The initial auth step (which dumps all the groups anyhow) is super fast.

      (Note: marked bug as 'minor' but I can't really point my team at my Jenkins instance until this works.)

          [JENKINS-22727] AD plugin times out for large user/group membership

          Greg R added a comment -

          +1 , same issue on our Windows 2008R2 Jenkins installation.

          On our RedHat Linux Jenkins installation, in Active Directory settings under Configure Global Security...it allows for 5 inputs, one of those being Bind DN and Bind Password. When we set this to the service account user, it fixed this. (I assume by filtering it's search)

          But on the Windows server, this option isn't available and I don't see anywhere else to enter Bind DN and narrow down my search, avoiding the timeouts. I tried manually copying value into Config.xml, but that didn't work either.

          Greg R added a comment - +1 , same issue on our Windows 2008R2 Jenkins installation. On our RedHat Linux Jenkins installation, in Active Directory settings under Configure Global Security...it allows for 5 inputs, one of those being Bind DN and Bind Password. When we set this to the service account user, it fixed this. (I assume by filtering it's search) But on the Windows server, this option isn't available and I don't see anywhere else to enter Bind DN and narrow down my search, avoiding the timeouts. I tried manually copying value into Config.xml, but that didn't work either.

          Mark Syms added a comment -

          Catching the LimitExceededException in chainGroupLookup and returning false seems to work but is a bit brute force.

          Something that did some deferred lookups on demand would probably be a better solution but I'm not sure how that would map into the architecture.

          Mark Syms added a comment - Catching the LimitExceededException in chainGroupLookup and returning false seems to work but is a bit brute force. Something that did some deferred lookups on demand would probably be a better solution but I'm not sure how that would map into the architecture.

          pixman20 added a comment -

          I am also experiencing this issue. Whenever I restart Jenkins with AD and multiple domains I need to login once to let it timeout/fail and then wait 15+ minutes before logging in again to have it work.
          This makes restarting Jenkins a very painful process.

          pixman20 added a comment - I am also experiencing this issue. Whenever I restart Jenkins with AD and multiple domains I need to login once to let it timeout/fail and then wait 15+ minutes before logging in again to have it work. This makes restarting Jenkins a very painful process.

          Daniel Beck added a comment -

          Try connecting to the global catalog if you're not.

          There's an advanced option on /configureSecurity named 'Group Looking Strategy', try both values to see if that changes anything.

          Daniel Beck added a comment - Try connecting to the global catalog if you're not. There's an advanced option on /configureSecurity named 'Group Looking Strategy', try both values to see if that changes anything.

          M Chon added a comment -

          I also see this when running base Jenkins version 1.580.2 on Ubuntu, and attempting to upgrade AD plugin from 1.33 to 1.39.
          After upgrading the plugin and attempting to log in, I get the below error.
          The only way to recover is to manually de-install the 1.39 plugin and manually restore the 1.33 plugin.
          Then I can log in again.

          Jan 21, 2015 2:20:56 PM hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider retrieveUser
          WARNING: Failed to retrieve user information for XXXXX
          javax.naming.TimeLimitExceededException: [LDAP: error code 3 - Timelimit Exceeded]; remaining name 'DC=XXXXX,DC=XXXXX'
          at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3143)
          at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033)
          at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2840)
          at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1849)
          at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1772)
          at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1789)

          M Chon added a comment - I also see this when running base Jenkins version 1.580.2 on Ubuntu, and attempting to upgrade AD plugin from 1.33 to 1.39. After upgrading the plugin and attempting to log in, I get the below error. The only way to recover is to manually de-install the 1.39 plugin and manually restore the 1.33 plugin. Then I can log in again. Jan 21, 2015 2:20:56 PM hudson.plugins.active_directory.ActiveDirectoryUnixAuthenticationProvider retrieveUser WARNING: Failed to retrieve user information for XXXXX javax.naming.TimeLimitExceededException: [LDAP: error code 3 - Timelimit Exceeded] ; remaining name 'DC=XXXXX,DC=XXXXX' at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3143) at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033) at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2840) at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1849) at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1772) at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1789)

            Unassigned Unassigned
            jdwiswall Jon Wiswall
            Votes:
            4 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: