Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64158

Jenkins loses connection to ldap same time every day on all 3 environments for same length of time

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • core
    • None
    • Jenkins Version 2.249.1, chrome browser,

      For 3 days in a row now, all of a sudden we can't login to Jenkins. Invalid Username or Password is error received. This has now happened on all 3 of our servers (dev, qa, prod) at the exact same time every day and for the same length of time, about 1-1.5 hours each day it won't let us log in. I try making some changes and troubleshooting, it doesn't work. Magically all of a sudden you can start logging in again after that 1-1.5 hours. This is the error messages I'm receiving in the log file:

      06-Nov-2020 08:44:07.962 WARNING [Handling POST /jenkins/j_acegi_security_check from 10.231.112.168 : https-jsse-nio-28443-exec-2] hudson.security.LDAPSecurityRealm$LDAPAuthenticationManager.authenticate Failed communication with ldap server.
      org.acegisecurity.AuthenticationServiceException: LdapCallback;LDAP response read timed out, timeout used:10000ms.; nested exception is javax.naming.NamingException: LDAP response read timed out, timeout used:10000ms.; remaining name ''; nested exception is org.acegisecurity.ldap.LdapDataAccessException: LdapCallback;LDAP response read timed out, timeout used:10000ms.; nested exception is javax.naming.NamingException: LDAP response read timed out, timeout used:10000ms.; remaining name ''
      at org.acegisecurity.providers.ldap.LdapAuthenticationProvider.retrieveUser(LdapAuthenticationProvider.java:238)
      at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:122)
      at org.acegisecurity.providers.ProviderManager.doAuthentication(ProviderManager.java:200)
      at org.acegisecurity.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:47)
      at hudson.security.LDAPSecurityRealm$LDAPAuthenticationManager.authenticate(LDAPSecurityRealm.java:1020)
      at org.acegisecurity.ui.webapp.AuthenticationProcessingFilter.attemptAuthentication(AuthenticationProcessingFilter.java:74)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:252)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:36)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
      at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
      at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
      at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
      at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
      at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: org.acegisecurity.ldap.LdapDataAccessException: LdapCallback;LDAP response read timed out, timeout used:10000ms.; nested exception is javax.naming.NamingException: LDAP response read timed out, timeout used:10000ms.; remaining name ''
      at org.acegisecurity.ldap.LdapTemplate$LdapExceptionTranslator.translate(LdapTemplate.java:295)
      at org.acegisecurity.ldap.LdapTemplate.execute(LdapTemplate.java:128)
      at org.acegisecurity.ldap.LdapTemplate.searchForSingleAttributeValues(LdapTemplate.java:227)
      at org.acegisecurity.providers.ldap.populator.DefaultLdapAuthoritiesPopulator.getGroupMembershipRoles(DefaultLdapAuthoritiesPopulator.java:228)
      at hudson.security.LDAPSecurityRealm$AuthoritiesPopulatorImpl.getGroupMembershipRoles(LDAPSecurityRealm.java:1513)
      at org.acegisecurity.providers.ldap.populator.DefaultLdapAuthoritiesPopulator.getGrantedAuthorities(DefaultLdapAuthoritiesPopulator.java:181)
      at jenkins.security.plugins.ldap.FromGroupSearchLDAPGroupMembershipStrategy.getGrantedAuthorities(FromGroupSearchLDAPGroupMembershipStrategy.java:81)
      at hudson.security.LDAPSecurityRealm$WrappedAuthoritiesPopulator.getGrantedAuthorities(LDAPSecurityRealm.java:1466)
      at org.acegisecurity.providers.ldap.LdapAuthenticationProvider.createUserDetails(LdapAuthenticationProvider.java:203)
      at org.acegisecurity.providers.ldap.LdapAuthenticationProvider.retrieveUser(LdapAuthenticationProvider.java:235)
      ... 44 more
      Caused by: javax.naming.NamingException: LDAP response read timed out, timeout used:10000ms.; remaining name ''
      at com.sun.jndi.ldap.Connection.readReply(Connection.java:507)
      at com.sun.jndi.ldap.LdapClient.getSearchReply(LdapClient.java:632)
      at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:555)
      at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1985)
      at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1844)
      at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1769)
      at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1786)
      at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:418)
      at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:396)
      at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:378)
      at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:286)
      at org.acegisecurity.ldap.LdapTemplate$1SingleAttributeSearchCallback.doInDirContext(LdapTemplate.java:202)
      at org.acegisecurity.ldap.LdapTemplate.execute(LdapTemplate.java:126)
      ... 52 more

            Unassigned Unassigned
            brumms Sara Brumm
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: