Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-1304

repo-proxy pods are regularly restart

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: azure
    • Labels:
      None
    • Similar Issues:

      Description

      Repo-proxy pods have restarted way to often on k8s which is not a normal behavior.

      As we have defined a LivenessProbe and a ReadynessProbe in the deployment resource, it means that the HTTP check must be improved

      -> https://github.com/jenkins-infra/jenkins-infra/blob/staging/dist/profile/templates/kubernetes/resources/repo_proxy/deployment.yaml.erb


      k8s@jenkins-radish:~$ .bin/kubectl get pods -l app=repo-proxy --kubeconfig .kube/config
      NAME                          READY     STATUS    RESTARTS   AGE
      repo-proxy-1417053348-k40ie   1/1       Running   144        92d
      repo-proxy-1417053348-rk73p   0/1       Running   55         92d


        Attachments

          Activity

          Hide
          rtyler R. Tyler Croy added a comment -

          I don't understand what this ticket is saying. Is there a problem where the pods are regularly restarting and they shouldn't?

          Or is it, the pods should regularly restart, but currently we don't have supposed for that?

          Show
          rtyler R. Tyler Croy added a comment - I don't understand what this ticket is saying. Is there a problem where the pods are regularly restarting and they shouldn't? Or is it, the pods should regularly restart, but currently we don't have supposed for that?
          Hide
          olblak Olivier Vernin added a comment -

          I updated the description, I hope it's clearier

          Show
          olblak Olivier Vernin added a comment - I updated the description, I hope it's clearier
          Hide
          olblak Olivier Vernin added a comment -

          Waiting for this to PR to be in production.

          The previous http check generate false positive because the http request take too many time to respond.

           

          Show
          olblak Olivier Vernin added a comment - Waiting for this to PR to be in production. The previous http check generate false positive because the http request take too many time to respond.  
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: R. Tyler Croy
          Path:
          dist/profile/templates/kubernetes/resources/repo_proxy/deployment.yaml.erb
          http://jenkins-ci.org/commit/jenkins-infra/06a5b956638b1f2e462d50c8ac776faaf2ec332d
          Log:
          Merge pull request #830 from olblak/INFRA-1304

          INFRA-1304 Improve repo-proxy http check endpoint

          Compare: https://github.com/jenkins-infra/jenkins-infra/compare/e011b396f378...06a5b956638b

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: dist/profile/templates/kubernetes/resources/repo_proxy/deployment.yaml.erb http://jenkins-ci.org/commit/jenkins-infra/06a5b956638b1f2e462d50c8ac776faaf2ec332d Log: Merge pull request #830 from olblak/ INFRA-1304 INFRA-1304 Improve repo-proxy http check endpoint Compare: https://github.com/jenkins-infra/jenkins-infra/compare/e011b396f378...06a5b956638b
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Vernin Olivier
          Path:
          Gemfile
          Gemfile.lock
          Jenkinsfile
          Rakefile
          dist/profile/manifests/kubernetes/apply.pp
          dist/profile/manifests/kubernetes/backup.pp
          dist/profile/manifests/kubernetes/delete.pp
          dist/profile/manifests/kubernetes/kubectl.pp
          dist/profile/manifests/kubernetes/params.pp
          dist/profile/manifests/kubernetes/reload.pp
          dist/profile/manifests/kubernetes/resources/accountapp.pp
          dist/profile/manifests/kubernetes/resources/datadog.pp
          dist/profile/manifests/kubernetes/resources/fluentd.pp
          dist/profile/manifests/kubernetes/resources/lego.pp
          dist/profile/manifests/kubernetes/resources/nginx.pp
          dist/profile/manifests/kubernetes/resources/pluginsite.pp
          dist/profile/manifests/kubernetes/resources/repo_proxy.pp
          dist/profile/templates/kubernetes/backup.sh.erb
          dist/profile/templates/kubernetes/config.erb
          dist/profile/templates/kubernetes/resources/datadog/secret.yaml.erb
          dist/profile/templates/kubernetes/resources/repo_proxy/deployment.yaml.erb
          hieradata/common.yaml
          hieradata/env/production.yaml
          spec/classes/profile/kubernetes/accountapp_spec.rb
          spec/classes/profile/kubernetes/datadog_spec.rb
          spec/classes/profile/kubernetes/fluentd_spec.rb
          spec/classes/profile/kubernetes/kubectl_spec.rb
          spec/classes/profile/kubernetes/lego_spec.rb
          spec/classes/profile/kubernetes/nginx_spec.rb
          spec/classes/profile/kubernetes/pluginsite_spec.rb
          spec/classes/profile/kubernetes/repo_proxy_spec.rb
          spec/defines/profile/kubernetes/apply_spec.rb
          spec/defines/profile/kubernetes/backup_spec.rb
          spec/defines/profile/kubernetes/delete_spec.rb
          spec/defines/profile/kubernetes/reload_spec.rb
          spec/server/kubernetes/kubectl_spec.rb
          http://jenkins-ci.org/commit/jenkins-infra/70fadfecb22e5d97199b9bd189d2c571f69344a7
          Log:
          Merge pull request #831 from jenkins-infra/staging

          INFRA-1304 INFRA-1287 Improve Kubernetes configurations

          Compare: https://github.com/jenkins-infra/jenkins-infra/compare/58db8c700906...70fadfecb22e

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Vernin Olivier Path: Gemfile Gemfile.lock Jenkinsfile Rakefile dist/profile/manifests/kubernetes/apply.pp dist/profile/manifests/kubernetes/backup.pp dist/profile/manifests/kubernetes/delete.pp dist/profile/manifests/kubernetes/kubectl.pp dist/profile/manifests/kubernetes/params.pp dist/profile/manifests/kubernetes/reload.pp dist/profile/manifests/kubernetes/resources/accountapp.pp dist/profile/manifests/kubernetes/resources/datadog.pp dist/profile/manifests/kubernetes/resources/fluentd.pp dist/profile/manifests/kubernetes/resources/lego.pp dist/profile/manifests/kubernetes/resources/nginx.pp dist/profile/manifests/kubernetes/resources/pluginsite.pp dist/profile/manifests/kubernetes/resources/repo_proxy.pp dist/profile/templates/kubernetes/backup.sh.erb dist/profile/templates/kubernetes/config.erb dist/profile/templates/kubernetes/resources/datadog/secret.yaml.erb dist/profile/templates/kubernetes/resources/repo_proxy/deployment.yaml.erb hieradata/common.yaml hieradata/env/production.yaml spec/classes/profile/kubernetes/accountapp_spec.rb spec/classes/profile/kubernetes/datadog_spec.rb spec/classes/profile/kubernetes/fluentd_spec.rb spec/classes/profile/kubernetes/kubectl_spec.rb spec/classes/profile/kubernetes/lego_spec.rb spec/classes/profile/kubernetes/nginx_spec.rb spec/classes/profile/kubernetes/pluginsite_spec.rb spec/classes/profile/kubernetes/repo_proxy_spec.rb spec/defines/profile/kubernetes/apply_spec.rb spec/defines/profile/kubernetes/backup_spec.rb spec/defines/profile/kubernetes/delete_spec.rb spec/defines/profile/kubernetes/reload_spec.rb spec/server/kubernetes/kubectl_spec.rb http://jenkins-ci.org/commit/jenkins-infra/70fadfecb22e5d97199b9bd189d2c571f69344a7 Log: Merge pull request #831 from jenkins-infra/staging INFRA-1304 INFRA-1287 Improve Kubernetes configurations Compare: https://github.com/jenkins-infra/jenkins-infra/compare/58db8c700906...70fadfecb22e

            People

            Assignee:
            olblak Olivier Vernin
            Reporter:
            olblak Olivier Vernin
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: