Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59967

getting 403 for invalid crumb on trigger remote job

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • Jenkins 2.198
      Parameterized-Remote-Trigger 3.1.0

      both jenkins masters running in Kubernetes

      When I trigger a job from a pipeline with:

      triggerRemoteJob([
                      auth                   : CredentialsAuth(credentials: 'something'),
                      job                    : jobName,
                      remoteJenkinsName      : 'Prod Jenkins',
                      useCrumbCache          : true,
                      useJobInfoCache        : true,
                      blockBuildUntilComplete: false,
                      parameters             : "VERSION=1.2.3",
      ])
      

      I get a 403 when it is trying to do crsf crumb stuff. On remote jenkins log I see:

      WARNING	hudson.security.csrf.CrumbFilter#doFilter: Found invalid crumb 163***********************************************************3c.  Will check remaining parameters for a valid one...
      WARNING	hudson.security.csrf.CrumbFilter#doFilter: No valid crumb was included in request for /job/prod_folder/job/some_repo/job/default/buildWithParameters by Prod-Build-User. Returning 403.
      

      In the build log on the non-remote Jenkins I see output that makes me think it's calling the code to get the crsf.

      CSRF protection is enabled on the remote server.
      

      Which is from this line:
      https://github.com/jenkinsci/parameterized-remote-trigger-plugin/blob/Parameterized-Remote-Trigger-3.1.0/src/main/java/org/jenkinsci/plugins/ParameterizedRemoteTrigger/utils/HttpHelper.java#L253

      But for some reason the one it gets and sends is not working. I've tried with the csrf caching turned on and turned off. It doesn't make any difference.

      There was recently a change to how Jenkins does something with CSRF in 2.190, JENKINS-58734

      Not sure if that's playing into this or not

          [JENKINS-59967] getting 403 for invalid crumb on trigger remote job

          Dan Bateman added a comment -

          Facing the same issue on v2.190.1 of Jenkins LTS Release and Parameterized Remote Trigger Plugin v3.1.0

          https://jenkins.io/security/advisory/2019-07-17/#SECURITY-626 appears to have broken the remote trigger plugin. 

          We will attempt to use the strict crumb issuer plugin to disable the session id and see if that works.

           

          Dan Bateman added a comment - Facing the same issue on v2.190.1 of Jenkins LTS Release and Parameterized Remote Trigger Plugin v3.1.0 https://jenkins.io/security/advisory/2019-07-17/#SECURITY-626  appears to have broken the remote trigger plugin.  We will attempt to use the strict crumb issuer plugin to disable the session id and see if that works.  

          How about set the crumb cache to false?

          KaiHsiang Chang added a comment - How about set the crumb cache to false?

          Dan Bateman added a comment -

          cashlalala nope, that didn't seem to work. If I disable CSF completely, it works, but I will look at strict crumb issuer instead until this is resolved.

          Dan Bateman added a comment - cashlalala nope, that didn't seem to work. If I disable CSF completely, it works, but I will look at strict crumb issuer instead until this is resolved.

          Dan Bateman added a comment -

          Also, if anyone interested in the groovy for setting this up, strict crumb issuer - as couldn't find this documented anywhere, here it is (this worked at time of writing):

          import hudson.security.csrf.DefaultCrumbIssuer
          import hudson.security.csrf.CrumbIssuer
          import org.jenkinsci.plugins.strictcrumbissuer.StrictCrumbIssuer
          import jenkins.model.Jenkins
          
          def instance = Jenkins.instance
          boolean checkClientIP = false
          boolean checkSameSource = false
          boolean checkOnlyLocalPath = false
          boolean checkSessionMatch = false
          int hoursValid = 1
          boolean xorMasking = true
          
          instance.setCrumbIssuer(new StrictCrumbIssuer(checkClientIP, checkSameSource, checkOnlyLocalPath, checkSessionMatch, hoursValid, xorMasking))
          
          instance.save()
          

          Dan Bateman added a comment - Also, if anyone interested in the groovy for setting this up, strict crumb issuer - as couldn't find this documented anywhere, here it is (this worked at time of writing): import hudson.security.csrf.DefaultCrumbIssuer import hudson.security.csrf.CrumbIssuer import org.jenkinsci.plugins.strictcrumbissuer.StrictCrumbIssuer import jenkins.model.Jenkins def instance = Jenkins.instance boolean checkClientIP = false boolean checkSameSource = false boolean checkOnlyLocalPath = false boolean checkSessionMatch = false int hoursValid = 1 boolean xorMasking = true instance.setCrumbIssuer(new StrictCrumbIssuer(checkClientIP, checkSameSource, checkOnlyLocalPath, checkSessionMatch, hoursValid, xorMasking)) instance.save()

          Mike Huang added a comment -

          Would definitely prefer an update to the plugin rather than disabling parts of the CSRF check.

          I think this is more than minor as the plugin would pretty much fail on any recent Jenkins remote version?

          Mike Huang added a comment - Would definitely prefer an update to the plugin rather than disabling parts of the CSRF check. I think this is more than minor as the plugin would pretty much fail on any recent Jenkins remote version?

          Strajan Sebastian added a comment - - edited

          found the same issue. after an upgrade can't use the plugin because CSRF can't be disabled anymore.

          Using Jenkins ver. 2.222.3

           

          fixed with: https://plugins.jenkins.io/strict-crumb-issuer/ + Configure Global Security -> CSRF Protection -> set strict crumb issuer and disable 'Check the session ID'.

          Strajan Sebastian added a comment - - edited found the same issue. after an upgrade can't use the plugin because CSRF can't be disabled anymore. Using Jenkins ver. 2.222.3   fixed with:  https://plugins.jenkins.io/strict-crumb-issuer/  + Configure Global Security -> CSRF Protection -> set strict crumb issuer and disable 'Check the session ID'.

          strajan_sebastian : I wasn't able to get this to work, with your exact setup (on both Jenkins services).

          I do get a 403 error back on the calling Jenkins.

          What Authorization strategy are you using (our setup use role-based) for instance ?

          Anything else you could suggest we look at?

          Thanks much in advance.

          Patrice Matignon added a comment - strajan_sebastian : I wasn't able to get this to work, with your exact setup (on both Jenkins services). I do get a 403 error back on the calling Jenkins. What Authorization strategy are you using (our setup use role-based) for instance ? Anything else you could suggest we look at? Thanks much in advance.

          We are using AD integration and for the security strategy 'logged in can do anything' (with anon users can read data unchecked).

          Try to see if changing the authorization strategy works.

          Strajan Sebastian added a comment - We are using AD integration and for the security strategy 'logged in can do anything' (with anon users can read data unchecked). Try to see if changing the authorization strategy works.

            cashlalala KaiHsiang Chang
            b_dean Ben Dean
            Votes:
            6 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated: