Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-12543

CliAuthenticator (username/password) called too late to parse arguments (like job names)

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Minor Minor
    • core
    • Linux AWS, 64 bit

      I am using Jenkins 1.448, with 'Jenkins own user database' and 'Project-
      based Matrix Authorization Strategy'. I am able to get CLI to work as
      anonymous with permissions Overall build, Job read, and Job build enabled using
      the following command:

      java -jar jenkins-cli.jar -s http://my.domain.com:8080/ build -c
      myJobName

      However, when I try to do this as an authenticated user:

      java -jar jenkins-cli.jar -s http://my.domain.com:8080/ build -c
      myJobName --username myUsername --password myPassword

      I get the following error:

      No such job 'myJobName'

      If I however have Build read for anonymous checked, it works even though I have it
      checked for my other username also.

          [JENKINS-12543] CliAuthenticator (username/password) called too late to parse arguments (like job names)

          evernat added a comment -

          Is it reproduced with a recent Jenkins version?

          evernat added a comment - Is it reproduced with a recent Jenkins version?

          I am still encountering this in Jenkins 1.515

          Richard Scorer added a comment - I am still encountering this in Jenkins 1.515

          TGF added a comment -

          I would reccommend to raise the priority, as this compromises security - it does noe make much sense to use the Authorization stragegy when we must have still enable the read job access to anonymous.

          TGF added a comment - I would reccommend to raise the priority, as this compromises security - it does noe make much sense to use the Authorization stragegy when we must have still enable the read job access to anonymous.

          sogabe added a comment -

          Try to enable "Job discover" to the user.

          sogabe added a comment - Try to enable "Job discover" to the user.

          All available access is given to the user, in order to run anything from command line I have to give specific access to the anonymous user.

          Richard Scorer added a comment - All available access is given to the user, in order to run anything from command line I have to give specific access to the anonymous user.

          Jesse Glick added a comment -

          JENKINS-14745 fixed a bug whereby no authentication was available, ever, while parsing CLI arguments such as job names. That fix worked for SSH authentication (-i …), which sets a transport authentication, but did not help with a CLIAuthenticator like --username … --password …. Meaning that unless the anonymous user can see your jobs (or computers, etc.), the command cannot be run.

          This is because CLICommand.main first sets the transport authentication, if any; then parses arguments, including both authenticator arguments like --username and specific command arguments; then asks the authenticator for its authentication, if any (also uses stored authentication from login here); then checks Overall/Read; and finally runs the command. But the parsing of regular command arguments (for e.g. get-job) often needs to be done while authenticated.

          The situation with CLIRegisterer, used by commands defined implicitly with @CLIMethod (like disable-job), is different, because the MethodBinder list is called after using the authenticator. So SSH authentication worked fine without any special help from JENKINS-14745. Unfortunately this code creates an authenticator but never configures it! So it always falls back to using transport authentication—and thus suffers from identical symptoms as regular CLI commands, though for a completely different reason.

          Jesse Glick added a comment - JENKINS-14745 fixed a bug whereby no authentication was available, ever, while parsing CLI arguments such as job names. That fix worked for SSH authentication ( -i … ), which sets a transport authentication, but did not help with a CLIAuthenticator like --username … --password … . Meaning that unless the anonymous user can see your jobs (or computers, etc.), the command cannot be run. This is because CLICommand.main first sets the transport authentication, if any; then parses arguments, including both authenticator arguments like --username and specific command arguments; then asks the authenticator for its authentication, if any (also uses stored authentication from login here); then checks Overall/Read; and finally runs the command. But the parsing of regular command arguments (for e.g. get-job ) often needs to be done while authenticated. The situation with CLIRegisterer , used by commands defined implicitly with @CLIMethod (like disable-job ), is different, because the MethodBinder list is called after using the authenticator. So SSH authentication worked fine without any special help from JENKINS-14745 . Unfortunately this code creates an authenticator but never configures it! So it always falls back to using transport authentication—and thus suffers from identical symptoms as regular CLI commands, though for a completely different reason.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/cli/handlers/GenericItemOptionHandler.java
          core/src/main/java/jenkins/model/Jenkins.java
          http://jenkins-ci.org/commit/jenkins/b261ab212270179fd286588534e9771dfb15f5fe
          Log:
          Clearer diagnosis for JENKINS-12543 and similar issues.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/cli/handlers/GenericItemOptionHandler.java core/src/main/java/jenkins/model/Jenkins.java http://jenkins-ci.org/commit/jenkins/b261ab212270179fd286588534e9771dfb15f5fe Log: Clearer diagnosis for JENKINS-12543 and similar issues.

          Jesse Glick added a comment -

          How would this be fixed? The second problem with CLIRegisterer seems easy enough: just use a ClassParser on the CliAuthenticator to configure it; an oversight, apparently. (Need to have tests which actually pass --username and --password to CLI commands rather than assuming transport authentication!)

          But the first problem, with CLICommand.main, seems tricky due to the design of CliAuthenticator, specifically the fact that there is no syntactic delineation in argv between the authenticator’s options and the command’s options and arguments. We want to half-parse the command to configure the authenticator; authenticate; then finish parsing the command with that authentication set.

          I thought of calling getCmdLineParser() twice, once just to configure the CliAuthenticator, then again with a fresh instance to really configure the command. But then GenericItemOptionHandler (for example) is going to throw a CmdLineException during the first parse, which will be thrown up and out of parseArgument. If --username and --password are both at the front of the command (rather than after the job names), then I guess these will already be set and we could just swallow the exception, but this seems fragile, and the behavior would be confusing since these arguments would work in some cases (where anonymous can see jobs) but not others (no anonymous read access).

          Or we could specially extract --username and related arguments and configure the authenticator with a special argument line. This will work only for a AbstractPasswordBasedSecurityRealm however; would break if someone introduced a security realm with another kind of authenticator. I am not sure if that would ever happen; the one plausible extension would be support API tokens rather than passwords, but this cannot be done as a CliTransportAuthenticator (since it would be command-line args, not transport) and cannot be done as a CliAuthenticator either (since it cuts across all security realms).

          Or we could somehow try to automatically determine which portions of the argument line belong to the authenticator, but this would be delving deeper into args4j than I dare at the moment.

          Jesse Glick added a comment - How would this be fixed? The second problem with CLIRegisterer seems easy enough: just use a ClassParser on the CliAuthenticator to configure it; an oversight, apparently. (Need to have tests which actually pass --username and --password to CLI commands rather than assuming transport authentication!) But the first problem, with CLICommand.main , seems tricky due to the design of CliAuthenticator , specifically the fact that there is no syntactic delineation in argv between the authenticator’s options and the command’s options and arguments. We want to half-parse the command to configure the authenticator; authenticate; then finish parsing the command with that authentication set. I thought of calling getCmdLineParser() twice, once just to configure the CliAuthenticator , then again with a fresh instance to really configure the command. But then GenericItemOptionHandler (for example) is going to throw a CmdLineException during the first parse, which will be thrown up and out of parseArgument . If --username and --password are both at the front of the command (rather than after the job names), then I guess these will already be set and we could just swallow the exception, but this seems fragile, and the behavior would be confusing since these arguments would work in some cases (where anonymous can see jobs) but not others (no anonymous read access). Or we could specially extract --username and related arguments and configure the authenticator with a special argument line. This will work only for a AbstractPasswordBasedSecurityRealm however; would break if someone introduced a security realm with another kind of authenticator. I am not sure if that would ever happen; the one plausible extension would be support API tokens rather than passwords, but this cannot be done as a CliTransportAuthenticator (since it would be command-line args, not transport) and cannot be done as a CliAuthenticator either (since it cuts across all security realms). Or we could somehow try to automatically determine which portions of the argument line belong to the authenticator, but this would be delving deeper into args4j than I dare at the moment.

          Hope we will have soon a fix for this old bug
          Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing

          Christophe LACOMBE added a comment - Hope we will have soon a fix for this old bug Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing

          Daniel Beck added a comment -

          Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing

          Is anything preventing anyone from just using SSH key authentication instead of username/password? (Real restrictions, not "My IT department won't allow it")

          https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI#JenkinsCLI-1.419andlater

          Daniel Beck added a comment - Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing Is anything preventing anyone from just using SSH key authentication instead of username/password? (Real restrictions, not "My IT department won't allow it") https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI#JenkinsCLI-1.419andlater

          Jesse Glick added a comment -

          Probably not, though setting up SSH keys is a bit awkward on Windows. Anyway username/password is a supported option, and this is clearly a bug.

          Jesse Glick added a comment - Probably not, though setting up SSH keys is a bit awkward on Windows. Anyway username/password is a supported option, and this is clearly a bug.

          Daniel Beck added a comment -

          Given the presence of an easy workaround in using SSH key authentication, adjusting issue priority accordingly.

          Daniel Beck added a comment - Given the presence of an easy workaround in using SSH key authentication, adjusting issue priority accordingly.

          Amit Ron added a comment -

          I'm getting this failure on get-job
          windows environment + ldap + Jenkins v1.600

          Amit Ron added a comment - I'm getting this failure on get-job windows environment + ldap + Jenkins v1.600

          Andrew Bayer added a comment -

          jglick - I'd lean towards the second approach, pulling the credentials from the argument list and authenticating using those if necessary.

          That said, I'm confused. Aren't we already populating the authenticator at https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/cli/CLICommand.java#L228?

          Andrew Bayer added a comment - jglick - I'd lean towards the second approach, pulling the credentials from the argument list and authenticating using those if necessary. That said, I'm confused. Aren't we already populating the authenticator at https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/cli/CLICommand.java#L228?

          Owen Wood added a comment - - edited

          A work around, using cURL, that doesn't require SSH Key Authentication:

          Example build:

          curl -X POST http://developer:developer@localhost:8080/job/test/build
          

          Obviously, replace:

          • developer:developer with your username:password
          • localhost:8080 with your Jenkins URL
          • test with your job name

          Example build with String parameter:

          curl -X POST http://developer:developer@localhost:8080/job/test/build --data-urlencode json='{"parameter": [{"name":"paramA", "value":"123"}]}'
          

          Obviously, replace:

          • developer:developer with your username:password
          • localhost:8080 with your Jenkins URL
          • test with your job name
          • paramA with your parameter name
          • 123 with your parameter value

          Owen Wood added a comment - - edited A work around, using cURL, that doesn't require SSH Key Authentication: Example build: curl -X POST http: //developer:developer@localhost:8080/job/test/build Obviously, replace: developer:developer with your username:password localhost:8080 with your Jenkins URL test with your job name Example build with String parameter: curl -X POST http: //developer:developer@localhost:8080/job/test/build --data-urlencode json= '{ "parameter" : [{ "name" : "paramA" , "value" : "123" }]}' Obviously, replace: developer:developer with your username:password localhost:8080 with your Jenkins URL test with your job name paramA with your parameter name 123 with your parameter value

          Daniel Beck added a comment -

          owood FWIW that is not the supported way to trigger a build with parameters. That one's documented on /job/foo/api.

          Daniel Beck added a comment - owood FWIW that is not the supported way to trigger a build with parameters. That one's documented on /job/foo/api.

          Jay Atwork added a comment - - edited

          This issue has broken CLI operation for me. I am using Windows and can not use anonymous read access on Jobs nor curl due to organizational restrictions in our prod environment and we need to be able to run builds via the CLI. I am on Jenkins 1.646, perhaps would updating to a newer version of Jenkins resolve this issue? I doubt it as this issue is still open. Is there a way to get the name of the jobs that the build method needs, since it is not what is provided from list-jobs?

          Jay Atwork added a comment - - edited This issue has broken CLI operation for me. I am using Windows and can not use anonymous read access on Jobs nor curl due to organizational restrictions in our prod environment and we need to be able to run builds via the CLI. I am on Jenkins 1.646, perhaps would updating to a newer version of Jenkins resolve this issue? I doubt it as this issue is still open. Is there a way to get the name of the jobs that the build method needs, since it is not what is provided from list-jobs?

          Heiko Nardmann added a comment - - edited

          I've come across this issue when migrating jobs from one Jenkins (test setup) to another:

          java -jar /opt/jenkins-cli.jar -s http://somehost:8080 list-jobs --username admin --password ...
          

          This works fine and gives e.g.

          JobA
          JobB
          JobC
          

          Now

          java -jar /opt/jenkins-cli.jar -s http://frsgtcshare01:8080 get-job JobA --username admin --password ...
          

          gives

          ERROR: No such job 'JobA'
          

          Or is this issue unrelated? The Jenkins involved is v2.13.

          With anonymous read access enabled get-job works.

          Heiko Nardmann added a comment - - edited I've come across this issue when migrating jobs from one Jenkins (test setup) to another: java -jar /opt/jenkins-cli.jar -s http://somehost:8080 list-jobs --username admin --password ... This works fine and gives e.g. JobA JobB JobC Now java -jar /opt/jenkins-cli.jar -s http://frsgtcshare01:8080 get-job JobA --username admin --password ... gives ERROR: No such job 'JobA' Or is this issue unrelated? The Jenkins involved is v2.13. With anonymous read access enabled get-job works.

          Matthias T added a comment - - edited

          I ran into this issue with Jenkins 2.19.2. Calling aforementioned

          java -jar /opt/jenkins-cli.jar -s http://jenkins-url.com get-job JobA --username admin --password ...
          

          gives the same "No such job" error.

          My workaround was:

          • Install the Extended Read Permission Plugin
          • Global security option "Matrix based security" has to be active
          • Right "Job - Extended Read" has to be activated for user anonymous
          • For "authenticated" users all rights can be activated
          • Use java -jar /opt/jenkins-cli.jar -s http://jenkins-url.com get-job JobA without credentials

          Matthias T added a comment - - edited I ran into this issue with Jenkins 2.19.2. Calling aforementioned java -jar /opt/jenkins-cli.jar -s http: //jenkins-url.com get-job JobA --username admin --password ... gives the same "No such job" error. My workaround was: Install the Extended Read Permission Plugin Global security option "Matrix based security" has to be active Right "Job - Extended Read" has to be activated for user anonymous For "authenticated" users all rights can be activated Use java -jar /opt/jenkins-cli.jar -s http://jenkins-url.com get-job JobA without credentials

          Jesse Glick added a comment -

          Right "Job - Extended Read" has to be activated for user anonymous

          Generally this is dangerous. Do not do this on a public-facing server.

          Jesse Glick added a comment - Right "Job - Extended Read" has to be activated for user anonymous Generally this is dangerous. Do not do this on a public-facing server.

          Jesse Glick added a comment -

          Again: the recommended workaround is to use SSH authentication.

          Jesse Glick added a comment - Again: the recommended workaround is to use SSH authentication.

          Jesse Glick added a comment -

          The situation with CLIRegisterer […] is different, because the MethodBinder list is called after using the authenticator. […] Unfortunately this code creates an authenticator but never configures it!

          FTR this was fixed in JENKINS-23988.

          Jesse Glick added a comment - The situation with CLIRegisterer […] is different, because the MethodBinder list is called after using the authenticator. […] Unfortunately this code creates an authenticator but never configures it! FTR this was fixed in JENKINS-23988 .

          Code changed in jenkins
          User: Jesse Glick
          Path:
          test/src/test/java/hudson/cli/CLIActionTest.java
          http://jenkins-ci.org/commit/jenkins/c2a5d8512356aca5532be83a5444b2f941e72510
          Log:
          Establishing baseline behavior of JENKINS-12543: no workaround when using Remoting transport other than SSH authentication.
          (Verifying that this affects only @Argument in CLICommand, not @CLIMethod.)
          With the new HTTP protocol in JENKINS-41745, API tokens may be used to set a transport authentication.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: test/src/test/java/hudson/cli/CLIActionTest.java http://jenkins-ci.org/commit/jenkins/c2a5d8512356aca5532be83a5444b2f941e72510 Log: Establishing baseline behavior of JENKINS-12543 : no workaround when using Remoting transport other than SSH authentication. (Verifying that this affects only @Argument in CLICommand, not @CLIMethod.) With the new HTTP protocol in JENKINS-41745 , API tokens may be used to set a transport authentication.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          cli/src/main/java/hudson/cli/CLI.java
          cli/src/main/java/hudson/cli/CLIConnectionFactory.java
          cli/src/main/resources/hudson/cli/client/Messages.properties
          core/src/main/java/hudson/cli/CLICommand.java
          core/src/main/java/hudson/cli/ClientAuthenticationCache.java
          core/src/main/java/hudson/cli/LoginCommand.java
          core/src/main/java/hudson/cli/LogoutCommand.java
          core/src/main/java/hudson/security/AbstractPasswordBasedSecurityRealm.java
          core/src/main/java/hudson/security/CliAuthenticator.java
          core/src/main/java/hudson/security/SecurityRealm.java
          core/src/main/resources/hudson/cli/Messages.properties
          test/src/test/java/hudson/cli/CLIActionTest.java
          http://jenkins-ci.org/commit/jenkins/12ae48ebb491b4f45ccb40ca8394bca8426f4e64
          Log:
          Deprecating -username/-password and login/logout in favor of new -auth option passing BASIC authentication to /cli endpoint.
          Simpler, does not rely on Remoting, allows use of API tokens, and bypasses JENKINS-12543.
          (You could actually do this before but only by embedding userinfo in the -s URL, especially awkward for usernames containing @.)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cli/src/main/java/hudson/cli/CLI.java cli/src/main/java/hudson/cli/CLIConnectionFactory.java cli/src/main/resources/hudson/cli/client/Messages.properties core/src/main/java/hudson/cli/CLICommand.java core/src/main/java/hudson/cli/ClientAuthenticationCache.java core/src/main/java/hudson/cli/LoginCommand.java core/src/main/java/hudson/cli/LogoutCommand.java core/src/main/java/hudson/security/AbstractPasswordBasedSecurityRealm.java core/src/main/java/hudson/security/CliAuthenticator.java core/src/main/java/hudson/security/SecurityRealm.java core/src/main/resources/hudson/cli/Messages.properties test/src/test/java/hudson/cli/CLIActionTest.java http://jenkins-ci.org/commit/jenkins/12ae48ebb491b4f45ccb40ca8394bca8426f4e64 Log: Deprecating - username/ -password and login/logout in favor of new -auth option passing BASIC authentication to /cli endpoint. Simpler, does not rely on Remoting, allows use of API tokens, and bypasses JENKINS-12543 . (You could actually do this before but only by embedding userinfo in the -s URL, especially awkward for usernames containing @.)

          Jesse Glick added a comment -

          Only affects deprecated Remoting-based CLI.

          Jesse Glick added a comment - Only affects deprecated Remoting-based CLI.

            Unassigned Unassigned
            mattfair Matt Fair
            Votes:
            15 Vote for this issue
            Watchers:
            31 Start watching this issue

              Created:
              Updated:
              Resolved: