Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-12543

CliAuthenticator (username/password) called too late to parse arguments (like job names)

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Minor
    • Resolution: Won't Fix
    • core
    • Linux AWS, 64 bit

    Description

      I am using Jenkins 1.448, with 'Jenkins own user database' and 'Project-
      based Matrix Authorization Strategy'. I am able to get CLI to work as
      anonymous with permissions Overall build, Job read, and Job build enabled using
      the following command:

      java -jar jenkins-cli.jar -s http://my.domain.com:8080/ build -c
      myJobName

      However, when I try to do this as an authenticated user:

      java -jar jenkins-cli.jar -s http://my.domain.com:8080/ build -c
      myJobName --username myUsername --password myPassword

      I get the following error:

      No such job 'myJobName'

      If I however have Build read for anonymous checked, it works even though I have it
      checked for my other username also.

      Attachments

        Issue Links

          Activity

            evernat evernat added a comment -

            Is it reproduced with a recent Jenkins version?

            evernat evernat added a comment - Is it reproduced with a recent Jenkins version?

            I am still encountering this in Jenkins 1.515

            rscorer Richard Scorer added a comment - I am still encountering this in Jenkins 1.515
            aguno TGF added a comment -

            I would reccommend to raise the priority, as this compromises security - it does noe make much sense to use the Authorization stragegy when we must have still enable the read job access to anonymous.

            aguno TGF added a comment - I would reccommend to raise the priority, as this compromises security - it does noe make much sense to use the Authorization stragegy when we must have still enable the read job access to anonymous.
            sogabe sogabe added a comment -

            Try to enable "Job discover" to the user.

            sogabe sogabe added a comment - Try to enable "Job discover" to the user.

            All available access is given to the user, in order to run anything from command line I have to give specific access to the anonymous user.

            rscorer Richard Scorer added a comment - All available access is given to the user, in order to run anything from command line I have to give specific access to the anonymous user.
            jglick Jesse Glick added a comment -

            JENKINS-14745 fixed a bug whereby no authentication was available, ever, while parsing CLI arguments such as job names. That fix worked for SSH authentication (-i …), which sets a transport authentication, but did not help with a CLIAuthenticator like --username … --password …. Meaning that unless the anonymous user can see your jobs (or computers, etc.), the command cannot be run.

            This is because CLICommand.main first sets the transport authentication, if any; then parses arguments, including both authenticator arguments like --username and specific command arguments; then asks the authenticator for its authentication, if any (also uses stored authentication from login here); then checks Overall/Read; and finally runs the command. But the parsing of regular command arguments (for e.g. get-job) often needs to be done while authenticated.

            The situation with CLIRegisterer, used by commands defined implicitly with @CLIMethod (like disable-job), is different, because the MethodBinder list is called after using the authenticator. So SSH authentication worked fine without any special help from JENKINS-14745. Unfortunately this code creates an authenticator but never configures it! So it always falls back to using transport authentication—and thus suffers from identical symptoms as regular CLI commands, though for a completely different reason.

            jglick Jesse Glick added a comment - JENKINS-14745 fixed a bug whereby no authentication was available, ever, while parsing CLI arguments such as job names. That fix worked for SSH authentication ( -i … ), which sets a transport authentication, but did not help with a CLIAuthenticator like --username … --password … . Meaning that unless the anonymous user can see your jobs (or computers, etc.), the command cannot be run. This is because CLICommand.main first sets the transport authentication, if any; then parses arguments, including both authenticator arguments like --username and specific command arguments; then asks the authenticator for its authentication, if any (also uses stored authentication from login here); then checks Overall/Read; and finally runs the command. But the parsing of regular command arguments (for e.g. get-job ) often needs to be done while authenticated. The situation with CLIRegisterer , used by commands defined implicitly with @CLIMethod (like disable-job ), is different, because the MethodBinder list is called after using the authenticator. So SSH authentication worked fine without any special help from JENKINS-14745 . Unfortunately this code creates an authenticator but never configures it! So it always falls back to using transport authentication—and thus suffers from identical symptoms as regular CLI commands, though for a completely different reason.

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/cli/handlers/GenericItemOptionHandler.java
            core/src/main/java/jenkins/model/Jenkins.java
            http://jenkins-ci.org/commit/jenkins/b261ab212270179fd286588534e9771dfb15f5fe
            Log:
            Clearer diagnosis for JENKINS-12543 and similar issues.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/cli/handlers/GenericItemOptionHandler.java core/src/main/java/jenkins/model/Jenkins.java http://jenkins-ci.org/commit/jenkins/b261ab212270179fd286588534e9771dfb15f5fe Log: Clearer diagnosis for JENKINS-12543 and similar issues.
            jglick Jesse Glick added a comment -

            How would this be fixed? The second problem with CLIRegisterer seems easy enough: just use a ClassParser on the CliAuthenticator to configure it; an oversight, apparently. (Need to have tests which actually pass --username and --password to CLI commands rather than assuming transport authentication!)

            But the first problem, with CLICommand.main, seems tricky due to the design of CliAuthenticator, specifically the fact that there is no syntactic delineation in argv between the authenticator’s options and the command’s options and arguments. We want to half-parse the command to configure the authenticator; authenticate; then finish parsing the command with that authentication set.

            I thought of calling getCmdLineParser() twice, once just to configure the CliAuthenticator, then again with a fresh instance to really configure the command. But then GenericItemOptionHandler (for example) is going to throw a CmdLineException during the first parse, which will be thrown up and out of parseArgument. If --username and --password are both at the front of the command (rather than after the job names), then I guess these will already be set and we could just swallow the exception, but this seems fragile, and the behavior would be confusing since these arguments would work in some cases (where anonymous can see jobs) but not others (no anonymous read access).

            Or we could specially extract --username and related arguments and configure the authenticator with a special argument line. This will work only for a AbstractPasswordBasedSecurityRealm however; would break if someone introduced a security realm with another kind of authenticator. I am not sure if that would ever happen; the one plausible extension would be support API tokens rather than passwords, but this cannot be done as a CliTransportAuthenticator (since it would be command-line args, not transport) and cannot be done as a CliAuthenticator either (since it cuts across all security realms).

            Or we could somehow try to automatically determine which portions of the argument line belong to the authenticator, but this would be delving deeper into args4j than I dare at the moment.

            jglick Jesse Glick added a comment - How would this be fixed? The second problem with CLIRegisterer seems easy enough: just use a ClassParser on the CliAuthenticator to configure it; an oversight, apparently. (Need to have tests which actually pass --username and --password to CLI commands rather than assuming transport authentication!) But the first problem, with CLICommand.main , seems tricky due to the design of CliAuthenticator , specifically the fact that there is no syntactic delineation in argv between the authenticator’s options and the command’s options and arguments. We want to half-parse the command to configure the authenticator; authenticate; then finish parsing the command with that authentication set. I thought of calling getCmdLineParser() twice, once just to configure the CliAuthenticator , then again with a fresh instance to really configure the command. But then GenericItemOptionHandler (for example) is going to throw a CmdLineException during the first parse, which will be thrown up and out of parseArgument . If --username and --password are both at the front of the command (rather than after the job names), then I guess these will already be set and we could just swallow the exception, but this seems fragile, and the behavior would be confusing since these arguments would work in some cases (where anonymous can see jobs) but not others (no anonymous read access). Or we could specially extract --username and related arguments and configure the authenticator with a special argument line. This will work only for a AbstractPasswordBasedSecurityRealm however; would break if someone introduced a security realm with another kind of authenticator. I am not sure if that would ever happen; the one plausible extension would be support API tokens rather than passwords, but this cannot be done as a CliTransportAuthenticator (since it would be command-line args, not transport) and cannot be done as a CliAuthenticator either (since it cuts across all security realms). Or we could somehow try to automatically determine which portions of the argument line belong to the authenticator, but this would be delving deeper into args4j than I dare at the moment.

            Hope we will have soon a fix for this old bug
            Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing

            clacombe Christophe LACOMBE added a comment - Hope we will have soon a fix for this old bug Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing
            danielbeck Daniel Beck added a comment -

            Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing

            Is anything preventing anyone from just using SSH key authentication instead of username/password? (Real restrictions, not "My IT department won't allow it")

            https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI#JenkinsCLI-1.419andlater

            danielbeck Daniel Beck added a comment - Having all jobs that need to be run from Jenkins-cli.jar to have read access for anonymous, is not a good thing Is anything preventing anyone from just using SSH key authentication instead of username/password? (Real restrictions, not "My IT department won't allow it") https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI#JenkinsCLI-1.419andlater
            jglick Jesse Glick added a comment -

            Probably not, though setting up SSH keys is a bit awkward on Windows. Anyway username/password is a supported option, and this is clearly a bug.

            jglick Jesse Glick added a comment - Probably not, though setting up SSH keys is a bit awkward on Windows. Anyway username/password is a supported option, and this is clearly a bug.
            danielbeck Daniel Beck added a comment -

            Given the presence of an easy workaround in using SSH key authentication, adjusting issue priority accordingly.

            danielbeck Daniel Beck added a comment - Given the presence of an easy workaround in using SSH key authentication, adjusting issue priority accordingly.
            amitron Amit Ron added a comment -

            I'm getting this failure on get-job
            windows environment + ldap + Jenkins v1.600

            amitron Amit Ron added a comment - I'm getting this failure on get-job windows environment + ldap + Jenkins v1.600
            abayer Andrew Bayer added a comment -

            jglick - I'd lean towards the second approach, pulling the credentials from the argument list and authenticating using those if necessary.

            That said, I'm confused. Aren't we already populating the authenticator at https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/cli/CLICommand.java#L228?

            abayer Andrew Bayer added a comment - jglick - I'd lean towards the second approach, pulling the credentials from the argument list and authenticating using those if necessary. That said, I'm confused. Aren't we already populating the authenticator at https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/cli/CLICommand.java#L228?
            owood Owen Wood added a comment - - edited

            A work around, using cURL, that doesn't require SSH Key Authentication:

            Example build:

            curl -X POST http://developer:developer@localhost:8080/job/test/build
            

            Obviously, replace:

            • developer:developer with your username:password
            • localhost:8080 with your Jenkins URL
            • test with your job name

            Example build with String parameter:

            curl -X POST http://developer:developer@localhost:8080/job/test/build --data-urlencode json='{"parameter": [{"name":"paramA", "value":"123"}]}'
            

            Obviously, replace:

            • developer:developer with your username:password
            • localhost:8080 with your Jenkins URL
            • test with your job name
            • paramA with your parameter name
            • 123 with your parameter value
            owood Owen Wood added a comment - - edited A work around, using cURL, that doesn't require SSH Key Authentication: Example build: curl -X POST http: //developer:developer@localhost:8080/job/test/build Obviously, replace: developer:developer with your username:password localhost:8080 with your Jenkins URL test with your job name Example build with String parameter: curl -X POST http: //developer:developer@localhost:8080/job/test/build --data-urlencode json= '{ "parameter" : [{ "name" : "paramA" , "value" : "123" }]}' Obviously, replace: developer:developer with your username:password localhost:8080 with your Jenkins URL test with your job name paramA with your parameter name 123 with your parameter value
            danielbeck Daniel Beck added a comment -

            owood FWIW that is not the supported way to trigger a build with parameters. That one's documented on /job/foo/api.

            danielbeck Daniel Beck added a comment - owood FWIW that is not the supported way to trigger a build with parameters. That one's documented on /job/foo/api.
            jhelix Jay Atwork added a comment - - edited

            This issue has broken CLI operation for me. I am using Windows and can not use anonymous read access on Jobs nor curl due to organizational restrictions in our prod environment and we need to be able to run builds via the CLI. I am on Jenkins 1.646, perhaps would updating to a newer version of Jenkins resolve this issue? I doubt it as this issue is still open. Is there a way to get the name of the jobs that the build method needs, since it is not what is provided from list-jobs?

            jhelix Jay Atwork added a comment - - edited This issue has broken CLI operation for me. I am using Windows and can not use anonymous read access on Jobs nor curl due to organizational restrictions in our prod environment and we need to be able to run builds via the CLI. I am on Jenkins 1.646, perhaps would updating to a newer version of Jenkins resolve this issue? I doubt it as this issue is still open. Is there a way to get the name of the jobs that the build method needs, since it is not what is provided from list-jobs?
            heiko_nardmann Heiko Nardmann added a comment - - edited

            I've come across this issue when migrating jobs from one Jenkins (test setup) to another:

            java -jar /opt/jenkins-cli.jar -s http://somehost:8080 list-jobs --username admin --password ...
            

            This works fine and gives e.g.

            JobA
            JobB
            JobC
            

            Now

            java -jar /opt/jenkins-cli.jar -s http://frsgtcshare01:8080 get-job JobA --username admin --password ...
            

            gives

            ERROR: No such job 'JobA'
            

            Or is this issue unrelated? The Jenkins involved is v2.13.

            With anonymous read access enabled get-job works.

            heiko_nardmann Heiko Nardmann added a comment - - edited I've come across this issue when migrating jobs from one Jenkins (test setup) to another: java -jar /opt/jenkins-cli.jar -s http://somehost:8080 list-jobs --username admin --password ... This works fine and gives e.g. JobA JobB JobC Now java -jar /opt/jenkins-cli.jar -s http://frsgtcshare01:8080 get-job JobA --username admin --password ... gives ERROR: No such job 'JobA' Or is this issue unrelated? The Jenkins involved is v2.13. With anonymous read access enabled get-job works.
            scaronthesky Matthias T added a comment - - edited

            I ran into this issue with Jenkins 2.19.2. Calling aforementioned

            java -jar /opt/jenkins-cli.jar -s http://jenkins-url.com get-job JobA --username admin --password ...
            

            gives the same "No such job" error.

            My workaround was:

            • Install the Extended Read Permission Plugin
            • Global security option "Matrix based security" has to be active
            • Right "Job - Extended Read" has to be activated for user anonymous
            • For "authenticated" users all rights can be activated
            • Use java -jar /opt/jenkins-cli.jar -s http://jenkins-url.com get-job JobA without credentials
            scaronthesky Matthias T added a comment - - edited I ran into this issue with Jenkins 2.19.2. Calling aforementioned java -jar /opt/jenkins-cli.jar -s http: //jenkins-url.com get-job JobA --username admin --password ... gives the same "No such job" error. My workaround was: Install the Extended Read Permission Plugin Global security option "Matrix based security" has to be active Right "Job - Extended Read" has to be activated for user anonymous For "authenticated" users all rights can be activated Use java -jar /opt/jenkins-cli.jar -s http://jenkins-url.com get-job JobA without credentials
            jglick Jesse Glick added a comment -

            Right "Job - Extended Read" has to be activated for user anonymous

            Generally this is dangerous. Do not do this on a public-facing server.

            jglick Jesse Glick added a comment - Right "Job - Extended Read" has to be activated for user anonymous Generally this is dangerous. Do not do this on a public-facing server.
            jglick Jesse Glick added a comment -

            Again: the recommended workaround is to use SSH authentication.

            jglick Jesse Glick added a comment - Again: the recommended workaround is to use SSH authentication.
            jglick Jesse Glick added a comment -

            The situation with CLIRegisterer […] is different, because the MethodBinder list is called after using the authenticator. […] Unfortunately this code creates an authenticator but never configures it!

            FTR this was fixed in JENKINS-23988.

            jglick Jesse Glick added a comment - The situation with CLIRegisterer […] is different, because the MethodBinder list is called after using the authenticator. […] Unfortunately this code creates an authenticator but never configures it! FTR this was fixed in JENKINS-23988 .

            Code changed in jenkins
            User: Jesse Glick
            Path:
            test/src/test/java/hudson/cli/CLIActionTest.java
            http://jenkins-ci.org/commit/jenkins/c2a5d8512356aca5532be83a5444b2f941e72510
            Log:
            Establishing baseline behavior of JENKINS-12543: no workaround when using Remoting transport other than SSH authentication.
            (Verifying that this affects only @Argument in CLICommand, not @CLIMethod.)
            With the new HTTP protocol in JENKINS-41745, API tokens may be used to set a transport authentication.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: test/src/test/java/hudson/cli/CLIActionTest.java http://jenkins-ci.org/commit/jenkins/c2a5d8512356aca5532be83a5444b2f941e72510 Log: Establishing baseline behavior of JENKINS-12543 : no workaround when using Remoting transport other than SSH authentication. (Verifying that this affects only @Argument in CLICommand, not @CLIMethod.) With the new HTTP protocol in JENKINS-41745 , API tokens may be used to set a transport authentication.

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cli/src/main/java/hudson/cli/CLI.java
            cli/src/main/java/hudson/cli/CLIConnectionFactory.java
            cli/src/main/resources/hudson/cli/client/Messages.properties
            core/src/main/java/hudson/cli/CLICommand.java
            core/src/main/java/hudson/cli/ClientAuthenticationCache.java
            core/src/main/java/hudson/cli/LoginCommand.java
            core/src/main/java/hudson/cli/LogoutCommand.java
            core/src/main/java/hudson/security/AbstractPasswordBasedSecurityRealm.java
            core/src/main/java/hudson/security/CliAuthenticator.java
            core/src/main/java/hudson/security/SecurityRealm.java
            core/src/main/resources/hudson/cli/Messages.properties
            test/src/test/java/hudson/cli/CLIActionTest.java
            http://jenkins-ci.org/commit/jenkins/12ae48ebb491b4f45ccb40ca8394bca8426f4e64
            Log:
            Deprecating -username/-password and login/logout in favor of new -auth option passing BASIC authentication to /cli endpoint.
            Simpler, does not rely on Remoting, allows use of API tokens, and bypasses JENKINS-12543.
            (You could actually do this before but only by embedding userinfo in the -s URL, especially awkward for usernames containing @.)

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cli/src/main/java/hudson/cli/CLI.java cli/src/main/java/hudson/cli/CLIConnectionFactory.java cli/src/main/resources/hudson/cli/client/Messages.properties core/src/main/java/hudson/cli/CLICommand.java core/src/main/java/hudson/cli/ClientAuthenticationCache.java core/src/main/java/hudson/cli/LoginCommand.java core/src/main/java/hudson/cli/LogoutCommand.java core/src/main/java/hudson/security/AbstractPasswordBasedSecurityRealm.java core/src/main/java/hudson/security/CliAuthenticator.java core/src/main/java/hudson/security/SecurityRealm.java core/src/main/resources/hudson/cli/Messages.properties test/src/test/java/hudson/cli/CLIActionTest.java http://jenkins-ci.org/commit/jenkins/12ae48ebb491b4f45ccb40ca8394bca8426f4e64 Log: Deprecating - username/ -password and login/logout in favor of new -auth option passing BASIC authentication to /cli endpoint. Simpler, does not rely on Remoting, allows use of API tokens, and bypasses JENKINS-12543 . (You could actually do this before but only by embedding userinfo in the -s URL, especially awkward for usernames containing @.)
            jglick Jesse Glick added a comment -

            Only affects deprecated Remoting-based CLI.

            jglick Jesse Glick added a comment - Only affects deprecated Remoting-based CLI.

            People

              Unassigned Unassigned
              mattfair Matt Fair
              Votes:
              15 Vote for this issue
              Watchers:
              31 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: