Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-67862

Getting error [org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokens]

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • token-macro-plugin
    • None
    • 280.v97a_82642793c

      After switching to version 277.v7c8f82a_d66b_3 we are seeing the following error without any further information:

      13:51:58.044 Exception caught evaluating condition: [org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokens], action = [Fail the build]

      There is no easy way to get find the root cause of this error without getting more details. 

          [JENKINS-67862] Getting error [org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokens]

          Alex Earl added a comment -

          Do you know what context you are getting this in? Build step? Email? Etc?

          Alex Earl added a comment - Do you know what context you are getting this in? Build step? Email? Etc?

          Boon Elschenbroich added a comment - - edited

          It looks like it is before the build step. I can see the prebuild steps being evaluated and then I just see this error almost immediately after starting the job without any more context. 

          This is the log with the new release:

          13:51:57.971 Run condition [Not] preventing prebuild for step [BuilderChain]
          13:51:57.971 Run condition [Boolean condition] enabling prebuild for step [Invoke top-level Maven targets]
          13:51:58.044 Exception caught evaluating condition: [org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokens], action = [Fail the build]

          I switched back to the previous release of the token-macro plugin. And this is what I see instead (working):

          16:56:20.320 Run condition [Not] preventing prebuild for step [BuilderChain]
          16:56:20.320 Run condition [Boolean condition] enabling prebuild for step [Invoke top-level Maven targets]
          16:56:20.321 New run name is '#13528 deploy 2.3.0.0.969 to xxxxx'
          16:56:20.423 Regular expression run condition: Expression=[^(?:.*/)?master$|^(?:.*/)?(feature|release)/.*], Label=[2.3.0.0.969]
          16:56:20.424 Run condition [Not] enabling perform for step [BuilderChain]

          Looks like this conditional step is failing that is using a regular expression 

          Boon Elschenbroich added a comment - - edited It looks like it is before the build step. I can see the prebuild steps being evaluated and then I just see this error almost immediately after starting the job without any more context.  This is the log with the new release: 13:51:57.971 Run condition [Not] preventing prebuild for step [BuilderChain] 13:51:57.971 Run condition [Boolean condition] enabling prebuild for step [Invoke top-level Maven targets] 13:51:58.044 Exception caught evaluating condition: [org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokens] , action =  [Fail the build] I switched back to the previous release of the token-macro plugin. And this is what I see instead (working): 16:56:20.320 Run condition [Not] preventing prebuild for step [BuilderChain] 16:56:20.320 Run condition [Boolean condition] enabling prebuild for step [Invoke top-level Maven targets] 16:56:20.321 New run name is '#13528 deploy 2.3.0.0.969 to xxxxx' 16:56:20.423 Regular expression run condition: Expression= [^(?:.*/)?master$|^(?:.*/)?(feature|release)/.*] , Label= [2.3.0.0.969] 16:56:20.424 Run condition [Not] enabling perform for step  [BuilderChain] Looks like this conditional step is failing that is using a regular expression 

          Alex Earl added a comment - - edited

          Can you show what the token you are using?

          Alex Earl added a comment - - edited Can you show what the token you are using?

          We are not using any out of the ordinary tokens. These are the ones that we are using in this particular build job:

          #${BUILD_NUMBER} deploy ${DEPLOY_FROM} to ${INSTANCE_NAME}

          ${ENV,var="DEPLOY_FROM"}

          ${CWA_VERSION}

          ${BUILD_COMPLETE_REACTOR}

          ${DEPLOYMENT_ITEMS}

          And so on. So all tokens are quite simple. But, after doing some investigation I have the strong feeling that the conditional step, in which we use this regex, is causing the build to fail:

          ^(?:.*/)?master$|^(?:.*/)?(feature|release)/.* 

          This seems to be evaluated as a token because of the $ sign. I ran a quick unit test in the token macro plugin and it shows the same error message with the regex $ sign in place. Without it, there is no error.

           

          Boon Elschenbroich added a comment - We are not using any out of the ordinary tokens. These are the ones that we are using in this particular build job: #${BUILD_NUMBER} deploy ${DEPLOY_FROM} to ${INSTANCE_NAME} ${ENV,var="DEPLOY_FROM"} ${CWA_VERSION} ${BUILD_COMPLETE_REACTOR} ${DEPLOYMENT_ITEMS} And so on. So all tokens are quite simple. But, after doing some investigation I have the strong feeling that the conditional step, in which we use this regex, is causing the build to fail: ^(?:.*/)?master$|^(?:.*/)?(feature|release)/.* This seems to be evaluated as a token because of the $ sign. I ran a quick unit test in the token macro plugin and it shows the same error message with the regex $ sign in place. Without it, there is no error.  

          This is the test I executed:

          assertEquals("^(?:.*/)?master$|^(?:.*/)?(feature|release)/.* ",TokenMacro.expandAll(b, TaskListener.NULL, "^(?:.*/)?master$|^(?:.*/)?(feature|release)/.* ")); 

          and it gave me this error:

          org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokensorg.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokens
           at org.jenkinsci.plugins.tokenmacro.Parser.parse(Parser.java:97) at org.jenkinsci.plugins.tokenmacro.Parser.process(Parser.java:75) at org.jenkinsci.plugins.tokenmacro.Parser.process(Parser.java:68) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expand(TokenMacro.java:199) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:237) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:211) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:203) at org.jenkinsci.plugins.tokenmacro.TokenMacroTest.testBasics(TokenMacroTest.java:41) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:603) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang.Thread.run(Thread.java:829)Caused by: org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Invalid identifier in macro at org.jenkinsci.plugins.tokenmacro.Parser.parseIdentifier(Parser.java:203) at org.jenkinsci.plugins.tokenmacro.Parser.parseNonDelimitedToken(Parser.java:168) at org.jenkinsci.plugins.tokenmacro.Parser.parseToken(Parser.java:112) at org.jenkinsci.plugins.tokenmacro.Parser.parse(Parser.java:87) 

          Boon Elschenbroich added a comment - This is the test I executed: assertEquals( "^(?:.*/)?master$|^(?:.*/)?(feature|release)/.* " ,TokenMacro.expandAll(b, TaskListener.NULL, "^(?:.*/)?master$|^(?:.*/)?(feature|release)/.* " )); and it gave me this error: org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokensorg.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Error processing tokens at org.jenkinsci.plugins.tokenmacro.Parser.parse(Parser.java:97) at org.jenkinsci.plugins.tokenmacro.Parser.process(Parser.java:75) at org.jenkinsci.plugins.tokenmacro.Parser.process(Parser.java:68) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expand(TokenMacro.java:199) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:237) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:211) at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:203) at org.jenkinsci.plugins.tokenmacro.TokenMacroTest.testBasics(TokenMacroTest.java:41) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:603) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang. Thread .run( Thread .java:829)Caused by: org.jenkinsci.plugins.tokenmacro.MacroEvaluationException: Invalid identifier in macro at org.jenkinsci.plugins.tokenmacro.Parser.parseIdentifier(Parser.java:203) at org.jenkinsci.plugins.tokenmacro.Parser.parseNonDelimitedToken(Parser.java:168) at org.jenkinsci.plugins.tokenmacro.Parser.parseToken(Parser.java:112) at org.jenkinsci.plugins.tokenmacro.Parser.parse(Parser.java:87)

          Alex Earl added a comment -

          Yeah, I was thinking it was something to do with a token, but it looks like it is doing something different than it used to with things that aren't tokens. I will get to the bottom of it today hopefully.

          Alex Earl added a comment - Yeah, I was thinking it was something to do with a token, but it looks like it is doing something different than it used to with things that aren't tokens. I will get to the bottom of it today hopefully.

          Thanks a lot for fixing this so quickly!

          Boon Elschenbroich added a comment - Thanks a lot for fixing this so quickly!

            slide_o_mix Alex Earl
            boon_el Boon Elschenbroich
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: