Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15895

Job stuck collecting compiler warnings with the eclipse compiler

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Major Major
    • warnings-plugin
    • None
    • Jenkins 1.491 - Warnings plugin 4.18 - Static Analysis Utilities 1.48 - Violations 0.7.10 - Dashboard view 2.2

      I have a maven project style with the following configuration for the maven compiler plugin

                  <plugin>
                      <artifactId>maven-compiler-plugin</artifactId>
                      <version>2.3.2</version>
                      <configuration>
                          <source>1.6</source>
                          <target>1.6</target>
                          <compilerId>jdt</compilerId>
                          <showWarnings>true</showWarnings>
                      </configuration>               
                  </plugin>
      

      The jenkins config scan for compiler warnings using two parsers: Eclipse Java compiler and Java compiler. None of the advanced options are enabled (see screnshot). I have a not so big project with around 20 compiler warnings.

      I am running Jenkins 1.459 with the warnings plugin 3.26 and this is working fine (the graph is generated, new warnings are found, etc).

      If I switch to the new configuration (and that's been a long time I want to upgrade), the job is stuck when parsing the warnings. This is exactly similar to JENKINS-10097. See the configuration of 1.459 with the latest warnings plugins.

        1. thread-dump-2.txt
          29 kB
        2. thread-dump.txt
          34 kB
        3. compiler-warnings-2.png
          compiler-warnings-2.png
          79 kB
        4. compiler-warnings.png
          compiler-warnings.png
          31 kB
        5. config-491-warnings-4.18.png
          config-491-warnings-4.18.png
          60 kB
        6. config-491-warnings-4.18.png
          config-491-warnings-4.18.png
          57 kB
        7. config-459-warnings-3.26.png
          config-459-warnings-3.26.png
          57 kB

          [JENKINS-15895] Job stuck collecting compiler warnings with the eclipse compiler

          Ulli Hafner added a comment -

          Can you please attach a thread dump?

          Do the warnings contain relative paths?

          Ulli Hafner added a comment - Can you please attach a thread dump? Do the warnings contain relative paths?

          Hi,

          Sorry I was in the middle of updating this issue and the network went down. The only threadump I can attach is from my-jenkins:8080/threadDump and I haven't found any useful information there. Do you have a way to generate a reliable thread dump of the process?

          The warnings are all in java source files. I am attaching the config for the instance that is stuck now.

          Stéphane Nicoll added a comment - Hi, Sorry I was in the middle of updating this issue and the network went down. The only threadump I can attach is from my-jenkins:8080/threadDump and I haven't found any useful information there. Do you have a way to generate a reliable thread dump of the process? The warnings are all in java source files. I am attaching the config for the instance that is stuck now.

          sorry, wrong screenshot. Here's the right one.

          Stéphane Nicoll added a comment - sorry, wrong screenshot. Here's the right one.

          Ulli Hafner added a comment -

          For thread dumps see: https://wiki.jenkins-ci.org/display/JENKINS/Build+is+hanging

          Do the warnings contain relative paths? Can you please show a single warning?

          Ulli Hafner added a comment - For thread dumps see: https://wiki.jenkins-ci.org/display/JENKINS/Build+is+hanging Do the warnings contain relative paths? Can you please show a single warning?

          The compiler warnings. They all relates to Java source files. I don't know what you mean by relative path.

          Stéphane Nicoll added a comment - The compiler warnings. They all relates to Java source files. I don't know what you mean by relative path.

          Ulli Hafner added a comment - - edited

          I need something like the following snippet from YOUR console log:

          compile-warning:
              [mkdir] Created dir: C:\Desenvolvimento\Java\jfg\build\build\warning
              [javac] Compiling 34 source files to C:\Desenvolvimento\Java\jfg\build\build\warning
              [javac] ----------
              [javac] 1. WARNING in C:\Desenvolvimento\Java\jfg\src\jfg\AttributeException.java (at line 3)
              [javac] 	public class AttributeException extends RuntimeException
              [javac] 	             ^^^^^^^^^^^^^^^^^^
              [javac] The serializable class AttributeException does not declare a static final serialVersionUID field of type long
              [javac] ----------
              [javac] ----------
              [javac] 2. WARNING in C:\Desenvolvimento\Java\jfg\src\jfg\gui\swt\SWTTextBuilder.java (at line 85)
              [javac] 	protected void addValidation(Text text, Object type)
              [javac] 	                                  ^^^^
              [javac] The parameter text is never read
          

          Then I need to know, if the path in a warning above is relative (contains no C: or / in the beginning) or absolute. If you would paste one of your actual warnings I will see it anyway...

          Ulli Hafner added a comment - - edited I need something like the following snippet from YOUR console log: compile-warning: [mkdir] Created dir: C:\Desenvolvimento\Java\jfg\build\build\warning [javac] Compiling 34 source files to C:\Desenvolvimento\Java\jfg\build\build\warning [javac] ---------- [javac] 1. WARNING in C:\Desenvolvimento\Java\jfg\src\jfg\AttributeException.java (at line 3) [javac] public class AttributeException extends RuntimeException [javac] ^^^^^^^^^^^^^^^^^^ [javac] The serializable class AttributeException does not declare a static final serialVersionUID field of type long [javac] ---------- [javac] ---------- [javac] 2. WARNING in C:\Desenvolvimento\Java\jfg\src\jfg\gui\swt\SWTTextBuilder.java (at line 85) [javac] protected void addValidation(Text text, Object type) [javac] ^^^^ [javac] The parameter text is never read Then I need to know, if the path in a warning above is relative (contains no C: or / in the beginning) or absolute. If you would paste one of your actual warnings I will see it anyway...

          Ulli Hafner added a comment -

          And the thread dump is still missing

          Ulli Hafner added a comment - And the thread dump is still missing

          [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ com.bsb.fluid.core.ui-incubator ---
          [INFO]  Compiling (JDT) 26 source files to C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\test-classes
          [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: Unchecked invocation cast(PathFactory) of the generic method cast(PathFactory<A>) of type BasePath<DummyPersonCrudTaskController>
          [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: The expression of type MDefaultCrudTaskController needs unchecked conversion to conform to MDefaultCrudTaskController<Person,InMemoryCrudTaskEntityHandler<Person>>
          [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: Unchecked invocation type(Class<Person>, Class<InMemoryCrudTaskEntityHandler>) of the generic method type(Class<MT>, Class<MH>) of type MDefaultCrudTaskController
          [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: The expression of type PathFactory needs unchecked conversion to conform to PathFactory<MDefaultCrudTaskController<Person,InMemoryCrudTaskEntityHandler<Person>>>
          mojoSucceeded org.apache.maven.plugins:maven-compiler-plugin:2.3.2(default-testCompile)
          mojoStarted org.apache.maven.plugins:maven-surefire-plugin:2.12(default-test)
          [INFO] 
          

          Stéphane Nicoll added a comment - [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ com.bsb.fluid.core.ui-incubator --- [INFO] Compiling (JDT) 26 source files to C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\test-classes [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: Unchecked invocation cast(PathFactory) of the generic method cast(PathFactory<A>) of type BasePath<DummyPersonCrudTaskController> [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: The expression of type MDefaultCrudTaskController needs unchecked conversion to conform to MDefaultCrudTaskController<Person,InMemoryCrudTaskEntityHandler<Person>> [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: Unchecked invocation type(Class<Person>, Class<InMemoryCrudTaskEntityHandler>) of the generic method type(Class<MT>, Class<MH>) of type MDefaultCrudTaskController [WARNING] C:\stuff\ci\jenkins\jobs\incubator-fluid\workspace\core\com.bsb.fluid.core.ui-incubator\target\generated-sources\test-annotations\com\bsb\fluid\test\core\ui\task\MDummyPersonCrudTaskController.java:[34,0] Type safety: The expression of type PathFactory needs unchecked conversion to conform to PathFactory<MDefaultCrudTaskController<Person,InMemoryCrudTaskEntityHandler<Person>>> mojoSucceeded org.apache.maven.plugins:maven-compiler-plugin:2.3.2(default-testCompile) mojoStarted org.apache.maven.plugins:maven-surefire-plugin:2.12(default-test) [INFO]

          thread dump

          Stéphane Nicoll added a comment - thread dump

          Sorry it took so long to get those info. Let me know if you need anything else.

          Stéphane Nicoll added a comment - Sorry it took so long to get those info. Let me know if you need anything else.

          Ulli Hafner added a comment -

          Ok, thanks, seems that the problem is unrelated to JENKINS-10097. The process "hangs" while scanning for warnings using a regular expression matcher. How large is your console log? The eclipse parsers is a slow multi-line parser that reads the whole console log into a buffer. Would it be possible to pipe the warnings to an additional file?

          BTW: I think you do not need to activate the Eclipse parser, since the warnings seems to be in the default javac format (due to the maven plugin).

          Ulli Hafner added a comment - Ok, thanks, seems that the problem is unrelated to JENKINS-10097 . The process "hangs" while scanning for warnings using a regular expression matcher. How large is your console log? The eclipse parsers is a slow multi-line parser that reads the whole console log into a buffer. Would it be possible to pipe the warnings to an additional file? BTW: I think you do not need to activate the Eclipse parser, since the warnings seems to be in the default javac format (due to the maven plugin).

          350kb.

          After 1h15, it is still on at hudson.plugins.warnings.parser.RegexpParser.findAnnotations(RegexpParser.java:86)

          Seems a lot for ~4000 lines

          Stéphane Nicoll added a comment - 350kb. After 1h15, it is still on at hudson.plugins.warnings.parser.RegexpParser.findAnnotations(RegexpParser.java:86) Seems a lot for ~4000 lines

          Ulli Hafner added a comment -

          Can you please check if that happens with the plain Java parser, too? Or why are you using the Eclipse Parser? Are there warnings not covered?

          Ulli Hafner added a comment - Can you please check if that happens with the plain Java parser, too? Or why are you using the Eclipse Parser? Are there warnings not covered?

          it does work with only the Javac compiler as the problem is in the eclipse parser it seems. I can only use the Javac compiler and I get a proper list of warnings.

          Thanks!

          (Hint: maybe we should document somewhere that Maven-based jobs using the jdt compiler do not need to use the eclipse parser)

          Stéphane Nicoll added a comment - it does work with only the Javac compiler as the problem is in the eclipse parser it seems. I can only use the Javac compiler and I get a proper list of warnings. Thanks! (Hint: maybe we should document somewhere that Maven-based jobs using the jdt compiler do not need to use the eclipse parser)

          Ulli Hafner added a comment -

          Well, actually the warnings are different if you build with maven and tycho (then you need the Eclipse parser...).

          But I think it would be worth adding a note in the documentation at least.

          Are you using a maven job or freestyle job to build your project? Currently there is no special support for maven jobs in my plug-in, maybe I should add that.

          Ulli Hafner added a comment - Well, actually the warnings are different if you build with maven and tycho (then you need the Eclipse parser...). But I think it would be worth adding a note in the documentation at least. Are you using a maven job or freestyle job to build your project? Currently there is no special support for maven jobs in my plug-in, maybe I should add that.

          Ulli Hafner added a comment -

          I create JENKINS-15933 as a followup for this issue.

          Ulli Hafner added a comment - I create JENKINS-15933 as a followup for this issue.

          Ulli Hafner added a comment -

          Setting the resolution to won't fix since there is almost no way to improve the speed of the multi-line Eclipse parser.

          Ulli Hafner added a comment - Setting the resolution to won't fix since there is almost no way to improve the speed of the multi-line Eclipse parser.

          Stéphane Nicoll added a comment - - edited

          I am using a maven-based job. According to JENINS-15933, eclipse would be detected automatically as far as I understand. That's probably not what I want.

          By the way, I am not so sure that it's a performance problem, really. Still Parsing 4000 lines after 2 hours, something must definitely be wrong. Maybe I should left it running for the night to see how it goes.

          Thanks.

          Stéphane Nicoll added a comment - - edited I am using a maven-based job. According to JENINS-15933, eclipse would be detected automatically as far as I understand. That's probably not what I want. By the way, I am not so sure that it's a performance problem, really. Still Parsing 4000 lines after 2 hours, something must definitely be wrong. Maybe I should left it running for the night to see how it goes. Thanks.

            drulli Ulli Hafner
            snicoll Stéphane Nicoll
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: