Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54982

Using Java8 Streams evaluates whole expression to Boolean

      I'm trying to write a common library building Docker images from Dockerfile. I wrote a reusable global pipeline, where I'm iterating over Dockerfile lines. I tried to use Java 8 Streams, however I ended up with issue potentially related to JENKINS-26307 and JENKINS-42769, which is not related not to Groovy functional features, but Java 8 feature.

       An example code used:

      def getMissingLabels(Map<String, String> labels) {
       ['label1', 'label2', 'label3', 'label4'].stream()
              .filter({ label -> !isLabelDefined(label, labels) })
              .collect(Collectors.toList())
      }
      

      Whole stream execution results in "false" boolean result, instead of a list. Even complex streams are passing through, but evaluates to "false". When using more complex stream like:

       

      Arrays.stream(splitted)
                  .filter { it.startsWith('LABEL') }
                  .map { it.replaceAll('(LABEL)|"', '') }
                  .map { it.trim() }
                  .map { it.split('=', 2) }
                  .collect(Collectors.toMap(
                  { val -> val[0].toLowerCase() },
                  { val -> val[1] }))
      

      it enters only first filter expression and does not evaluate the rest.

      When replaced functional style code with typical imperative style (for loop) it works.

       

          [JENKINS-54982] Using Java8 Streams evaluates whole expression to Boolean

          Devin Nusbaum added a comment -

          I reproduced this with some quick testing. I think this is very similar to JENKINS-26481, for which I am not familiar with the full fix, but I think it involved rewriting DefaultGroovyMethods directly in groovy-cps, which I guess we can't really do here as this involves Java Platform code rather than groovy code, but I'm not completely sure. CC abayer who worked on the fix for JENKINS-26481 and might have a better idea.

          You might be able to replace this with something equivalent that uses Groovy language features instead of the Stream API as a workaround.

          Devin Nusbaum added a comment - I reproduced this with some quick testing. I think this is very similar to JENKINS-26481 , for which I am not familiar with the full fix, but I think it involved rewriting DefaultGroovyMethods directly in groovy-cps, which I guess we can't really do here as this involves Java Platform code rather than groovy code, but I'm not completely sure. CC abayer who worked on the fix for JENKINS-26481 and might have a better idea. You might be able to replace this with something equivalent that uses Groovy language features instead of the Stream API as a workaround.

          Thomas Fox added a comment -

          I have a similar problem when trying to use a Java8 parallelStream() in a Jenkins shared library. The stream processes only a single item in the list. The problem is also with stream() but not with the groovy each(which I cannot use because it does not execute in parallel). The shared library code is

          def streamtest()
          {
            def myList = ["1","2","3","4"]
            
            echo "each:"
            myList.each{ entry -> echo "$entry" }
            
            echo "parallelStream:"
            myList.parallelStream().forEach{ entry -> echo "$entry" }
            
            echo "stream:"
            myList.stream().forEach{ entry -> echo "$entry" }
          }
          

          Sample output is

           

           

          [Pipeline] stage
          [Pipeline] { (streamtest)
          [Pipeline] echo
          each:
          [Pipeline] echo
          1
          [Pipeline] echo
          2
          [Pipeline] echo
          3
          [Pipeline] echo
          4
          [Pipeline] echo
          parallelStream:
          [Pipeline] echo
          3
          [Pipeline] echo
          stream:
          [Pipeline] echo
          1
          [Pipeline] }
          [Pipeline] // stage
          

          As an aside, I have also tried to use GPars for parallelism but I could not get it to work with CPS. I also have tried an ThreadPoolExecutor which I also could not get to work (it did not seem to start any threads but I did not do a thorough investigation).

           

          My hopes are

          • If a java language feature does not work, it would be cool if it would fail fast instead of doing unexpected things
          • Is there any working method to use parallelism with threads (not with nodes) in a shared library?

          Thomas Fox added a comment - I have a similar problem when trying to use a Java8 parallelStream() in a Jenkins shared library. The stream processes only a single item in the list. The problem is also with stream() but not with the groovy each(which I cannot use because it does not execute in parallel). The shared library code is def streamtest() { def myList = [ "1" , "2" , "3" , "4" ] echo "each:" myList.each{ entry -> echo "$entry" } echo "parallelStream:" myList.parallelStream().forEach{ entry -> echo "$entry" } echo "stream:" myList.stream().forEach{ entry -> echo "$entry" } } Sample output is     [Pipeline] stage [Pipeline] { (streamtest) [Pipeline] echo each: [Pipeline] echo 1 [Pipeline] echo 2 [Pipeline] echo 3 [Pipeline] echo 4 [Pipeline] echo parallelStream: [Pipeline] echo 3 [Pipeline] echo stream: [Pipeline] echo 1 [Pipeline] } [Pipeline] // stage As an aside, I have also tried to use GPars for parallelism but I could not get it to work with CPS. I also have tried an ThreadPoolExecutor which I also could not get to work (it did not seem to start any threads but I did not do a thorough investigation).   My hopes are If a java language feature does not work, it would be cool if it would fail fast instead of doing unexpected things Is there any working method to use parallelism with threads (not with nodes) in a shared library?

            Unassigned Unassigned
            filipowm Mateusz Filipowicz
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: