• 2.320,2.319.1

      See https://github.com/jenkinsci/jenkins/runs/4107177325
      https://github.com/jenkinsci/jenkins/pull/5881/checks?check_run_id=4107861004

      Logs:
      https://ci.jenkins.io/job/Core/job/jenkins/job/PR-5881/2/execution/node/116/

      java.nio.file.NoSuchFileException: /home/jenkins/workspace/Core_jenkins_PR-5881/ath/athSources/src/main/resources/ath-container/run.sh
      	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
      	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
      	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
      	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
      	at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:484)
      	at java.base/java.nio.file.Files.newOutputStream(Files.java:228)
      	at hudson.util.IOUtils.copy(IOUtils.java:52)
      	at hudson.FilePath.readFromTar(FilePath.java:2862)
      Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to EC2 (aws-us-east-2) - High memory ubuntu 20.04 (i-02b6bda105ac6c6d9)
      		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1797)
      		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
      		at hudson.remoting.Channel.call(Channel.java:1001)
      		at hudson.FilePath.act(FilePath.java:1166)
      		at hudson.FilePath.act(FilePath.java:1155)
      		at hudson.FilePath.untar(FilePath.java:617)
      		at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:161)
      		at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:77)
      		at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64)
      		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
      		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      		at java.base/java.lang.Thread.run(Thread.java:829)
      Caused: java.io.IOException: Failed to extract athSources.tar.gz
      	at hudson.FilePath.readFromTar(FilePath.java:2872)
      	at hudson.FilePath.access$500(FilePath.java:213)
      	at hudson.FilePath$UntarRemote.invoke(FilePath.java:633)
      	at hudson.FilePath$UntarRemote.invoke(FilePath.java:622)
      	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3338)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
      	at hudson.remoting.Request$2.run(Request.java:376)
      	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      	at java.base/java.lang.Thread.run(Thread.java:833)
      

      Started just after the controller was upgraded to the security release

      cc danielbeck wfollonier jglick

          [JENKINS-67063] Unstash with symlinks fails

          Tim Jacomb added a comment -

          Reproduced locally with script extracted from pipeline library,

          should run on anything with maven and git installed, inline pipeline script.
          Recommended plugins + pipeline utility steps

          #!/usr/bin/env groovy
          
          /**
           * Wrapper for running the ATH see README and runATH.txt for full documentation
           */
          
          void stashJenkinsWar(String jenkins, String stashName = "jenkinsWar") {
              def isVersionNumber = (jenkins =~ /^\d+([.]\d+)*(-rc[0-9]+[.][0-9a-f]{12})?$/).matches()
              def isLocalJenkins = jenkins.startsWith("file://")
              def mirror = "http://mirrors.jenkins.io/"
          
              def jenkinsURL
          
              if (jenkins == "latest") {
                  jenkinsURL = mirror + "war/latest/jenkins.war"
              } else if (jenkins == "latest-rc") {
                  jenkinsURL = mirror + "/war-rc/latest/jenkins.war"
              } else if (jenkins == "lts") {
                  jenkinsURL = mirror + "war-stable/latest/jenkins.war"
              } else if (jenkins == "lts-rc") {
                  jenkinsURL = mirror + "war-stable-rc/latest/jenkins.war"
              }
          
              if (isLocalJenkins) {
                  if (!fileExists(jenkins - "file://")) {
                      error "Specified Jenkins file does not exists"
                  }
              }
              if (!isVersionNumber && !isLocalJenkins) {
                  if (jenkinsURL == null) {
                      error "Not sure how to interpret $jenkins as a version, alias, or URL"
                  }
                  echo 'Checking whether Jenkins WAR is available…'
                  sh "curl -ILf ${jenkinsURL}"
              }
          
              if (isVersionNumber) {
                  List<String> downloadCommand = [
                          "dependency:copy",
                          "-Dartifact=org.jenkins-ci.main:jenkins-war:${jenkins}:war",
                          "-DoutputDirectory=.",
                          "-Dmdep.stripVersion=true"
                  ]
                  dir("deps") {
                      runMaven(downloadCommand)
                      sh "cp jenkins-war.war jenkins.war"
                      stash includes: 'jenkins.war', name: stashName
                  }
              } else if (isLocalJenkins) {
                  dir(pwd(tmp: true)) {
                      sh "cp ${jenkins - 'file://'} jenkins.war"
                      stash includes: "*.war", name: "jenkinsWar"
                  }
              } else {
                  sh("curl -o jenkins.war -L ${jenkinsURL}")
                  stash includes: '*.war', name: 'jenkinsWar'
              }
          }
          
          def runATH(Map params = [:]) {
              def athUrl = params.get('athUrl', 'https://github.com/jenkinsci/acceptance-test-harness.git')
              def athRevision = params.get('athRevision', 'master')
              def metadataFile = params.get('metadataFile', 'essentials.yml')
              def jenkins = params.get('jenkins', 'latest')
              def jdks = params.get('jdks', [8])
              def athContainerImageTag = params.get("athImage", "jenkins/ath");
              def configFile = params.get("configFile", null)
              def defaultJavaOptions = params.get('javaOptions', ['--no-transfer-progress'])
          
              def mirror = "http://mirrors.jenkins.io/"
              def defaultCategory = "org.jenkinsci.test.acceptance.junit.SmokeTest"
              def metadata
              def athContainerImage
              def isLocalATH
              def isVersionNumber
          
              def athSourcesFolder = "athSources"
          
              def supportedBrowsers = ["firefox"]
              def supportedJdks = [8, 11]
          
              def skipExecution = false
          
              def localPluginsStashName = env.RUN_ATH_LOCAL_PLUGINS_STASH_NAME ?: "localPlugins"
          
          
                  if (!fileExists(metadataFile)) {
                      echo "Skipping ATH execution because the metadata file does not exist. Current value is ${metadataFile}."
                      skipExecution = true
                      return
                  }
          
              stage("Getting ATH sources and Jenkins war") {
                  // Start validation
                  metadata = readYaml(file: metadataFile)?.ath
                  if (metadata == null) {
                      echo "Skipping ATH execution because the metadata file does not contain an ath section"
                      skipExecution = true
                      return
                  }
                  if (metadata == 'default') {
                      echo "Using default configuration for ATH"
                      metadata = [:]
                  } else if (metadata.browsers == null) {
                      echo "The provided metadata file does not include the browsers property, using firefox as default"
                  }
                  // Allow to override athUrl and athRevision from metadata file
                  athUrl = metadata.athUrl ?: athUrl
                  isLocalATH = athUrl.startsWith("file://")
                  athRevision = metadata.athRevision ?: athRevision
                  athContainerImageTag = metadata.athImage ?: athContainerImageTag
          
                  // Allow override of jenkins version from metadata file
                  jenkins = metadata.jenkins ?: jenkins
                  isVersionNumber = (jenkins =~ /^\d+([.]\d+)*$/).matches()
          
                  // Allow override of JDK version from metadata file
                  jdks = metadata.jdks ?: jdks
          
                  if (!isLocalATH) {
                      echo 'Checking connectivity to ATH sources…'
                      sh "git ls-remote --exit-code -h ${athUrl}"
                  }
                  stashJenkinsWar(jenkins)
                  // Validation ended
          
                  // ATH
                  if (isLocalATH) { // Deal with already existing ATH sources
                      athSourcesFolder = athUrl - "file://"
                  } else {
                      dir(athSourcesFolder) {
                          checkout changelog: true, poll: false, scm: [$class           : 'GitSCM', branches: [[name: athRevision]],
                                                                       userRemoteConfigs: [[url: athUrl]]]
                      }
                  }
                  dir(athSourcesFolder) {
                      // We may need to run things in parallel later, so avoid several checkouts
                      stash name: "athSources"
                  }
          
              }
          
              
                  if (skipExecution) {
                      return
                  }
                  stage("Running ATH") {
                      dir("athSources") {
                          unstash name: "athSources"
                          if (athContainerImageTag == "local") {
                              echo "'local' ATH container image specified, building it"
                              def uid = sh(script: "id -u", returnStdout: true)
                              def gid = sh(script: "id -g", returnStdout: true)
                              athContainerImage = docker.build('jenkins/ath', "--build-arg=uid='$uid' --build-arg=gid='$gid' -f src/main/resources/ath-container/Dockerfile .")
                          } else {
                              echo "No building ATH docker container image. Using ${athContainerImageTag} as specified"
                              athContainerImage = docker.image(athContainerImageTag)
                              athContainerImage.pull() //Use the latest available version
                          }
                      }
          
                      def testsToRun = metadata.tests?.join(",")
                      def categoriesToRun = metadata.categories?.join(",")
                      def browsers = metadata.browsers ?: ["firefox"]
                      def failFast = metadata.failFast ?: false
                      def rerunCount = metadata.rerunFailingTestsCount ?: 0
                      // Elvis fails in case useLocalSnapshots == false in metadata File
                      def localSnapshots = metadata.useLocalSnapshots != null ? metadata.useLocalSnapshots : true
          
                      if (testsToRun == null && categoriesToRun == null) {
                          categoriesToRun = defaultCategory
                      }
          
                      // Shorthand for running all tests
                      if (testsToRun == "all") {
                          testsToRun = categoriesToRun = null
                      }
          
                      def testingbranches = ["failFast": failFast]
                      for (jdk in jdks) {
                          if (supportedJdks.contains(jdk)) {
                              def currentJdk = jdk
                              def javaOptions = defaultJavaOptions.clone()
                              //Add shm-size to avoid selenium.WebDriverException exceptions like 'Failed to decode response from marionette' and webdriver closed
                              def containerArgs = "-v /var/run/docker.sock:/var/run/docker.sock -u ath-user --shm-size 2g"
          
                              if(configFile) {
                                  containerArgs += " -e CONFIG=../${configFile}" // ATH runs are executed in a subfolder, hence path needs to take that into account
                              }
          
                              // Add options for jdks
                              if ( currentJdk  > 8) {
                                  // Add environment variable
                                  containerArgs += " -e java_version=${currentJdk}"
                              }
          
                              for (browser in browsers) {
                                  if (supportedBrowsers.contains(browser)) {
                                      def currentBrowser = browser
          
                                      def commandBase = "./run.sh ${currentBrowser} ./jenkins.war -B -ntp -Dmaven.test.failure.ignore=true -DforkCount=1 -Dsurefire.rerunFailingTestsCount=${rerunCount}"
          
                                      if (testsToRun) {
                                          testingbranches["ATH individual tests-${currentBrowser}-jdk${currentJdk}"] = {
                                              dir("test${currentBrowser}jdk${currentJdk}") {
                                                  def discriminator = "-Dtest=${testsToRun}"
                                                  test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions)
                                              }
                                          }
                                      }
                                      if (categoriesToRun) {
                                          testingbranches["ATH categories-${currentBrowser}-jdk${currentJdk}"] = {
                                              dir("categories${currentBrowser}jdk${currentJdk}") {
                                                  def discriminator = "-Dgroups=${categoriesToRun}"
                                                  test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions)
                                              }
                                          }
                                      }
                                      if (testsToRun == null && categoriesToRun == null) {
                                          testingbranches["ATH ${currentBrowser}-jdk${currentJdk}"] = {
                                              dir("ath${currentBrowser}${currentJdk}") {
                                                  def discriminator = ""
                                                  test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions)
                                              }
                                          }
                                      }
          
                                  } else {
                                      echo "${browser} is not yet supported"
                                  }
                              }
                          } else {
                              echo "${jdk} is not yet supported"
                          }
                      }
          
                      parallel testingbranches
                  }
          }
          
          private void test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions) {
              unstashResources(localSnapshots, localPluginsStashName)
              athContainerImage.inside(containerArgs) {
                  realtimeJUnit(testResults: 'target/surefire-reports/TEST-*.xml', testDataPublishers: [[$class: 'AttachmentPublisher']]) {
                      /*
                      If you want to compile with java > 8 and have all needed, do it
                      If you want to compile with java > 8 and you lack the set-java script, FAILURE
                      If other case (java8 and lack set-java), write a message in the log
                       */
                      def command = '''
                      if [ -x ./set-java.sh ] && [ -n "$java_version" ] && [ "$java_version" -gt "8" ]; then
                          ./set-java.sh $java_version;
                      elif [ -n "$java_version" ] && [ "$java_version" -gt "8" ] && [ ! -x ./set-java.sh ]; then
                          echo "ERROR: ./set-java.sh not found because you are using old ATH sources and \\$java_version = $java_version specified. We cannot run on this JDK";
                          exit 1;
                      elif [ ! -x ./set-java.sh ]; then
                          echo "INFO: ./set-java.sh not found because you are using old ATH sources, please consider to update ATH sources and docker image";
                      fi
          
                      eval "$(./vnc.sh)" \
                      && export DISPLAY=$BROWSER_DISPLAY \
                      && export SHARED_DOCKER_SERVICE=true \
                      && export EXERCISEDPLUGINREPORTER=textfile \
                      && \
                      '''
          
                      command += prepareCommand(commandBase, discriminator, localSnapshots, localPluginsStashName)
                      if (!javaOptions.isEmpty()) {
                          command = """export JAVA_OPTS="${javaOptions.join(' ')}" && ${command}"""
                      }
          
                      sh command
                  }
              }
          }
          
          private String prepareCommand(commandBase, discriminator, localSnapshots, localPluginsStashName ) {
              def command = commandBase + " ${discriminator}"
              if (localSnapshots && localPluginsStashName) {
                  command = "LOCAL_JARS=${getLocalPluginsList()} " + command
              }
              command
          }
          
          private void unstashResources(localSnapshots, localPluginsStashName) {
              unstash name: "athSources"
              unstash name: "jenkinsWar"
              dir("localPlugins") {
                  if (localSnapshots && localPluginsStashName) {
                      unstash name: localPluginsStashName
                  }
              }
          }
          
          private String getLocalPluginsList() {
              return sh(script : "ls -p -d -1 localPlugins/*.* | tr '\n' ':'| sed 's/.\$//'", returnStdout: true).trim()
          }
          
          node("docker-highmem") {
              // Just to be safe
              deleteDir()
              def fileUri
              def metadataPath
              dir("sources") {
                  git 'https://github.com/jenkinsci/jenkins.git'
                  sh 'mvn --batch-mode --show-version -ntp -Pquick-build -am -pl war package'
                  dir("war/target") {
                      fileUri = "file://" + pwd() + "/jenkins.war"
                  }
                  metadataPath = pwd() + "/essentials.yml"
              }
              dir("ath") {
                  runATH jenkins: fileUri, metadataFile: metadataPath
              }
          }
          

          Tim Jacomb added a comment - Reproduced locally with script extracted from pipeline library, should run on anything with maven and git installed, inline pipeline script. Recommended plugins + pipeline utility steps #!/usr/bin/env groovy /** * Wrapper for running the ATH see README and runATH.txt for full documentation */ void stashJenkinsWar( String jenkins, String stashName = "jenkinsWar" ) { def isVersionNumber = (jenkins =~ /^\d+([.]\d+)*(-rc[0-9]+[.][0-9a-f]{12})?$/).matches() def isLocalJenkins = jenkins.startsWith( "file: //" ) def mirror = "http: //mirrors.jenkins.io/" def jenkinsURL if (jenkins == "latest" ) { jenkinsURL = mirror + "war/latest/jenkins.war" } else if (jenkins == "latest-rc" ) { jenkinsURL = mirror + "/war-rc/latest/jenkins.war" } else if (jenkins == "lts" ) { jenkinsURL = mirror + "war-stable/latest/jenkins.war" } else if (jenkins == "lts-rc" ) { jenkinsURL = mirror + "war-stable-rc/latest/jenkins.war" } if (isLocalJenkins) { if (!fileExists(jenkins - "file: //" )) { error "Specified Jenkins file does not exists" } } if (!isVersionNumber && !isLocalJenkins) { if (jenkinsURL == null ) { error "Not sure how to interpret $jenkins as a version, alias, or URL" } echo 'Checking whether Jenkins WAR is available…' sh "curl -ILf ${jenkinsURL}" } if (isVersionNumber) { List< String > downloadCommand = [ "dependency:copy" , "-Dartifact=org.jenkins-ci.main:jenkins-war:${jenkins}:war" , "-DoutputDirectory=." , "-Dmdep.stripVersion= true " ] dir( "deps" ) { runMaven(downloadCommand) sh "cp jenkins-war.war jenkins.war" stash includes: 'jenkins.war' , name: stashName } } else if (isLocalJenkins) { dir(pwd(tmp: true )) { sh "cp ${jenkins - 'file: //' } jenkins.war" stash includes: "*.war" , name: "jenkinsWar" } } else { sh( "curl -o jenkins.war -L ${jenkinsURL}" ) stash includes: '*.war' , name: 'jenkinsWar' } } def runATH(Map params = [:]) { def athUrl = params.get( 'athUrl' , 'https: //github.com/jenkinsci/acceptance-test-harness.git' ) def athRevision = params.get( 'athRevision' , 'master' ) def metadataFile = params.get( 'metadataFile' , 'essentials.yml' ) def jenkins = params.get( 'jenkins' , 'latest' ) def jdks = params.get( 'jdks' , [8]) def athContainerImageTag = params.get( "athImage" , "jenkins/ath" ); def configFile = params.get( "configFile" , null ) def defaultJavaOptions = params.get( 'javaOptions' , [ '--no-transfer-progress' ]) def mirror = "http: //mirrors.jenkins.io/" def defaultCategory = "org.jenkinsci.test.acceptance.junit.SmokeTest" def metadata def athContainerImage def isLocalATH def isVersionNumber def athSourcesFolder = "athSources" def supportedBrowsers = [ "firefox" ] def supportedJdks = [8, 11] def skipExecution = false def localPluginsStashName = env.RUN_ATH_LOCAL_PLUGINS_STASH_NAME ?: "localPlugins" if (!fileExists(metadataFile)) { echo "Skipping ATH execution because the metadata file does not exist. Current value is ${metadataFile}." skipExecution = true return } stage( "Getting ATH sources and Jenkins war" ) { // Start validation metadata = readYaml(file: metadataFile)?.ath if (metadata == null ) { echo "Skipping ATH execution because the metadata file does not contain an ath section" skipExecution = true return } if (metadata == ' default ' ) { echo "Using default configuration for ATH" metadata = [:] } else if (metadata.browsers == null ) { echo "The provided metadata file does not include the browsers property, using firefox as default " } // Allow to override athUrl and athRevision from metadata file athUrl = metadata.athUrl ?: athUrl isLocalATH = athUrl.startsWith( "file: //" ) athRevision = metadata.athRevision ?: athRevision athContainerImageTag = metadata.athImage ?: athContainerImageTag // Allow override of jenkins version from metadata file jenkins = metadata.jenkins ?: jenkins isVersionNumber = (jenkins =~ /^\d+([.]\d+)*$/).matches() // Allow override of JDK version from metadata file jdks = metadata.jdks ?: jdks if (!isLocalATH) { echo 'Checking connectivity to ATH sources…' sh "git ls-remote --exit-code -h ${athUrl}" } stashJenkinsWar(jenkins) // Validation ended // ATH if (isLocalATH) { // Deal with already existing ATH sources athSourcesFolder = athUrl - "file: //" } else { dir(athSourcesFolder) { checkout changelog: true , poll: false , scm: [$class : 'GitSCM' , branches: [[name: athRevision]], userRemoteConfigs: [[url: athUrl]]] } } dir(athSourcesFolder) { // We may need to run things in parallel later, so avoid several checkouts stash name: "athSources" } } if (skipExecution) { return } stage( "Running ATH" ) { dir( "athSources" ) { unstash name: "athSources" if (athContainerImageTag == "local" ) { echo " 'local' ATH container image specified, building it" def uid = sh(script: "id -u" , returnStdout: true ) def gid = sh(script: "id -g" , returnStdout: true ) athContainerImage = docker.build( 'jenkins/ath' , "--build-arg=uid= '$uid' --build-arg=gid= '$gid' -f src/main/resources/ath-container/Dockerfile ." ) } else { echo "No building ATH docker container image. Using ${athContainerImageTag} as specified" athContainerImage = docker.image(athContainerImageTag) athContainerImage.pull() //Use the latest available version } } def testsToRun = metadata.tests?.join( "," ) def categoriesToRun = metadata.categories?.join( "," ) def browsers = metadata.browsers ?: [ "firefox" ] def failFast = metadata.failFast ?: false def rerunCount = metadata.rerunFailingTestsCount ?: 0 // Elvis fails in case useLocalSnapshots == false in metadata File def localSnapshots = metadata.useLocalSnapshots != null ? metadata.useLocalSnapshots : true if (testsToRun == null && categoriesToRun == null ) { categoriesToRun = defaultCategory } // Shorthand for running all tests if (testsToRun == "all" ) { testsToRun = categoriesToRun = null } def testingbranches = [ "failFast" : failFast] for (jdk in jdks) { if (supportedJdks.contains(jdk)) { def currentJdk = jdk def javaOptions = defaultJavaOptions.clone() //Add shm-size to avoid selenium.WebDriverException exceptions like 'Failed to decode response from marionette' and webdriver closed def containerArgs = "-v / var /run/docker.sock:/ var /run/docker.sock -u ath-user --shm-size 2g" if (configFile) { containerArgs += " -e CONFIG=../${configFile}" // ATH runs are executed in a subfolder, hence path needs to take that into account } // Add options for jdks if ( currentJdk > 8) { // Add environment variable containerArgs += " -e java_version=${currentJdk}" } for (browser in browsers) { if (supportedBrowsers.contains(browser)) { def currentBrowser = browser def commandBase = "./run.sh ${currentBrowser} ./jenkins.war -B -ntp -Dmaven.test.failure.ignore= true -DforkCount=1 -Dsurefire.rerunFailingTestsCount=${rerunCount}" if (testsToRun) { testingbranches[ "ATH individual tests-${currentBrowser}-jdk${currentJdk}" ] = { dir( "test${currentBrowser}jdk${currentJdk}" ) { def discriminator = "-Dtest=${testsToRun}" test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions) } } } if (categoriesToRun) { testingbranches[ "ATH categories-${currentBrowser}-jdk${currentJdk}" ] = { dir( "categories${currentBrowser}jdk${currentJdk}" ) { def discriminator = "-Dgroups=${categoriesToRun}" test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions) } } } if (testsToRun == null && categoriesToRun == null ) { testingbranches[ "ATH ${currentBrowser}-jdk${currentJdk}" ] = { dir( "ath${currentBrowser}${currentJdk}" ) { def discriminator = "" test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions) } } } } else { echo "${browser} is not yet supported" } } } else { echo "${jdk} is not yet supported" } } parallel testingbranches } } private void test(discriminator, commandBase, localSnapshots, localPluginsStashName, containerArgs, athContainerImage, javaOptions) { unstashResources(localSnapshots, localPluginsStashName) athContainerImage.inside(containerArgs) { realtimeJUnit(testResults: 'target/surefire-reports/TEST-*.xml' , testDataPublishers: [[$class: 'AttachmentPublisher' ]]) { /* If you want to compile with java > 8 and have all needed, do it If you want to compile with java > 8 and you lack the set-java script, FAILURE If other case (java8 and lack set-java), write a message in the log */ def command = ''' if [ -x ./set-java.sh ] && [ -n "$java_version" ] && [ "$java_version" -gt "8" ]; then ./set-java.sh $java_version; elif [ -n "$java_version" ] && [ "$java_version" -gt "8" ] && [ ! -x ./set-java.sh ]; then echo "ERROR: ./set-java.sh not found because you are using old ATH sources and \\$java_version = $java_version specified. We cannot run on this JDK" ; exit 1; elif [ ! -x ./set-java.sh ]; then echo "INFO: ./set-java.sh not found because you are using old ATH sources, please consider to update ATH sources and docker image" ; fi eval "$(./vnc.sh)" \ && export DISPLAY=$BROWSER_DISPLAY \ && export SHARED_DOCKER_SERVICE= true \ && export EXERCISEDPLUGINREPORTER=textfile \ && \ ''' command += prepareCommand(commandBase, discriminator, localSnapshots, localPluginsStashName) if (!javaOptions.isEmpty()) { command = """export JAVA_OPTS=" ${javaOptions.join( ' ' )} " && ${command}" "" } sh command } } } private String prepareCommand(commandBase, discriminator, localSnapshots, localPluginsStashName ) { def command = commandBase + " ${discriminator}" if (localSnapshots && localPluginsStashName) { command = "LOCAL_JARS=${getLocalPluginsList()} " + command } command } private void unstashResources(localSnapshots, localPluginsStashName) { unstash name: "athSources" unstash name: "jenkinsWar" dir( "localPlugins" ) { if (localSnapshots && localPluginsStashName) { unstash name: localPluginsStashName } } } private String getLocalPluginsList() { return sh(script : "ls -p -d -1 localPlugins/*.* | tr '\n' ':' | sed 's/.\$ //' " , returnStdout: true ).trim() } node( "docker-highmem" ) { // Just to be safe deleteDir() def fileUri def metadataPath dir( "sources" ) { git 'https: //github.com/jenkinsci/jenkins.git' sh 'mvn --batch-mode --show-version -ntp -Pquick-build -am -pl war package ' dir( "war/target" ) { fileUri = "file: //" + pwd() + "/jenkins.war" } metadataPath = pwd() + "/essentials.yml" } dir( "ath" ) { runATH jenkins: fileUri, metadataFile: metadataPath } }

          Jesse Glick added a comment -
          total 16
          drwxrwxr-x 2 jglick jglick 4096 Nov  4 17:43 .
          drwxrwxr-x 4 jglick jglick 4096 Nov  4 17:43 ..
          -rw-rw-r-- 1 jglick jglick 2730 Nov  4 17:43 Dockerfile
          lrwxrwxrwx 1 jglick jglick   39 Nov  4 17:43 run.sh -> src/main/resources/ath-container/run.sh
          lrwxrwxrwx 1 jglick jglick   44 Nov  4 17:43 set-java.sh -> src/main/resources/ath-container/set-java.sh
          -rwxrwxr-x 1 jglick jglick  752 Nov  4 17:43 vnc.sh
          

          Something amiss with symlinks, perhaps due to the new use of canonicalization.

          Jesse Glick added a comment - total 16 drwxrwxr-x 2 jglick jglick 4096 Nov 4 17:43 . drwxrwxr-x 4 jglick jglick 4096 Nov 4 17:43 .. -rw-rw-r-- 1 jglick jglick 2730 Nov 4 17:43 Dockerfile lrwxrwxrwx 1 jglick jglick 39 Nov 4 17:43 run.sh -> src/main/resources/ath-container/run.sh lrwxrwxrwx 1 jglick jglick 44 Nov 4 17:43 set-java.sh -> src/main/resources/ath-container/set-java.sh -rwxrwxr-x 1 jglick jglick 752 Nov 4 17:43 vnc.sh Something amiss with symlinks, perhaps due to the new use of canonicalization.

          Daniel Beck added a comment - - edited

          Run this twice and it will fail:

          node {
              dir('athSources') {
                  checkout changelog: true, poll: false, scm: [$class: 'GitSCM', branches: [[name: 'master']],
                              userRemoteConfigs: [[url: 'https://github.com/jenkinsci/acceptance-test-harness.git']]]
                  stash name: 'athSources'
              }
              dir('foo2') {
                  unstash name: 'athSources'
              }
          }
          

          Daniel Beck added a comment - - edited Run this twice and it will fail: node { dir('athSources') { checkout changelog: true, poll: false, scm: [$class: 'GitSCM', branches: [[name: 'master']], userRemoteConfigs: [[url: 'https://github.com/jenkinsci/acceptance-test-harness.git']]] stash name: 'athSources' } dir('foo2') { unstash name: 'athSources' } }

          Daniel Beck added a comment -

          Reproduced in a test with the full athSources.tar.gz, but all attempts at recreating a tar file from scratch that has the same problem have failed so far.

          Daniel Beck added a comment - Reproduced in a test with the full athSources.tar.gz , but all attempts at recreating a tar file from scratch that has the same problem have failed so far.

          Daniel Beck added a comment -

          It looks like archiving artifacts from agents is unaffected, 2.303.2 and 2.303.3 have the same archived artifacts from a checkout of jenkinsci/acceptance-test-harness.

          Daniel Beck added a comment - It looks like archiving artifacts from agents is unaffected, 2.303.2 and 2.303.3 have the same archived artifacts from a checkout of jenkinsci/acceptance-test-harness .

          Daniel Beck added a comment -

          It looks like archiving artifacts from agents is unaffected

          TBD actually, I'm seeing some weird behavior in my local environment which may be caused by differences between Linux (breaking) and Mac OS (working). Or it's related to stashing/unstashing on controller or agent, unsure.

          If anyone else has a good reproduction, that would be useful.

          Daniel Beck added a comment - It looks like archiving artifacts from agents is unaffected TBD actually, I'm seeing some weird behavior in my local environment which may be caused by differences between Linux (breaking) and Mac OS (working). Or it's related to stashing/unstashing on controller or agent, unsure. If anyone else has a good reproduction, that would be useful.

          Tim Jacomb added a comment -

          I reproduced the above on Mac OS with the ath script I posted, but I don’t have a simple repro

          Tim Jacomb added a comment - I reproduced the above on Mac OS with the ath script I posted, but I don’t have a simple repro

          Daniel Beck added a comment -

          Linux controller (Docker on Mac), Mac OS Agent:

          properties([disableConcurrentBuilds()])
          node('master') {
              dir('athSources') {
                  checkout changelog: true, poll: false, scm: [$class: 'GitSCM', branches: [[name: 'master']],
                              userRemoteConfigs: [[url: 'https://github.com/jenkinsci/acceptance-test-harness.git']]]
                  stash name: 'athSources'
              }
              dir('dest') {
                  unstash name: 'athSources'
              }
          }
          

          master fails on 2.303.3 while it works on 2.303.2. !master (i.e. run on agent) works on both.

          Could you try a few permutations of this script in your environment?

          Daniel Beck added a comment - Linux controller (Docker on Mac), Mac OS Agent: properties([disableConcurrentBuilds()]) node('master') { dir('athSources') { checkout changelog: true, poll: false, scm: [$class: 'GitSCM', branches: [[name: 'master']], userRemoteConfigs: [[url: 'https://github.com/jenkinsci/acceptance-test-harness.git']]] stash name: 'athSources' } dir('dest') { unstash name: 'athSources' } } master fails on 2.303.3 while it works on 2.303.2. !master (i.e. run on agent) works on both. Could you try a few permutations of this script in your environment?

          Jesse Glick added a comment -

          Archiving copies symlinks intact to the controller. (Which is dangerous! But we already defend against escapes.) Stashing always creates a tarball on the controller. In this case the stash works as expected, but the unstash is messed up: canonicalization code intended to defend the controller is running on the agent.

          Jesse Glick added a comment - Archiving copies symlinks intact to the controller. (Which is dangerous! But we already defend against escapes.) Stashing always creates a tarball on the controller. In this case the stash works as expected, but the unstash is messed up: canonicalization code intended to defend the controller is running on the agent.

          Daniel Beck added a comment -

          Archiving copies symlinks intact to the controller. (Which is dangerous! But we already defend against escapes.) Stashing always creates a tarball on the controller. In this case the stash works as expected, but the unstash is messed up

          FTR archiving from an agent archives symlinks as whatever they point to, or if invalid on the agent, as symlinks (unless you disable following symlinks).

          And it sends the files through a tar stream, so it both tars and untars. See https://github.com/jenkinsci/jenkins/blob/2c1c5ee20ea887c0e2aecf94b68cba6e7107740c/core/src/main/java/jenkins/model/StandardArtifactManager.java#L73 and https://github.com/jenkinsci/jenkins/blob/2c1c5ee20ea887c0e2aecf94b68cba6e7107740c/core/src/main/java/hudson/FilePath.java#L2623-L2624

          Daniel Beck added a comment - Archiving copies symlinks intact to the controller. (Which is dangerous! But we already defend against escapes.) Stashing always creates a tarball on the controller. In this case the stash works as expected, but the unstash is messed up FTR archiving from an agent archives symlinks as whatever they point to, or if invalid on the agent, as symlinks (unless you disable following symlinks). And it sends the files through a tar stream, so it both tars and untars. See https://github.com/jenkinsci/jenkins/blob/2c1c5ee20ea887c0e2aecf94b68cba6e7107740c/core/src/main/java/jenkins/model/StandardArtifactManager.java#L73 and https://github.com/jenkinsci/jenkins/blob/2c1c5ee20ea887c0e2aecf94b68cba6e7107740c/core/src/main/java/hudson/FilePath.java#L2623-L2624

          Daniel Beck added a comment -

          Given that nobody else seems to have encountered the problem based on feedback, we probably don't need 2.303.4. Thoughts?

          Daniel Beck added a comment - Given that nobody else seems to have encountered the problem based on feedback, we probably don't need 2.303.4. Thoughts?

          Tim Jacomb added a comment -

          Agreed but we should backport it to 2.319.1

          Tim Jacomb added a comment - Agreed but we should backport it to 2.319.1

          Daniel Beck added a comment -

          Agreed.

          Daniel Beck added a comment - Agreed.

          Tapio Reijonen added a comment - - edited

          For our builds unstashing of a "stash" that contains NPM node_modules/ directory fails for the same reason. The node_modules/ directory has symbolic links. We are hitting this issue with Jenkins LTS 2.303.3 and had to revert to 2.303.2 to avoid the issue.

          java.nio.file.FileSystemException: /home/jenkins/workspace/pipeline_jenkins/node_modules/@babel/helper-compilation-targets/node_modules/browserslist/cli.js: Too many levels of symbolic links at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at hudson.util.IOUtils.copy(IOUtils.java:52) at hudson.FilePath.readFromTar(FilePath.java:2862) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-1-build-1 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1797) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at hudson.FilePath.act(FilePath.java:1166) at hudson.FilePath.act(FilePath.java:1155) at hudson.FilePath.untar(FilePath.java:617) at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:161) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:77) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) Also: java.nio.file.FileSystemException: /home/jenkins/workspace/pipeline_jenkins/node_modules/@babel/helper-compilation-targets/node_modules/browserslist/cli.js: Too many levels of symbolic links at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at hudson.util.IOUtils.copy(IOUtils.java:52) at hudson.FilePath.readFromTar(FilePath.java:2862) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-1-build-1 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1797) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at hudson.FilePath.act(FilePath.java:1166) at hudson.FilePath.act(FilePath.java:1155) at hudson.FilePath.untar(FilePath.java:617) at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:161) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:77) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) Caused: java.io.IOException: Failed to extract node_modules.tar.gz
          

          We confirmed that Jenkins 2.321 fixes the issue. Would imagine that this pattern of caching NPM node_modules/ directory between parallel pipeline stages is fairly common use case. We hope that the fix is also ported to next LTS fairly soon (e.g. 2.303.4).

          Tapio Reijonen added a comment - - edited For our builds unstashing of a "stash" that contains NPM node_modules/ directory fails for the same reason. The node_modules/ directory has symbolic links. We are hitting this issue with Jenkins LTS 2.303.3 and had to revert to 2.303.2 to avoid the issue. java.nio.file.FileSystemException: /home/jenkins/workspace/pipeline_jenkins/node_modules/@babel/helper-compilation-targets/node_modules/browserslist/cli.js: Too many levels of symbolic links at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at hudson.util.IOUtils.copy(IOUtils.java:52) at hudson.FilePath.readFromTar(FilePath.java:2862) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-1-build-1 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1797) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at hudson.FilePath.act(FilePath.java:1166) at hudson.FilePath.act(FilePath.java:1155) at hudson.FilePath.untar(FilePath.java:617) at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:161) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:77) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) Also: java.nio.file.FileSystemException: /home/jenkins/workspace/pipeline_jenkins/node_modules/@babel/helper-compilation-targets/node_modules/browserslist/cli.js: Too many levels of symbolic links at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at hudson.util.IOUtils.copy(IOUtils.java:52) at hudson.FilePath.readFromTar(FilePath.java:2862) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-1-build-1 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1797) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at hudson.FilePath.act(FilePath.java:1166) at hudson.FilePath.act(FilePath.java:1155) at hudson.FilePath.untar(FilePath.java:617) at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:161) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:77) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) Caused: java.io.IOException: Failed to extract node_modules.tar.gz We confirmed that Jenkins 2.321 fixes the issue. Would imagine that this pattern of caching NPM node_modules/ directory between parallel pipeline stages is fairly common use case. We hope that the fix is also ported to next LTS fairly soon (e.g. 2.303.4).

          Daniel Beck added a comment -

          Feedback on 2.303.3 (and not many watchers here) indicate it's not that common, so I doubt we'll release 2.303.4 when 2.319.1 (with this fix) is scheduled for release in less than two weeks.

          (I recommend installing https://plugins.jenkins.io/remoting-security-workaround/ if you cannot immediately upgrade. See docs for compatibility notes.)

          Daniel Beck added a comment - Feedback on 2.303.3 (and not many watchers here) indicate it's not that common, so I doubt we'll release 2.303.4 when 2.319.1 (with this fix) is scheduled for release in less than two weeks. (I recommend installing https://plugins.jenkins.io/remoting-security-workaround/ if you cannot immediately upgrade. See docs for compatibility notes.)

          suryatej yaramada added a comment - - edited
          [Pipeline] stash
          Stashed 92383 file(s)
          [Pipeline] step
          Recording test results
          None of the test reports contained any result
          [Checks API] No suitable checks publisher found.
          [Pipeline] }
          [Pipeline] // stage
          No test report files were found. Configuration error?
          [Pipeline] stage
          [Pipeline] { (Deploy-dev)
          [Pipeline] unstash
          [Pipeline] echo
          Exception: java.io.IOException: Failed to extract source.tar.gz
          [Pipeline] emailext
          Sending email to: dl-mcdautomation@discovery.com
          Connection error sending email, retrying once more in 10 seconds...
          Connection error sending email, retrying once more in 10 seconds...
          Failed after second try sending email
          [Pipeline] step
          Recording test results
          None of the test reports contained any result
          [Checks API] No suitable checks publisher found.
          [Pipeline] }
          [Pipeline] // stage
          No test report files were found. Configuration error?
          [Pipeline] }
          [Pipeline] // node
          [Pipeline] End of Pipeline
          java.nio.file.NoSuchFileException: /opt/jenkins-executor/workspace/Systems-Engineering/continuous-delivery/VanTamPipeline/03.VanTamEngineeringDSuiteUIPipeline/node-v12.18.3-linux-x64/lib/node_modules/npm/bin/npm-cli.js
          	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
          	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
          	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
          	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
          	at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)
          	at java.base/java.nio.file.Files.newOutputStream(Files.java:220)
          	at hudson.util.IOUtils.copy(IOUtils.java:52)
          	at hudson.FilePath.readFromTar(FilePath.java:2862)
          Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to dev-executor
          		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1797)
          		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
          		at hudson.remoting.Channel.call(Channel.java:1001)
          		at hudson.FilePath.act(FilePath.java:1166)
          		at hudson.FilePath.act(FilePath.java:1155)
          		at hudson.FilePath.untar(FilePath.java:617)
          		at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:161)
          		at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:77)
          		at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64)
          		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
          		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          		at java.base/java.lang.Thread.run(Thread.java:834)
          Caused: java.io.IOException: Failed to extract source.tar.gz
          	at hudson.FilePath.readFromTar(FilePath.java:2872)
          	at hudson.FilePath.access$500(FilePath.java:213)
          	at hudson.FilePath$UntarRemote.invoke(FilePath.java:633)
          	at hudson.FilePath$UntarRemote.invoke(FilePath.java:622)
          	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3338)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          	at hudson.remoting.Request$2.run(Request.java:376)
          	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
          	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          	at java.base/java.lang.Thread.run(Thread.java:829)
          [Checks API] No suitable checks publisher found.
          Finished: FAILURE
          

           

          suryatej yaramada added a comment - - edited [Pipeline] stash Stashed 92383 file(s) [Pipeline] step Recording test results None of the test reports contained any result [Checks API] No suitable checks publisher found. [Pipeline] } [Pipeline] // stage No test report files were found. Configuration error? [Pipeline] stage [Pipeline] { (Deploy-dev) [Pipeline] unstash [Pipeline] echo Exception: java.io.IOException: Failed to extract source.tar.gz [Pipeline] emailext Sending email to: dl-mcdautomation@discovery.com Connection error sending email, retrying once more in 10 seconds... Connection error sending email, retrying once more in 10 seconds... Failed after second try sending email [Pipeline] step Recording test results None of the test reports contained any result [Checks API] No suitable checks publisher found. [Pipeline] } [Pipeline] // stage No test report files were found. Configuration error? [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline java.nio.file.NoSuchFileException: /opt/jenkins-executor/workspace/Systems-Engineering/continuous-delivery/VanTamPipeline/03.VanTamEngineeringDSuiteUIPipeline/node-v12.18.3-linux-x64/lib/node_modules/npm/bin/npm-cli.js at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478) at java.base/java.nio.file.Files.newOutputStream(Files.java:220) at hudson.util.IOUtils.copy(IOUtils.java:52) at hudson.FilePath.readFromTar(FilePath.java:2862) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to dev-executor at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1797) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at hudson.FilePath.act(FilePath.java:1166) at hudson.FilePath.act(FilePath.java:1155) at hudson.FilePath.untar(FilePath.java:617) at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:161) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:77) at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:64) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang. Thread .run( Thread .java:834) Caused: java.io.IOException: Failed to extract source.tar.gz at hudson.FilePath.readFromTar(FilePath.java:2872) at hudson.FilePath.access$500(FilePath.java:213) at hudson.FilePath$UntarRemote.invoke(FilePath.java:633) at hudson.FilePath$UntarRemote.invoke(FilePath.java:622) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3338) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:376) at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang. Thread .run( Thread .java:829) [Checks API] No suitable checks publisher found. Finished: FAILURE  

          suryatej yaramada added a comment - - edited

          Not sure if this is related issue but after upgrading to 2.303.3 today we started seeing this issues for our UI deployments where we had following in our pipeline 

           

          stash includes: '**', excludes: 'node_modules/.cache/**', name: 'source'
          }
          catch(e){
          common.notifyOnFailure(e, owner, application)
          throw e
          }
          finally {
          step(: 'JUnitResultArchiver', testResults: '*/build/testresults/.xml', allowEmptyResults: true" target="_blank" rel="noopener">${color:#6cb6ff}class: 'JUnitResultArchiver', testResults: '*/build/testresults/.xml', allowEmptyResults: true)
          }
          }
          }
          stage('Deploy-dev'){
          if (!params.CLEANUP_ONLY) {
          try{
          unstash 'source'
          
          

           

           

           

          suryatej yaramada added a comment - - edited Not sure if this is related issue but after upgrading to 2.303.3 today we started seeing this issues for our UI deployments where we had following in our pipeline    stash includes:  '**' , excludes:  'node_modules/.cache/**' , name:  'source' } catch (e){ common.notifyOnFailure(e, owner, application) throw  e } finally  { step(: 'JUnitResultArchiver' , testResults:  '*/build/testresults/.xml' , allowEmptyResults:  true " target=" _blank " rel=" noopener">${color:#6cb6ff}class: 'JUnitResultArchiver' , testResults:  '*/build/testresults/.xml' , allowEmptyResults:  true ) } } } stage( 'Deploy-dev' ){ if  (!params.CLEANUP_ONLY) { try { unstash  'source'      

          Please let me know if there is any workaround for this issue? or planning to release any new version to patch this?

          suryatej yaramada added a comment - Please let me know if there is any workaround for this issue? or planning to release any new version to patch this?

          Tried reverting back to 2.303.2 but facing this issues

           [plugins : There are 21 failed plugins: workflow-durable-task-step; workflow-basic-steps; gradle; pipeline-model-definition; pipeline-as-yaml; external-workspace-manager; blueocean-pipeline-api-impl; blueocean-bitbucket-pipeline; workflow-aggregator; tekton-client; blueocean-github-pipeline; throttle-concurrents; blueocean-pipeline-editor; github-autostatus; docker-workflow; blueocean-events; templating-engine; artifactory; blueocean-git-pipeline; dark-theme; blueocean]
          
          

          suryatej yaramada added a comment - Tried reverting back to 2.303.2 but facing this issues [plugins : There are 21 failed plugins: workflow-durable-task-step; workflow-basic-steps; gradle; pipeline-model-definition; pipeline-as-yaml; external-workspace-manager; blueocean-pipeline-api-impl; blueocean-bitbucket-pipeline; workflow-aggregator; tekton-client; blueocean-github-pipeline; throttle-concurrents; blueocean-pipeline-editor; github-autostatus; docker-workflow; blueocean-events; templating-engine; artifactory; blueocean-git-pipeline; dark-theme; blueocean]

          After reinstalling Nodes and Processes Plugin looks like our deployments functionality restored where we are on 2.303.2 version .. But it would be nice if we can have a fix release for this

          suryatej yaramada added a comment - After reinstalling Nodes and Processes Plugin looks like our deployments functionality restored where we are on 2.303.2 version .. But it would be nice if we can have a fix release for this

          Jesse Glick added a comment -

          yrsurya as in issue metadata, this is fixed in 2.320, and planned for 2.319.1. We were not currently planning a 2.303.4 just for this issue.

          The workaround should be simple: do not attempt to stash symlinks. If you have a directory tree including symlinks which you need to save & restore in that form, you can always run for example tar from a shell before stash’ing the tarball, and untar after unstash; specific options to tar will give you full control over how the symlinks are interpreted in your build workspace.

          Jesse Glick added a comment - yrsurya as in issue metadata, this is fixed in 2.320, and planned for 2.319.1. We were not currently planning a 2.303.4 just for this issue. The workaround should be simple: do not attempt to stash symlinks. If you have a directory tree including symlinks which you need to save & restore in that form, you can always run for example tar from a shell before stash ’ing the tarball, and untar after unstash ; specific options to tar will give you full control over how the symlinks are interpreted in your build workspace.

          Daniel Beck added a comment -

          planned for 2.319.1

          Expected release date is Dec 1, so just another week too

          Daniel Beck added a comment - planned for 2.319.1 Expected release date is Dec 1, so just another week too

            danielbeck Daniel Beck
            timja Tim Jacomb
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: