Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55984

Unable to perform Unit tests due to error "started.FATAL: Log statements out of sync: current test suite not exists

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Fixed but Unreleased (View Workflow)
    • Priority: Blocker
    • Resolution: Fixed
    • Component/s: xcode-plugin
    • Labels:
      None
    • Similar Issues:
    • Sprint:
      Evergreen - Milestone 2

      Description

      Jenkins Version :

      2.150.2

      Xcode Plugin Version : 

        2.0.10

      Issue:

      Unit test job on jenkins stops with

      started.FATAL: Log statements out of sync: current test suite '.XXXXXX.xctest' not exists

      The same build, if i try unit test with command line it succeeds.

        Attachments

          Issue Links

            Activity

            Hide
            spinselling15 Spin Selling added a comment -

            Xcode used to compile : Version 10.1

            Show
            spinselling15 Spin Selling added a comment - Xcode used to compile : Version 10.1
            Hide
            kazuhidet Kazuhide Takahashi added a comment -

            Spin Selling

            This is a known problem, Xcode Plugin tries to parse the output from xcodebuild and generate JUnit compatible test results from it, but unfortunately it is incompatible with Xcode 10's xcodebuild output, It can not be parsed.

            If you only need to run tests and you do not intend to make any report on the results, you should not set logfileOutputDirectory then the output of xcodebuild will not be parsed, so no error should occur.

            Or the following ISSUE may be related?

            https://issues.jenkins-ci.org/browse/JENKINS-53663

            Show
            kazuhidet Kazuhide Takahashi added a comment - Spin Selling This is a known problem, Xcode Plugin tries to parse the output from xcodebuild and generate JUnit compatible test results from it, but unfortunately it is incompatible with Xcode 10's xcodebuild output, It can not be parsed. If you only need to run tests and you do not intend to make any report on the results, you should not set logfileOutputDirectory then the output of xcodebuild will not be parsed, so no error should occur. Or the following ISSUE may be related? https://issues.jenkins-ci.org/browse/JENKINS-53663
            Hide
            kazuhidet Kazuhide Takahashi added a comment -

            With the previous logic for the retrieve test results, the Xcode plugin tried to analyze the console output under processing of xcodebuild and try to obtain the test result from it, but this method did not work well in a new development environment such as Xcode 10 .
            So I change the this mechanism now test result is taken from test result summary "TestSummaries.plist" output by xcodebuild.

            Still this feature is not perfect because it is a test phase, but if you are interested, get a snapshot below and try using it.

            https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fxcode-plugin/detail/PR-100/1/artifacts

            Show
            kazuhidet Kazuhide Takahashi added a comment - With the previous logic for the retrieve test results, the Xcode plugin tried to analyze the console output under processing of xcodebuild and try to obtain the test result from it, but this method did not work well in a new development environment such as Xcode 10 . So I change the this mechanism now test result is taken from test result summary "TestSummaries.plist" output by xcodebuild. Still this feature is not perfect because it is a test phase, but if you are interested, get a snapshot below and try using it. https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fxcode-plugin/detail/PR-100/1/artifacts
            Hide
            kazuhidet Kazuhide Takahashi added a comment -

            Pipeline script like this. (resultBundlePath invokes new logic.)

                    stage('test') {
                        steps {
                            xcodeBuild(
                                xcodeName: "${XCODE_VERSION_NAME}",
                                cleanBeforeBuild: true,
                                allowFailingBuildResults: true,
                                xcodeSchema: "${PROJECT_SCHEMA}",
                                developmentTeamName: "${DEVELOPMENT_TEAM_NAME}",
                                cfBundleShortVersionStringValue: "${SHORT_VERSION}",
                                cfBundleVersionValue: "${VERSION}",
                                bundleID: "${BUNDLE_ID}",
                                configuration: "Debug",
                                sdk: 'iphonesimulator',
                                xcodebuildArguments: "test -destination 'platform=iOS Simulator,name=iPhone 7,OS=10.3.1' -enableCodeCoverage YES CODE_SIGNING_REQUIRED=NO COMPILER_INDEX_STORE_ENABLE=NO CLANG_ALLOW_NON_MODULAR_INCLUDES_IN_FRAMEWORK_MODULES=YES",
                                signingMethod: 'automatic',
                                buildIpa: false,
                                generateArchive: false,
                                ipaExportMethod: "development",
                                ipaName: "${BUILD_TARGET}",
                                logfileOutputDirectory: 'BuildLogs',
                                useLegacyBuildSystem: true,
                                resultBundlePath: 'TestResult',
                                cleanResultBundlePath: true
                            )
                        }
                    }
            
            Show
            kazuhidet Kazuhide Takahashi added a comment - Pipeline script like this. (resultBundlePath invokes new logic.) stage('test') { steps { xcodeBuild( xcodeName: "${XCODE_VERSION_NAME}", cleanBeforeBuild: true, allowFailingBuildResults: true, xcodeSchema: "${PROJECT_SCHEMA}", developmentTeamName: "${DEVELOPMENT_TEAM_NAME}", cfBundleShortVersionStringValue: "${SHORT_VERSION}", cfBundleVersionValue: "${VERSION}", bundleID: "${BUNDLE_ID}", configuration: "Debug", sdk: 'iphonesimulator', xcodebuildArguments: "test -destination 'platform=iOS Simulator,name=iPhone 7,OS=10.3.1' -enableCodeCoverage YES CODE_SIGNING_REQUIRED=NO COMPILER_INDEX_STORE_ENABLE=NO CLANG_ALLOW_NON_MODULAR_INCLUDES_IN_FRAMEWORK_MODULES=YES", signingMethod: 'automatic', buildIpa: false, generateArchive: false, ipaExportMethod: "development", ipaName: "${BUILD_TARGET}", logfileOutputDirectory: 'BuildLogs', useLegacyBuildSystem: true, resultBundlePath: 'TestResult', cleanResultBundlePath: true ) } }
            Hide
            kazuhidet Kazuhide Takahashi added a comment -

            With the previous logic for the retrieve test results, the Xcode plugin tried to analyze the console output under processing of xcodebuild and try to obtain the test result from it, but this method did not work well in a new development environment such as Xcode 10 .
            So I change the this mechanism now test result is taken from test result summary "TestSummaries.plist" output by xcodebuild.

            Show
            kazuhidet Kazuhide Takahashi added a comment - With the previous logic for the retrieve test results, the Xcode plugin tried to analyze the console output under processing of xcodebuild and try to obtain the test result from it, but this method did not work well in a new development environment such as Xcode 10 . So I change the this mechanism now test result is taken from test result summary "TestSummaries.plist" output by xcodebuild.

              People

              Assignee:
              kazuhidet Kazuhide Takahashi
              Reporter:
              spinselling15 Spin Selling
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: