Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62471

Xcode Plugin - Unable to run Unit Tests

    XMLWordPrintable

Details

    Description

      Plugin version: 2.0.6
      Xcode version: 11.3.1
      Issue:
      I am following the exact same steps to set up unit tests via Xcode plugin but it doesn't work. Throws the error saying: 

      {{xcodebuild: error: Failed to build workspace MyWorkspace with scheme My-scheme.Reason: The scheme 'My-scheme' is not configured for Running.

      It runs perfectly while using xcodebuild test CLI command but I would like to see if it is possible via Plugin to maintain consistency. }}

      Setting up a unit test step

      Add the XCode build step and this time specify your unit test target (e.g. MyAppTests), configuration (e.g. Debug) and the SDK (e.g. Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.1.sdk/).
      Leave all the checkboxes unselected. The SDK needs to be supplied here as the unit tests will only be run when building under the simulator's SDK.

      Attachments

        Activity

          Hello.

          +1 about this issue.

          I currently use this command line to run unit test. I'ld like to use the XCode Plugin instead.

          xcodebuild -workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test

           

          So i have the same issue while launching with plugin DSL method :

          xcodeBuild(
          xcodeSchema: "myprojectTests",
          xcodeName: 'latest',
          configuration: 'Debug',
          xcodeWorkspaceFile: 'myproject',
          sdk: 'iphonesimulator13.2',
          xcodebuildArguments: "test -destination \"name=iPhone 11\" -enableCodeCoverage YES",
          resultBundlePath: 'test-result',
          )

           

          In fact, by default, the "build" parameters is added in xcodebuild command instead of "test". 

          Can you confirm this is an issue ?

          I can eventually fix and create a PR to fix this issue.

          Thank you.

           

           

          chrismcx Christophe MICHAUX added a comment - Hello. +1 about this issue. I currently use this command line to run unit test. I'ld like to use the XCode Plugin instead. xcodebuild -workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test   So i have the same issue while launching with plugin DSL method : xcodeBuild( xcodeSchema: "myprojectTests", xcodeName: 'latest', configuration: 'Debug', xcodeWorkspaceFile: 'myproject', sdk: 'iphonesimulator13.2', xcodebuildArguments: "test -destination \"name=iPhone 11\" -enableCodeCoverage YES", resultBundlePath: 'test-result', )   In fact, by default, the "build" parameters is added in xcodebuild command instead of "test".  Can you confirm this is an issue ? I can eventually fix and create a PR to fix this issue. Thank you.    
          urmil025 Urmil Shah added a comment -

          Yes, I believe this IS the issue. The doc mentions that you pass on the test target/scheme to the same build plugin but ideally this runs it as a build and not test. 

          Have you tried using the xcodebuildArguments to specify "-workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test" keeping everything else blank? 

          urmil025 Urmil Shah added a comment - Yes, I believe this IS the issue. The doc mentions that you pass on the test target/scheme to the same build plugin but ideally this runs it as a build and not test.  Have you tried using the  xcodebuildArguments  to specify " -workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test"  keeping everything else blank? 
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 i am facing issue generating unit test report. Everything is configured properly as per document but still i am getting this "Recording test results None of the test reports contained any result".

          Jenkins version: 2.289.1

          Xcode: 12.5

          Your help would be really appreciated.

          pinkeshgjr Pinkesh added a comment - - edited urmil025  i am facing issue generating unit test report. Everything is configured properly as per document but still i am getting this "Recording test results None of the test reports contained any result". Jenkins version: 2.289.1 Xcode: 12.5 Your help would be really appreciated.
          urmil025 Urmil Shah added a comment -

          pinkeshgjr Are you using Junit? It is quite possible that Junit is not able to detect the test outputs and hence reports that there are no test results. 

          I have added this to my xcodebuild CLI command which reports the results in the format that is required by Jenkins. See if this helps. 

          xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS[0]}'
          urmil025 Urmil Shah added a comment - pinkeshgjr  Are you using Junit? It is quite possible that Junit is not able to detect the test outputs and hence reports that there are no test results.  I have added this to my xcodebuild CLI command which reports the results in the format that is required by Jenkins. See if this helps.  xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS[0]}'
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 Just tried using that command now i am getting this error.
          -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS[0]}
          xcodebuild: error: invalid option '–report'

          i have added this in Custom xcodebuild arguments. Let me know if i am doing it wrong.

          pinkeshgjr Pinkesh added a comment - - edited urmil025  Just tried using that command now i am getting this error. -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS [0] } xcodebuild: error: invalid option '–report' i have added this in Custom xcodebuild arguments. Let me know if i am doing it wrong.
          urmil025 Urmil Shah added a comment -

          Just add  

          -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS[0]}

          This will need you to have xcpretty installed on the machine. Find more details here - https://github.com/xcpretty/xcpretty 

          urmil025 Urmil Shah added a comment - Just add   -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS[0]} This will need you to have xcpretty installed on the machine. Find more details here - https://github.com/xcpretty/xcpretty  
          pinkeshgjr Pinkesh added a comment -

          urmil025 i just tried replacing command that you shared also installed xcpretty with this command 

          gem install xcpretty.

          Still getting same error, do i need to adding any plugin?

          pinkeshgjr Pinkesh added a comment - urmil025  i just tried replacing command that you shared also installed  xcpretty  with this command  gem install xcpretty. Still getting same error, do i need to adding any plugin?
          urmil025 Urmil Shah added a comment -

          What is your full command? Also, try running only

           xcpretty --help

          in your build and see if Jenkins recognizes it. you might need to specify the entire path or add it to your bash file. 

          urmil025 Urmil Shah added a comment - What is your full command? Also, try running only xcpretty --help in your build and see if Jenkins recognizes it. you might need to specify the entire path or add it to your bash file. 
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 i just tried adding xcpretty --help and Jenkins recognise it. There is no command --report

          pinkeshgjr Pinkesh added a comment - - edited urmil025  i just tried adding xcpretty --help and Jenkins recognise it. There is no command --report
          urmil025 Urmil Shah added a comment -

          I don't know how you installed it but I am running xcpretty with --report option and it is just fine. Try doing so locally, running the entire command with xcpretty, and see if that helps. 

          urmil025 Urmil Shah added a comment - I don't know how you installed it but I am running xcpretty with --report option and it is just fine. Try doing so locally, running the entire command with xcpretty, and see if that helps. 
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 I am using Jenkins locally. I tried adding -r junit in Custom xcodebuild arguments. Still same error, lool like xcpretty is not working with Jenkins. Is there any additional setup have you done?

          -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS[0]}

          pinkeshgjr Pinkesh added a comment - - edited urmil025  I am using Jenkins locally. I tried adding -r junit in Custom xcodebuild arguments. Still same error, lool like xcpretty is not working with Jenkins. Is there any additional setup have you done? -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS [0] }
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 i just removed xcode from jenkins and added execute shell command in configuration and it works fine.

          xcodebuild -workspace myproject.xcworkspace -scheme myprojectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -enableCodeCoverage YES test | xcpretty --report junit && exit ${PIPESTATUS[0]}

          Now i am facing some other issue like below. 
          xcodebuild[99034:318512] NSFileHandle couldn't write. Exception: *** -[_NSStdIOFileHandle writeData:]: Broken pipe

          pinkeshgjr Pinkesh added a comment - - edited urmil025  i just removed xcode from jenkins and added execute shell command in configuration and it works fine. xcodebuild -workspace myproject.xcworkspace -scheme myprojectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -enableCodeCoverage YES test | xcpretty --report junit && exit ${PIPESTATUS [0] } Now i am facing some other issue like below.  xcodebuild [99034:318512] NSFileHandle couldn't write. Exception: *** - [_NSStdIOFileHandle writeData:] : Broken pipe
          urmil025 Urmil Shah added a comment -

          That seems to be something in your code. I see some solutions on Stackoverflow. 

          urmil025 Urmil Shah added a comment - That seems to be something in your code. I see some solutions on Stackoverflow. 
          pinkeshgjr Pinkesh added a comment -

          urmil025 There was issue with path. It works fine now but is there any way to generate code coverage report?

           

          xcodebuild -workspace MyProject.xcworkspace -scheme MyProjectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -derivedDataPath DerivedData/ -enableCodeCoverage YES test | /usr/local/bin/xcpretty --report junit --output test-reports/reports.xml && exit ${PIPESTATUS[0]}

          pinkeshgjr Pinkesh added a comment - urmil025  There was issue with path. It works fine now but is there any way to generate code coverage report?   xcodebuild -workspace MyProject.xcworkspace -scheme MyProjectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -derivedDataPath DerivedData/ -enableCodeCoverage YES test | /usr/local/bin/xcpretty --report junit --output test-reports/reports.xml && exit ${PIPESTATUS [0] }

          People

            Unassigned Unassigned
            urmil025 Urmil Shah
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: