Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62471

Xcode Plugin - Unable to run Unit Tests

    XMLWordPrintable

Details

    Description

      Plugin version: 2.0.6
      Xcode version: 11.3.1
      Issue:
      I am following the exact same steps to set up unit tests via Xcode plugin but it doesn't work. Throws the error saying: 

      {{xcodebuild: error: Failed to build workspace MyWorkspace with scheme My-scheme.Reason: The scheme 'My-scheme' is not configured for Running.

      It runs perfectly while using xcodebuild test CLI command but I would like to see if it is possible via Plugin to maintain consistency. }}

      Setting up a unit test step

      Add the XCode build step and this time specify your unit test target (e.g. MyAppTests), configuration (e.g. Debug) and the SDK (e.g. Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.1.sdk/).
      Leave all the checkboxes unselected. The SDK needs to be supplied here as the unit tests will only be run when building under the simulator's SDK.

      Attachments

        Activity

          urmil025 Urmil Shah created issue -

          Hello.

          +1 about this issue.

          I currently use this command line to run unit test. I'ld like to use the XCode Plugin instead.

          xcodebuild -workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test

           

          So i have the same issue while launching with plugin DSL method :

          xcodeBuild(
          xcodeSchema: "myprojectTests",
          xcodeName: 'latest',
          configuration: 'Debug',
          xcodeWorkspaceFile: 'myproject',
          sdk: 'iphonesimulator13.2',
          xcodebuildArguments: "test -destination \"name=iPhone 11\" -enableCodeCoverage YES",
          resultBundlePath: 'test-result',
          )

           

          In fact, by default, the "build" parameters is added in xcodebuild command instead of "test". 

          Can you confirm this is an issue ?

          I can eventually fix and create a PR to fix this issue.

          Thank you.

           

           

          chrismcx Christophe MICHAUX added a comment - Hello. +1 about this issue. I currently use this command line to run unit test. I'ld like to use the XCode Plugin instead. xcodebuild -workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test   So i have the same issue while launching with plugin DSL method : xcodeBuild( xcodeSchema: "myprojectTests", xcodeName: 'latest', configuration: 'Debug', xcodeWorkspaceFile: 'myproject', sdk: 'iphonesimulator13.2', xcodebuildArguments: "test -destination \"name=iPhone 11\" -enableCodeCoverage YES", resultBundlePath: 'test-result', )   In fact, by default, the "build" parameters is added in xcodebuild command instead of "test".  Can you confirm this is an issue ? I can eventually fix and create a PR to fix this issue. Thank you.    
          urmil025 Urmil Shah added a comment -

          Yes, I believe this IS the issue. The doc mentions that you pass on the test target/scheme to the same build plugin but ideally this runs it as a build and not test. 

          Have you tried using the xcodebuildArguments to specify "-workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test" keeping everything else blank? 

          urmil025 Urmil Shah added a comment - Yes, I believe this IS the issue. The doc mentions that you pass on the test target/scheme to the same build plugin but ideally this runs it as a build and not test.  Have you tried using the  xcodebuildArguments  to specify " -workspace 'myproject.xcworkspace' -scheme "myprojectTests" -destination "name=iPhone 11,OS=13.3" -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES  test"  keeping everything else blank? 
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 i am facing issue generating unit test report. Everything is configured properly as per document but still i am getting this "Recording test results None of the test reports contained any result".

          Jenkins version: 2.289.1

          Xcode: 12.5

          Your help would be really appreciated.

          pinkeshgjr Pinkesh added a comment - - edited urmil025  i am facing issue generating unit test report. Everything is configured properly as per document but still i am getting this "Recording test results None of the test reports contained any result". Jenkins version: 2.289.1 Xcode: 12.5 Your help would be really appreciated.
          urmil025 Urmil Shah added a comment -

          pinkeshgjr Are you using Junit? It is quite possible that Junit is not able to detect the test outputs and hence reports that there are no test results. 

          I have added this to my xcodebuild CLI command which reports the results in the format that is required by Jenkins. See if this helps. 

          xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS[0]}'
          urmil025 Urmil Shah added a comment - pinkeshgjr  Are you using Junit? It is quite possible that Junit is not able to detect the test outputs and hence reports that there are no test results.  I have added this to my xcodebuild CLI command which reports the results in the format that is required by Jenkins. See if this helps.  xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS[0]}'
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 Just tried using that command now i am getting this error.
          -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS[0]}
          xcodebuild: error: invalid option '–report'

          i have added this in Custom xcodebuild arguments. Let me know if i am doing it wrong.

          pinkeshgjr Pinkesh added a comment - - edited urmil025  Just tried using that command now i am getting this error. -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list xcodebuild ... | xcpretty --report junit && exit ${PIPESTATUS [0] } xcodebuild: error: invalid option '–report' i have added this in Custom xcodebuild arguments. Let me know if i am doing it wrong.
          urmil025 Urmil Shah added a comment -

          Just add  

          -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS[0]}

          This will need you to have xcpretty installed on the machine. Find more details here - https://github.com/xcpretty/xcpretty 

          urmil025 Urmil Shah added a comment - Just add   -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS[0]} This will need you to have xcpretty installed on the machine. Find more details here - https://github.com/xcpretty/xcpretty  
          pinkeshgjr Pinkesh added a comment -

          urmil025 i just tried replacing command that you shared also installed xcpretty with this command 

          gem install xcpretty.

          Still getting same error, do i need to adding any plugin?

          pinkeshgjr Pinkesh added a comment - urmil025  i just tried replacing command that you shared also installed  xcpretty  with this command  gem install xcpretty. Still getting same error, do i need to adding any plugin?
          urmil025 Urmil Shah added a comment -

          What is your full command? Also, try running only

           xcpretty --help

          in your build and see if Jenkins recognizes it. you might need to specify the entire path or add it to your bash file. 

          urmil025 Urmil Shah added a comment - What is your full command? Also, try running only xcpretty --help in your build and see if Jenkins recognizes it. you might need to specify the entire path or add it to your bash file. 
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 i just tried adding xcpretty --help and Jenkins recognise it. There is no command --report

          pinkeshgjr Pinkesh added a comment - - edited urmil025  i just tried adding xcpretty --help and Jenkins recognise it. There is no command --report
          urmil025 Urmil Shah added a comment -

          I don't know how you installed it but I am running xcpretty with --report option and it is just fine. Try doing so locally, running the entire command with xcpretty, and see if that helps. 

          urmil025 Urmil Shah added a comment - I don't know how you installed it but I am running xcpretty with --report option and it is just fine. Try doing so locally, running the entire command with xcpretty, and see if that helps. 
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 I am using Jenkins locally. I tried adding -r junit in Custom xcodebuild arguments. Still same error, lool like xcpretty is not working with Jenkins. Is there any additional setup have you done?

          -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS[0]}

          pinkeshgjr Pinkesh added a comment - - edited urmil025  I am using Jenkins locally. I tried adding -r junit in Custom xcodebuild arguments. Still same error, lool like xcpretty is not working with Jenkins. Is there any additional setup have you done? -allowProvisioningUpdates -enableCodeCoverage YES ONLY_ACTIVE_ARCH=YES test -list | xcpretty --report junit && exit ${PIPESTATUS [0] }
          pinkeshgjr Pinkesh added a comment - - edited

          urmil025 i just removed xcode from jenkins and added execute shell command in configuration and it works fine.

          xcodebuild -workspace myproject.xcworkspace -scheme myprojectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -enableCodeCoverage YES test | xcpretty --report junit && exit ${PIPESTATUS[0]}

          Now i am facing some other issue like below. 
          xcodebuild[99034:318512] NSFileHandle couldn't write. Exception: *** -[_NSStdIOFileHandle writeData:]: Broken pipe

          pinkeshgjr Pinkesh added a comment - - edited urmil025  i just removed xcode from jenkins and added execute shell command in configuration and it works fine. xcodebuild -workspace myproject.xcworkspace -scheme myprojectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -enableCodeCoverage YES test | xcpretty --report junit && exit ${PIPESTATUS [0] } Now i am facing some other issue like below.  xcodebuild [99034:318512] NSFileHandle couldn't write. Exception: *** - [_NSStdIOFileHandle writeData:] : Broken pipe
          urmil025 Urmil Shah added a comment -

          That seems to be something in your code. I see some solutions on Stackoverflow. 

          urmil025 Urmil Shah added a comment - That seems to be something in your code. I see some solutions on Stackoverflow. 
          pinkeshgjr Pinkesh added a comment -

          urmil025 There was issue with path. It works fine now but is there any way to generate code coverage report?

           

          xcodebuild -workspace MyProject.xcworkspace -scheme MyProjectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -derivedDataPath DerivedData/ -enableCodeCoverage YES test | /usr/local/bin/xcpretty --report junit --output test-reports/reports.xml && exit ${PIPESTATUS[0]}

          pinkeshgjr Pinkesh added a comment - urmil025  There was issue with path. It works fine now but is there any way to generate code coverage report?   xcodebuild -workspace MyProject.xcworkspace -scheme MyProjectTests -destination 'platform=iOS Simulator,name=iPhone 12 Pro' -derivedDataPath DerivedData/ -enableCodeCoverage YES test | /usr/local/bin/xcpretty --report junit --output test-reports/reports.xml && exit ${PIPESTATUS [0] }

          People

            Unassigned Unassigned
            urmil025 Urmil Shah
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: