Add a flag to disable Xcode version check failure (#436)

* Add a flag to disable Xcode version check failure

For Xcode 10 and 11, many Xcode PRs to update bluepill have required simply
updating the default SDK and Xcode version. Throughout most releases of Xcode
11.4, bluepill has worked without a lot of major changes. At the speed that Apple
is moving on Xcode releases, it's a lot of effort to bump these: PRs,
branching, rebuilding, etc.

This flag makes it a bit easier for us to test new versions of Xcode
without having to create a build and PR for bluepill.

For now, it's defaulted, but perhaps we should make it default to on and print
a warning if it doesn't match. Additionally, we should consider dynamically
pulling in the runtime version

Co-authored-by: Ravi Mandala <rmandala@linkedin.com>

README.md

@@ -58,6 +58,7 @@ A full list supported options are listed here.
 |:----------------------:|:----------------------:|-------------------------------------------------------------------------------------|:--------:|:----------------:|
 |          app           |           -a           | The path to the host application to execute (your `.app`)                           |     N    | n/a              |
 |     xctestrun-path     |                        | The path to the `.xctestrun` file that xcode leaves when you `build-for-testing`.   |     Y    | n/a              |
+|     test-plan-path     |                        | The path of a json file which describes the test plan. It is equivalent to the `.xctestrun` file generated by Xcode, but it can be generated by a different build system, e.g. Bazel |     Y    | n/a              |
 |       output-dir       |           -o           | Directory where to put output log files. **(bluepill only)**                        |     Y    | n/a              |
 |         config         |           -c           | Read options from the specified configuration file instead of the command line.     |     N    | n/a              |
 |         device         |           -d           | On which device to run the app.                                                     |     N    | iPhone 8         |
@@ -87,6 +88,8 @@ A full list supported options are listed here.
 | screenshots-directory  |           n/a          | Directory where simulator screenshots for failed ui tests will be stored.           |     N    | n/a              |
 |       video-paths      |           -V           | A list of videos that will be saved in the simulators.                              |     N    | n/a              |
 |       image-paths      |           -I           | A list of images that will be saved in the simulators.                              |     N    | n/a              |
+| unsafe-skip-xcode-version-check |               | Skip Xcode version check                                                            |     N    | NO               |
+
 
 ## Exit Status
 
@@ -131,6 +134,8 @@ Bluepill only works with **Xcode 11.3**. If you're looking for old Xcode support
 * [Xcode-11.1](https://github.com/linkedin/bluepill/tree/xcode-11.1)
 * [Xcode-11.2](https://github.com/linkedin/bluepill/tree/xcode-11.2)
 
+**Note:** Refer to `unsafe-skip-xcode-version-check` flag introduced in `Bluepill v5.2.1` relaxing the Xcode version checks. Please make sure it is well tested and the underlying risks are understand.
+
 ## Acknowledgement
 
 Bluepill was inspired by [parallel iOS test](https://github.com/plu/parallel_ios_tests) and Facebook’s [xctool](https://github.com/facebook/xctool) and [FBSimulatorControl](https://github.com/facebook/FBSimulatorControl). The Bluepill icon was created by [Maria Iu](https://www.linkedin.com/in/mariaiu/).

bp/src/BPConfiguration.h

@@ -90,6 +90,7 @@ typedef NS_ENUM(NSInteger, BPProgram) {
 @property (nonatomic, strong) NSString *scriptFilePath;
 @property (nonatomic) BOOL headlessMode;
 @property (nonatomic) BOOL cloneSimulator;
+@property (nonatomic) BOOL unsafeSkipXcodeVersionCheck;
 @property (nonatomic, strong) NSNumber *numSims;
 @property (nonatomic) BOOL listTestsOnly;
 @property (nonatomic) BOOL quiet;

bp/src/BPConfiguration.m

@@ -142,6 +142,8 @@ typedef NS_OPTIONS(NSUInteger, BPOptionType) {
         "A script that will be called after the simulator is booted, but before tests are run. Can be used to do any setup (e.g. installing certs). The environment will contain $BP_DEVICE_ID with the ID of the simulator and $BP_DEVICE_PATH with its full path. Exit with zero for success and non-zero for failure."},
     {364, "test-plan-path", BP_MASTER | BP_SLAVE, NO, NO, required_argument, NULL, BP_VALUE | BP_PATH, "testPlanPath",
         "The path of a json file which describes the test plan. It is equivalent to the .xctestrun file generated by Xcode, but it can be generated by a different build system, e.g. Bazel"},
+    {365, "unsafe-skip-xcode-version-check", BP_MASTER | BP_SLAVE, NO, NO, no_argument, "Off", BP_VALUE | BP_BOOL , "unsafeSkipXcodeVersionCheck",
+    "   "},
 
     {0, 0, 0, 0, 0, 0, 0}
 };
@@ -674,18 +676,21 @@ - (BOOL)validateConfigWithError:(NSError *__autoreleasing *)errPtr {
     //Check if xcode version running on the host match the intended Bluepill branch: Xcode 9 branch is not backward compatible
     NSString *xcodeVersion = [BPUtils runShell:@"xcodebuild -version"];
     [BPUtils printInfo:DEBUGINFO withString:@"xcode build version: %@", xcodeVersion];
-    if ([xcodeVersion rangeOfString:@BP_DEFAULT_XCODE_VERSION].location == NSNotFound) {
-        BP_SET_ERROR(errPtr, @"ERROR: Invalid Xcode version:\n%s;\nOnly %s is supported\n", [xcodeVersion UTF8String], BP_DEFAULT_XCODE_VERSION);
-        return NO;
-    }
 
-    //Check if Bluepill compile time Xcode version is matched with Bluepill runtime Xcode version
-    //Senario to prevent: Bluepill is compiled with Xcode 8, but runs with host installed with Xcode 9
-    //Only compare major and minor version version Exg. 9.1 == 9.1
-    if (![[[BPUtils getXcodeRuntimeVersion] substringToIndex:4] isEqualToString:@BP_DEFAULT_XCODE_VERSION]) {
-        BP_SET_ERROR(errPtr, @"ERROR: Bluepill runtime version %s and compile time version %s are mismatched\n",
-                     [[[BPUtils getXcodeRuntimeVersion] substringToIndex:4] UTF8String], [@BP_DEFAULT_XCODE_VERSION UTF8String]);
-        return NO;
+    if (!self.unsafeSkipXcodeVersionCheck) {
+        if ([xcodeVersion rangeOfString:@BP_DEFAULT_XCODE_VERSION].location == NSNotFound) {
+            BP_SET_ERROR(errPtr, @"ERROR: Invalid Xcode version:\n%s;\nOnly %s is supported\n", [xcodeVersion UTF8String], BP_DEFAULT_XCODE_VERSION);
+            return NO;
+        }
+
+        // Check if Bluepill compile time Xcode version is matched with Bluepill runtime Xcode version
+        // This check prevents Bluepill compiled with Xcode 8 running on host installed with Xcode 9
+        // Only compare major and minor version version Eg. 11.2 ~ 11.2.1 but 11.2 <> 11.3
+        if (![[[BPUtils getXcodeRuntimeVersion] substringToIndex:4] isEqualToString:@BP_DEFAULT_XCODE_VERSION]) {
+            BP_SET_ERROR(errPtr, @"ERROR: Bluepill runtime version %s and compile time version %s are mismatched\n",
+                         [[[BPUtils getXcodeRuntimeVersion] substringToIndex:4] UTF8String], [@BP_DEFAULT_XCODE_VERSION UTF8String]);
+            return NO;
+        }
     }
 
     if (self.deleteSimUDID) {