Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔧 ⬆️ 🚸 Upgrade android to API 33 #1016

Merged

Conversation

shankari
Copy link
Contributor

  • ⬆️ Upgrade android to API 33
  • 🔧 build script updates
  • 🚸 permission usability updates

- Support building individual platforms with the prod JS but dev native code
    - This will allow us to test the prod JS by connecting with a debugger (if needed)
    - Also allows quicker dev cycle by building only one platform at a time
      (note that for testing outside the devapp, we need to build the app every time)
    - Ensures that we don't need to get off the VPN just to build iOS
        - Ensures that we can build prod JS iOS version for archival and submission
- Support building android with the prod JS and prod native code
    - To support building the aab to upload to the play store

Testing done:
- Used for building the NREL OpenPATH app
So that we use webpack for the javascript properly

Testing done:
- Used to build the NREL OpenPATH app
So that we can archive it to check out old functionality if needed
@shankari
Copy link
Contributor Author

@niccolopaganini for visibility into the state of this change

@shankari
Copy link
Contributor Author

Cleared the decks before starting further changes; to ensure no pollution with old state

$ rm ./hooks/before_prepare/download_translation.js.bak
$ rm -rf platforms/
$ rm -rf plugins/
$ rm -rf node_modules/
$ diff package.cordovabuild.json package.json
$ diff config.cordovabuild.xml config.xml

@shankari
Copy link
Contributor Author

shankari commented Aug 19, 2023

We will first implement the upgrade to the cordova platform and the cordova file plugin to avoid too many overlapping changes.

After making the changes to package.json,

$ bash setup/setup_ios_native.sh
Found       23 plugins, expected 23
All plugins installed successfully!
$ bash setup/setup_android_native.sh
ANDROID_HOME and ANDROID_SDK_ROOT not set, android SDK not found, exiting
$ export ANDROID_HOME=~/Library/Android/sdk
$ bash setup/setup_android_native.sh
Found       23 plugins, expected 23
All plugins installed successfully!
$ source setup/activate_native.sh
$ npm run build-dev

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.6/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 20s
1 actionable task: 1 executed
Subproject Path: CordovaLib
Subproject Path: app
Downloading https://services.gradle.org/distributions/gradle-7.6-all.zip
...............10%................20%................30%................40%...............50%................60%................70%................80%...............90%................100%

> Configure project :app
Adding classpath: com.google.gms:google-services:4.3.3
<====---------> 33% CONFIGURING [5m 14s]
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> :app
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE

       1.  Dependency 'androidx.appcompat:appcompat-resources:1.6.1' requires libraries and applications that
           depend on it to compile against version 33 or later of the
           Android APIs.

           :app is currently compiled against android-32.

           Recommended action: Update this project to use a newer compileSdkVersion
           of at least 33, for example 33.

Doh!

Full logs:
first_failed_build.gz

@shankari
Copy link
Contributor Author

Bumped up

        <preference name="android-minSdkVersion" value="24" />
        <preference name="android-targetSdkVersion" value="33" />

But still getting the same error

       7.  Dependency 'androidx.annotation:annotation-experimental:1.3.0' requires libraries and applications that
           depend on it to compile against version 33 or later of the
           Android APIs.

           :app is currently compiled against android-32.

           Recommended action: Update this project to use a newer compileSdkVersion
           of at least 33, for example 33.

I can set the compileSdkVersion, but I thought that android 12 was the more recent SDK version by default.

This release has increased the minimum supported SDK version to 24 which is Android 7.0. It also has increased the target SDK to 33, Android 13.

To use cordova-android@12, SDK Platform 33 and SDK Build Tools 33.0.2 must be installed. Older build tools version can be uninstalled if older versions of cordova-android is no longer used in your projects.

and they are installed
e-mission/e-mission-docs#934 (comment)

Are any of the plugins overriding this?

@shankari
Copy link
Contributor Author

Aha!

$ grep -r compileSdkVersion plugins/
plugins//cordova-plugin-em-datacollection/src/android/enable_api_31.gradle:            compileSdkVersion 32

and

node_modules//react-native/template/android/build.gradle:        compileSdkVersion = 33
node_modules//react-native/ReactAndroid/hermes-engine/build.gradle:    compileSdkVersion 31
node_modules//react-native/ReactAndroid/build.gradle:    compileSdkVersion 33

node_modules//@shopify/flash-list/android/build.gradle:def _compileSdkVersion = _ext.has('compileSdkVersion') ? _ext.compileSdkVersion : 30
node_modules//@shopify/flash-list/android/build.gradle:    compileSdkVersion _compileSdkVersion
node_modules//react-native-safe-area-context/android/build.gradle:    compileSdkVersion getExtOrDefault('compileSdkVersion', 30)
node_modules//react-native-vector-icons/android/build.gradle:  compileSdkVersion safeExtGet('compileSdkVersion', 28)

node_modules//react-native-screens/android/build.gradle:            if (safeExtGet('compileSdkVersion', rnsDefaultCompileSdkVersion) >= 33) {
node_modules//react-native-gesture-handler/android/build.gradle:    compileSdkVersion safeExtGet("compileSdkVersion", 28)

The final version is

$ cat build-extras.gradle
ext.postBuildExtras = {
    android {
        defaultConfig {
            compileSdkVersion 31
        }
    }
}

We could change the version in data-collection, but now that we have a plethora react native components that specify a compileSdkVersion, let's start by setting it at the top level

@shankari
Copy link
Contributor Author

Setting it at the top level returns the same error

        <preference name="android-minSdkVersion" value="24" />
        <preference name="android-compileSdkVersion" value="33" />
        <preference name="android-targetSdkVersion" value="33" />
$ npm run build-dev-android > /tmp/android_build.log 2>&1
       1.  Dependency 'androidx.appcompat:appcompat-resources:1.6.1' requires libraries and applications that
           depend on it to compile against version 33 or later of the
           Android APIs.

           :app is currently compiled against android-32.

@shankari
Copy link
Contributor Author

Ok. I am currently betting that the compileSdkVersion is coming from the data collection plugin because:

  • it is the only plugin with a compileSdkVersion; the others are in node_modules
  • it is set to 32, while the ones in node_modules are all over the map

Let's check on why we set this in the plugin instead of the top level compileSdkVersion last time...

@shankari
Copy link
Contributor Author

Ah, we added it while upgrading to API 31
e-mission/e-mission-data-collection@300336c
because cordova-android had removed the top level compileSDKVersion setting

I filed an issue for cordova-android

but added this is a workaround

They re-introduced the flag in android 11, with the
following behavior

  • If the target & compile sdk is not configured, it will default to the cordova-android default.
  • If the user sets the target sdk, the compile SDK will match.

which are what we want, depending on whether we want to force the targetSDK or not.

So we should remove the workaround....

@shankari
Copy link
Contributor Author

shankari commented Aug 19, 2023

android_build.log.gz

Removed $ rm src/android/enable_api_31.gradle from the directory and the plugin.xml

$ npx cordova plugin rm cordova-plugin-em-datacollection
$ npx cordova plugin add ../plugins/e-mission-data-collection

That worked! Compile was successful!

$ npm run build-dev-android > /tmp/android_build.log 2>&1
BUILD SUCCESSFUL in 59s
52 actionable tasks: 46 executed, 6 up-to-date
Built the following apk(s):
        /Users/kshankar/e-mission/native_code_upgrade/platforms/android/app/build/outputs/apk/debug/app-debug.apk

@shankari
Copy link
Contributor Author

shankari commented Aug 19, 2023

Let us now try to run in on an android emulator.

I have an android-32 emulator

---------
    Name: Pixel_2_API_32
  Device: pixel_2 (Google)
    Path: /.../.../.android/avd/Pixel_2_API_32.avd
  Target: Google Play (Google Inc.)
          Based on: Android 12L (Sv2) Tag/ABI: google_apis_playstore/x86_64
    Skin: pixel_2
  Sdcard: 512M

I don't have an android-33 avd, so I'll copy this one over and test on that as well.

$ mkdir -p ~/.android/avd/Pixel_2_API_33.avd
$ cp ~/.android/avd/Pixel_2_API_32.avd/config.ini ~/.android/avd/Pixel_2_API_33.avd/
$ cp -r ~/.android/avd/Pixel_2_API_32.ini ~/.android/avd/Pixel_2_API_33.ini

(edit both ini files)

$ ./emulator/emulator Pixel_2_API_33

the home screen seems to look the same as API 32, but I checked that the version is 13

Home screen version is 13
Screenshot 2023-08-19 at 2 41 27 PM Screenshot 2023-08-19 at 2 40 00 PM Screenshot 2023-08-19 at 2 40 47 PM

@shankari
Copy link
Contributor Author

shankari commented Aug 19, 2023

The app launches, but is stuck in the start screen. On connecting via the chrome debugger, looks like deviceready has not fired. Checking adb, might update this comment.

deviceready has not fired after 5 seconds.
​ Channel not fired: onDOMContentLoaded

After a Very Long Time, displayed this error.

Screenshot 2023-08-19 at 2 48 45 PM Screenshot 2023-08-19 at 2 57 46 PM

It is also visually very slow and sluggish
Killing and restarting (and turning off the snapshot to try and save on resources)

@shankari
Copy link
Contributor Author

Ah, this is much snappier (pun intended)

$ ./emulator/emulator -avd Pixel_2_API_33 -no-snapstorage
Launched! Got to permissions correctly but then hung while returning from setting permissions
Screenshot 2023-08-19 at 3 19 11 PM Screenshot 2023-08-19 at 3 20 58 PM Screenshot 2023-08-19 at 3 22 23 PM

@shankari
Copy link
Contributor Author

On trying to re-launch, ran into the application timeout again.
Then tried to re-re-launch, this time permission page was fine.
App launch was fine.

Phew! It really does appear to be almost as smooth as I had hoped.
Let's check this in and then start polishing.

@shankari shankari changed the base branch from aria_and_fall_2023_rewrite to maint_upgrade_2023 August 19, 2023 22:47
@shankari
Copy link
Contributor Author

Created a new branch called maint_upgrade_2023 so that we can run the tests on PRs

and rebased to it

@shankari
Copy link
Contributor Author

Removed all the overrides, build still succeeded.

-        <preference name="android-minSdkVersion" value="23" />
-        <preference name="android-targetSdkVersion" value="32" />

shankari added a commit to shankari/e-mission-data-collection that referenced this pull request Aug 19, 2023
In android 10, the cordova team removed support for `compileSdkVersion`
However, in order to support auto-reset of permissions, we needed to use a
later SDK than the target.

After filing an issue in the cordova-android repo, we added this grade file as
a workaround. However, they have now reintroduced it in android 11
apache/cordova-android#1373 (comment)
*and* our compile and target SDKs have caught up anyway

So we can remove the workaround.

More detail at:
e-mission/e-mission-phone#1016 (comment)
- `cordova-android` to 12
- `cordova-plugin-file` to 8.0.0
- Remove SDK version overrides and go with the cordova-android defaults

Note that this will not work until
e-mission/e-mission-data-collection#207
is merged

e-mission#1016 (comment)
@shankari
Copy link
Contributor Author

upgrading the android SDK fails, probably need to upgrade the java version

Downloading the android SDK. This will take a LONG time and will require you to agree to lots of licenses.
Do you wish to continue? (Y/N)BEGIN: About to start android SDK download
Error: LinkageError occurred while loading main class com.android.sdklib.tool.sdkmanager.SdkManagerCli
        java.lang.UnsupportedClassVersionError: com/android/sdklib/tool/sdkmanager/SdkManagerCli has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0
END: Done with android SDK download, exiting script

@shankari
Copy link
Contributor Author

Yep! It appears to need 16+
https://stackoverflow.com/questions/69674255/how-to-fix-linkageerror-occurred-while-loading-main-class-error-using-an-other

the most recent version is 17.0.8
https://adoptium.net/temurin/releases/?os=mac

@Abby-Wheelis @niccolopaganini how hard was it to get IT to install a new version of java for you? I am tempted to bump this up now, but don't want to block/affect your work

Everything should compile now.
We can check in the sdk changes independently; the main goal is to start
testing these changes before the release.
@shankari shankari marked this pull request as ready for review August 20, 2023 05:18
@shankari
Copy link
Contributor Author

shankari commented Aug 20, 2023

@niccolopaganini will wait for your review until Monday morning, otherwise I will merge since all the tests as passing
I will address android command line tools upgrade in separate PR since it is not critical for the API upgrade.

Copy link
Contributor

@niccolopaganini niccolopaganini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello ma'am. These changes look good to me.

@niccolopaganini
Copy link
Contributor

how hard was it to get IT to install a new version of java for you? I am tempted to bump this up now, but don't want to block/affect your work

One of the problems that I faced during the initial build was that the JDK version was installed differently. I was not able to build. I can try it with the updates on a separate computer and see how the changes affect.

@Abby-Wheelis
Copy link
Member

how hard was it to get IT to install a new version of java for you? I am tempted to bump this up now, but don't want to block/affect your work

It took a while, but the actual install was very simple. It took 11ish days from filing the request until I got it installed, but if I had it to do over again I'd follow up sooner, I asked for an update after the request hadn't moved for about a week and ended up getting to install later that day. Since I just went through the process, I'd be happy to help anyone else that needs get the request filed. Once installed I was able to set the path and run the android emulator easily.

@shankari
Copy link
Contributor Author

One of the problems that I faced during the initial build was that the JDK version was installed differently. I was not able to build. I can try it with the updates on a separate computer and see how the changes affect.

It took 11ish days from filing the request until I got it installed,

The command line tools update is not critical. Given this, I will merge this now.
@niccolopaganini can then work with IT to update his java version and test out the command line tools update.

@shankari shankari merged commit a5f8fab into e-mission:maint_upgrade_2023 Aug 21, 2023
4 checks passed
shankari added a commit to shankari/e-mission-phone that referenced this pull request Sep 9, 2023
This addresses updating the SDK tools to the latest versions:
e-mission/e-mission-docs#934 (comment)

And removing the obsolete HAXM package:
e-mission/e-mission-docs#958 (comment)

It also updates the README to indicate the required java version after the upgrade
e-mission#1016 (comment)
e-mission#1016 (comment)

Testing done:

After upgrading to the most recent version of OpenJDK (17)

```
$ java --version
openjdk 17.0.8.1 2023-08-24
OpenJDK Runtime Environment Temurin-17.0.8.1+1 (build 17.0.8.1+1)
OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (build 17.0.8.1+1, mixed mode, sharing)
```

Android SDK install succeeds

```
$ bash setup/prereq_android_sdk_install.sh

---------------------------------------
Accept? (y/N): Y
[=======================================] 100% Unzipping... android-12/zipalign

END: Done with android SDK download, exiting script
```

And an android build in an existing checked-out repo succeeds

```
$ npm run build-dev-android
BUILD SUCCESSFUL in 33s
52 actionable tasks: 12 executed, 40 up-to-date
Built the following apk(s):
	.../platforms/android/app/build/outputs/apk/debug/app-debug.apk

```

Albeit with several deprecated APIs

```
w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (169, 17): 'get(String!): Any?' is deprecated. Deprecated in Java
w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (316, 20): 'get(String!): Any?' is deprecated. Deprecated in Java
w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (627, 33): 'constructor Builder(Context)' is deprecated. Deprecated in Java
w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/FCMService.kt: (1190, 37): 'fromHtml(String!): Spanned!' is deprecated. Deprecated in Java
w: /Users/kshankar/in-house/openpath-phone/platforms/android/app/src/main/java/com/adobe/phonegap/push/PushPlugin.kt: (97, 25): 'get(String!): Any?' is deprecated. Deprecated in Java
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants