Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix flaky iOS integration test #1524

Merged
merged 9 commits into from
Jul 17, 2023
Merged

Fix flaky iOS integration test #1524

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e4d0c17
use xcrun directly instead of subosito/flutter-action
denrase Jun 19, 2023
7ba3cd1
close sentry in tearDown, don’t call pumpandsettle
denrase Jun 19, 2023
a3e1895
bump macos version and simulator
denrase Jun 26, 2023
a4e3c5b
downgrade runtime
denrase Jun 26, 2023
08505e6
Merge branch 'main' into fix/flaky-ios-tests
denrase Jun 26, 2023
c0ae3c5
enable android test
denrase Jun 27, 2023
e40f399
Merge branch 'main' into fix/flaky-ios-tests
denrase Jul 10, 2023
7f69c79
Merge branch 'main' into fix/flaky-ios-tests
marandaneto Jul 14, 2023
c6f1a14
Merge branch 'main' into fix/flaky-ios-tests
marandaneto Jul 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 46 additions & 48 deletions .github/workflows/flutter_integration_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,22 +20,56 @@ jobs:
test-android:
runs-on: macos-latest
timeout-minutes: 30
defaults:
run:
working-directory: ./flutter/example
strategy:
fail-fast: false
matrix:
sdk: ['stable', 'beta']
steps:
- name: checkout
uses: actions/checkout@v3

- uses: actions/setup-java@v3
with:
distribution: 'adopt'
java-version: '11'

- uses: subosito/flutter-action@48cafc24713cca54bbe03cdc3a423187d413aafa # pin@v2.10.0
with:
channel: ${{ matrix.sdk }}

- name: flutter upgrade
run: flutter upgrade

- name: flutter pub get
run: flutter pub get

- name: launch android emulator & run android integration test
uses: reactivecircus/android-emulator-runner@50986b1464923454c95e261820bc626f38490ec0 #pin@v2.27.0
with:
working-directory: ./flutter/example
api-level: 21
arch: x86_64
profile: Nexus 6
script: flutter test integration_test/integration_test.dart --verbose
Comment on lines +49 to +56
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can apply those changes after merged here #1543
This helps with speeding up and flakiness.


test-ios:
runs-on: macos-13
timeout-minutes: 30
defaults:
run:
working-directory: ./flutter/example
strategy:
fail-fast: false
matrix:
sdk: ['stable', 'beta']
# 'beta' is flaky because of https://github.com/flutter/flutter/issues/124340
sdk: ['stable']
steps:
- name: checkout
uses: actions/checkout@v3

- uses: actions/setup-java@v3
with:
distribution: 'adopt'
java-version: '11'

- uses: subosito/flutter-action@48cafc24713cca54bbe03cdc3a423187d413aafa # pin@v2.10.0
with:
channel: ${{ matrix.sdk }}
Expand All @@ -46,46 +80,10 @@ jobs:
- name: flutter pub get
run: flutter pub get

- name: launch android emulator & run android integration test
uses: reactivecircus/android-emulator-runner@50986b1464923454c95e261820bc626f38490ec0 #pin@v2.27.0
with:
working-directory: ./flutter/example
api-level: 21
arch: x86_64
profile: Nexus 6
script: flutter test integration_test/integration_test.dart --verbose

# Enable after fixing https://github.com/getsentry/sentry-dart/issues/1448
# test-ios:
# runs-on: macos-latest
# timeout-minutes: 30
# defaults:
# run:
# working-directory: ./flutter/example
# strategy:
# fail-fast: false
# matrix:
# # 'beta' is flaky because of https://github.com/flutter/flutter/issues/124340
# sdk: ['stable']
# steps:
# - name: checkout
# uses: actions/checkout@v3

# - uses: subosito/flutter-action@48cafc24713cca54bbe03cdc3a423187d413aafa # pin@v2.10.0
# with:
# channel: ${{ matrix.sdk }}

# - name: flutter upgrade
# run: flutter upgrade

# - name: flutter pub get
# run: flutter pub get

# - name: launch ios emulator
# uses: futureware-tech/simulator-action@ee05c113b79f056b47f354d7b313555f5491e158 #pin@v2
# with:
# model: 'iPhone 14'
# os_version: '16.2'
- name: launch ios simulator
run: |
simulator_id=$(xcrun simctl create sentryPhone com.apple.CoreSimulator.SimDeviceType.iPhone-14 com.apple.CoreSimulator.SimRuntime.iOS-16-2)
xcrun simctl boot ${simulator_id}

# - name: run ios integration test
# run: flutter test integration_test/integration_test.dart --verbose
- name: run ios integration test
run: flutter test integration_test/integration_test.dart --verbose
3 changes: 1 addition & 2 deletions flutter/example/integration_test/integration_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import 'package:sentry_flutter_example/main.dart';
void main() {
TestWidgetsFlutterBinding.ensureInitialized();

setUp(() async {
tearDown(() async {
await Sentry.close();
});

Expand All @@ -18,7 +18,6 @@ void main() {
bundle: SentryAssetBundle(enableStructuredDataTracing: true),
child: const MyApp(),
)));
await tester.pumpAndSettle();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that needed anymore?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not make a difference in the tests as far as i could tell.

}, 'https://abc@def.ingest.sentry.io/1234567');
}

Expand Down
Loading