[Impeller] Issue with Flutter 3.19 optimization for BackdropFilter with animation

[mikeesouth]

Steps to reproduce

I can reproduce this by adding a Rive asset (animated vector graphics) and then adding a BackdropFilter on top of that asset. The Rive asset does not have to be animated and the issue occurs only when the Rive asset contains two clipping masks. I'm not sure how the Rive runtime is built but everything worked fine in Flutter 3.16 but it no longer works with Flutter 3.19 on iOS - and I'm assuming it's related to the Impeller optimizations of BackdropFilter that was introduced in Flutter 3.19.

This issue does not affect Android or web because these platforms does not use Impeller. I have not tried to enable Impeller on Android.

The issue is clearly described in an issue that I filed on the the rive-flutter repo. However, one of the contributors to rive-flutter thought that this is primarily a Flutter problem, not a Rive problem.

This bug can be tested in this repo: https://github.com/mikeesouth/rive_mask_backdrop_bug

Expected results

I expect that the BackdropFilter optimizations in Flutter 3.19 still works on all scenarios that worked in Flutter 3.16.

Actual results

Impeller BackdropFilter in Flutter 3.19 does not affect Rive assets with two clipping masks - making these widgets look like they are rendered "on top" of the BackdropFilter. See rive-app/rive-flutter#360 for more details and screenshots.

Code sample

This bug can be tested in this repo: https://github.com/mikeesouth/rive_mask_backdrop_bug

Code sample

import 'dart:ui';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:rive/rive.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter 3.19',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Impeller issue with backdrop filter'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  static const width = 300.0;
  static const height = 200.0;

  static Artboard? _riveArtboard1;
  static Artboard? _riveArtboard2;

  @override
  void initState() {
    super.initState();

    _loadArtboard();
  }

  void _loadArtboard() async {
    final riveFileData1 = await rootBundle.load('assets/test1.riv');
    final riveFile1 = RiveFile.import(riveFileData1);
    final riveFileData2 = await rootBundle.load('assets/test2.riv');
    final riveFile2 = RiveFile.import(riveFileData2);

    setState(() {
      _riveArtboard1 = riveFile1.artboardByName('test');
      _riveArtboard2 = riveFile2.artboardByName('test');
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        title: Text(widget.title),
      ),
      body: Stack(
        alignment: Alignment.center,
        children: [
          Positioned(
            top: 0,
            child: Row(
              children: [
                const SizedBox(
                  width: width,
                  height: height,
                  child: RiveAnimation.network(
                    'https://cdn.rive.app/animations/vehicles.riv',
                  ),
                ),
                SizedBox(
                  width: width,
                  height: height,
                  child: _riveArtboard1 != null
                      ? Rive(artboard: _riveArtboard1!)
                      : Container(color: Colors.orange),
                ),
                SizedBox(
                  width: width,
                  height: height,
                  child: _riveArtboard2 != null
                      ? Rive(artboard: _riveArtboard2!)
                      : Container(color: Colors.orange),
                ),
              ],
            ),
          ),
          BackdropFilter(
            filter: ImageFilter.blur(sigmaX: 3, sigmaY: 3),
            child: Container(
              width: double.infinity,
              height: double.infinity,
              color: Colors.black.withOpacity(0.5),
              alignment: Alignment.center,
              child: Container(
                margin: const EdgeInsets.only(top: 100),
                width: 800,
                height: 400,
                color: Colors.white,
                padding: const EdgeInsets.all(15),
                child: const Text(
                  '''The three rive assets above should be blurred and dimmed by the backdrop filter.

- The left one (the bus) works correctly.

- The middle one (test1.riv) contains one clipping shape and it also works correct.

- The right one (test2.riv) contains two clipping shapes and it is not blurred or dimmed in Impeller on iOS, it works on Android, web and non-impeller on iOS. It also worked before the backdrop filter optimizations in Flutter 3.19 (so it works in 3.16.X)''',
                  style: TextStyle(fontSize: 20),
                ),
              ),
            ),
          ),
        ],
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration

Logs

Logs

N/A

Flutter Doctor output

Doctor output

[√] Flutter (Channel stable, 3.19.1, on Microsoft Windows [Version 10.0.22631.3155], locale sv-SE)
    • Flutter version 3.19.1 on channel stable at C:\Stuff\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision abb292a07e (6 days ago), 2024-02-20 14:35:05 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at C:\Users\micke\AppData\Local\Android\sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: C:\Program Files\Android\Android Studio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-10027231)
    • All Android licenses accepted.

[X] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[√] Visual Studio - develop Windows apps (Visual Studio Professional 2022 17.6.3)
    • Visual Studio at C:\Program Files\Microsoft Visual Studio\2022\Professional
    • Visual Studio Professional 2022 version 17.6.33801.468
    • Windows 10 SDK version 10.0.22000.0

[√] Android Studio (version 2022.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-10027231)

[√] VS Code (version 1.86.2)
    • VS Code at C:\Users\micke\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.82.0

[√] Connected device (2 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [Version 10.0.22631.3155]
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 121.0.2277.112

[√] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[huycozy]

Thanks for the report. I also can reproduce this issue on iOS iPhone 7, iOS 15.8 and Android Pixel 7, Android 14 (with enabled Impeller). Without Impeller, the issue doesn't occur.

Adding screenshot on smaller screen device to see the issue clearer.

flutter doctor -v (stable and master)

[✓] Flutter (Channel stable, 3.19.1, on macOS 14.1 23B74 darwin-x64, locale en-VN)
    • Flutter version 3.19.1 on channel stable at /Users/huynq/Documents/GitHub/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision abb292a07e (29 hours ago), 2024-02-20 14:35:05 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15A507
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Applications/Android Studio Hedgehog.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • android-studio-dir = /Applications/Android Studio.app/
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.86.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.82.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 14.1 23B74 darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 121.0.6167.184

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[!] Flutter (Channel master, 3.20.0-13.0.pre.46, on macOS 14.1 23B74 darwin-x64, locale en-VN)
    • Flutter version 3.20.0-13.0.pre.46 on channel master at /Users/huynq/Documents/GitHub/flutter_master
    ! Warning: `flutter` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/flutter, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
    ! Warning: `dart` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/dart, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 523b0c4d84 (3 hours ago), 2024-02-26 15:55:52 -0800
    • Engine revision 04ff2868ce
    • Dart version 3.4.0 (build 3.4.0-176.0.dev)
    • DevTools version 2.33.0-dev.11
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15A507
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Applications/Android Studio Hedgehog.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • android-studio-dir = /Applications/Android Studio.app/
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.86.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.82.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 14.1 23B74 darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 122.0.6261.69

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[jonahwilliams]

Assuming we can reproduce this, we will need to bisect to find the cause. AFAIK the only optimizations we did for backdropfilters in 3.19 were related to clip replay/restore which sounds related to clipping masks.

[huycozy]

This issue doesn't occur on the previous stable release 3.16.9. I bisected this to 1333e16 which is a Roll Flutter Engine commit.

git bisect (tailed output)

1333e16a4a1237fcc026dac844a29e4566368fa8 is the first bad commit
commit 1333e16a4a1237fcc026dac844a29e4566368fa8
Author: engine-flutter-autoroll <engine-flutter-autoroll@skia.org>
Date:   Tue Oct 31 23:26:23 2023 -0400

    Roll Flutter Engine from db06c2e10459 to a0ac6b432cdf (8 revisions) (#137656)
    
    https://github.com/flutter/engine/compare/db06c2e10459...a0ac6b432cdf
    
    2023-11-01 bdero@google.com [Impeller] Include cstdint everywhere that uint32_t is used. (flutter/engine#47533)
    2023-11-01 bdero@google.com [Impeller] Fix nullopt access and simplify coverage computation in GetSubpassCoverage. (flutter/engine#47347)
    2023-11-01 bdero@google.com [Impeller] OpenGLES: Ensure frag/vert textures are bound with unique texture units. (flutter/engine#47218)
    2023-11-01 skia-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from LCfhx_lTRJI51G0zc... to _TyF0etsONe5aqCbM... (flutter/engine#47532)
    2023-11-01 jonahwilliams@google.com [Impeller] stencil buffer record/replay instead of MSAA storage. (flutter/engine#47397)
    2023-11-01 chris@bracken.jp [macOS] Delete FlutterCompositor tests (flutter/engine#47527)
    2023-10-31 bdero@google.com [Impeller] Place Rect statics under the Rect template. (flutter/engine#47529)
    2023-10-31 skia-flutter-autoroll@skia.org Roll Skia from aaa225e0cc6d to 34ef20100acc (1 revision) (flutter/engine#47530)
    

[jonahwilliams]

Thanks @huycozy , that makes the likely culprit:

2023-11-01 jonahwilliams@google.com [Impeller] stencil buffer record/replay instead of MSAA storage. (flutter/engine#47397)

[dnfield]

Good news! This appears to be fixed on master. Going to try to bisect.

[jason-simmons]

This issue does not happen when stencil-then-cover is enabled.

[dnfield]

flutter/engine#47397 did indeed break it.

[bdero]

Revert w/ conflicts resolved for 3.19: https://github.com/flutter/engine/compare/flutter-3.19-candidate.1...bdero:flutter-engine:bdero/cp-3.19-revert47397?expand=1

[dnfield]

Here's a very small repro using dart:ui.

I'mgoing to see if I can write an aiks unittest for this.

import 'dart:ui';

typedef CanvasCallback = void Function(Canvas canvas);

Picture makePicture(CanvasCallback callback) {
  final recorder = PictureRecorder();
  final canvas = Canvas(recorder);
  callback(canvas);
  return recorder.endRecording();
}

void main() {
  final sceneBuilder = SceneBuilder();
  sceneBuilder.addPicture(Offset.zero, makePicture((Canvas canvas) {
    canvas.clipRect(const Rect.fromLTRB(10, 10, 200, 200));
    // canvas.clipRect(const Rect.fromLTRB(15, 10, 100, 200));
    canvas.drawPaint(Paint()..color = const Color(0xFFFF0000));
  }));
  sceneBuilder.pushBackdropFilter(ImageFilter.blur(sigmaX: 3, sigmaY: 3));
  sceneBuilder.addPicture(Offset.zero, makePicture((Canvas canvas) {
    canvas.drawPaint(Paint()..color = const Color(0x88000000));
  }));
  final scene = sceneBuilder.build();

  PlatformDispatcher.instance.onBeginFrame = (_) {
    PlatformDispatcher.instance.implicitView!.render(scene);
  };
  PlatformDispatcher.instance.scheduleFrame();
}

[dnfield]

If you uncomment the second canvas.clipRect it reproduces.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.