request: run goldens with multiple configurations

[alestiago]

Is there an existing feature request for this?

• I have searched the existing issues.

Command

I would love if we could specify a configuration (or more) per test

Description

As a developer, I would like to be able to only have to write a single goldenTest and be able to specify multiple configuration where the test would run.

For example:

goldenTest('foo', configuration: config1, ...);

I would expect that if the configuration parameter is not required the global configuration is used (if any) instead. Hence, once the test finishes the global configuration is restored. I would also expect, to be able to give the Configuration some kind of parameter in order if the null properties should be inherited.

The above is just an example. I'm not sure if it is to be considered the best API. Alternatives, could be, for example, to have a configurations be a List and avoid extracting the test method in order to reuse it. Or another possibility is to have a GoldenTestConfiguration widget:

class GoldenTestConfiguration extends StatelessWidget {
  const GoldenTestConfiguration({
    super.key,
    required this.config,
    required this.child,
    this.inheritConfig = true,
  });

  final AlchemistConfig config;

  final bool inheritConfig;

  final Widget child;

  @override
  Widget build(BuildContext context) {
    return AlchemistConfig.runWithConfig<Widget>(
      config: inheritConfig ? AlchemistConfig.current().merge(config) : config,
      run: () => child,
    );
  }
}

// Or with a widget builder
class GoldenTestConfiguration extends StatelessWidget {
  const GoldenTestConfiguration({
    super.key,
    required this.config,
    required this.builder,
    this.inheritConfig = true,
  });

  final AlchemistConfig config;

  final bool inheritConfig;

  final WidgetBuilder builder;

  @override
  Widget build(BuildContext context) {
    return AlchemistConfig.runWithConfig<Widget>(
      config: inheritConfig ? AlchemistConfig.current().merge(config) : config,
      run: () => builder(context),
    );
  }
}

--

I'm unsure if having logic within setUp and tearDown that alters the config (similar to the snippet below) would work; but if so, I think it feels very verbose in comparison with just providing the configuration to goldenTest.

  AlchemistConfig.runWithConfig(
    config: AlchemistConfig(
      forceUpdateGoldenFiles: true,
    ),
    run: () {
      // test here.
    },
  );

Reasoning

In some scenarios, a developer would like to specify more than a single configuration. This could be motivated, for example, when there are different functionalities to be run when a given flavour or target is given. Hence, as a developer you would be interested on running the same test with different configurations.

Example 1
You have different themes for different platforms and you wish to have a config with each theme. Currently the flutter_test_config.dart file looks like the above snippet. This is similar to #21 .

// flutter_test_config.dart
Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  final enablePlatformTests =
      !Platform.environment.containsKey('GITHUB_ACTIONS');

  return AlchemistConfig.runWithConfig(
    config: AlchemistConfig(
      theme: AppTheme(platform: TargetPlatform.android).themeData,
      platformGoldensConfig:
          AlchemistConfig.current().platformGoldensConfig.copyWith(
                enabled: enablePlatformTests,
              ),
    ),
    run: testMain,
  );
}

Additional context and comments

N/A

[adamhalesworth]

This is similar to what I was trying to achieve. I'm testing button widgets that have an inverted state, which ideally means the theme is different for that specific GoldenTestGroup.

Looking at the code for goldenTest, it reaches out for the config directly using current:

final config = AlchemistConfig.current();

final currentPlatform = HostPlatform.current();
final variant = AlchemistTestVariant(
  config: config,
  currentPlatform: currentPlatform,
);

Unless I'm missing something from the wider context, this looks like it would be simple to implement for goldenTest for a single config.

Which property/ies on AlchemistConfig would you like to change per test? Could you achieve it using nested GoldenTestGroup and have a function generate a reusable "widget under test"?