MemoryFileSystem is 10x slower to write than LocalFileSystem

[mymikemiller]

With the code below, MemoryFileSystem is taking 10 times longer to write to a file than it takes when switching out with LocalFileSystem.

import 'dart:io';

import 'package:file/file.dart' as f;
import 'package:path/path.dart' as p;
import 'package:file/memory.dart';
import 'package:file/local.dart';

final memoryFileSystem = MemoryFileSystem();
final localFileSystem = LocalFileSystem();

f.Directory createTempDirectory(f.FileSystem filesystem) {
  final root = filesystem.systemTempDirectory.childDirectory('test');
  root.createSync();
  final newTempFolder = root.createTempSync();
  return newTempFolder;
}

Future<void> main() async {
  final fs = memoryFileSystem;
  // final fs = localFileSystem;

  final tempDirectory = createTempDirectory(fs);
  final file = fs.file(p.join(tempDirectory.path, 'test.txt'));
  var output = file.openWrite(mode: FileMode.writeOnlyAppend);

  var i = 0;

  final stopwatch = Stopwatch()..start();
  while (i++ < 500000) {
    output.add([i]);
  }
  print('closing after ${stopwatch.elapsed}');
  await output.close();

  print('done in ${stopwatch.elapsed}');
}

My results are as follows:

localFileSystem:
closing after 0:00:00.093014
done in 0:00:22.261997

memoryFileSystem:
closing after 0:00:00.255479
done in 0:03:48.186438

I would think memoryFileSystem should actually be faster since it's not writing to disk. Either way, a 10x slowdown seems excessive and means I'll need to find another solution for my needs. Or maybe you can spot something I'm doing wrong here so I can use MemoryFileSystem in my production app where write speed is important.

[tsavo-at-pieces]

@mymikemiller did you end up finding a different solution? We are also interested in this over at code.pieces.app!

Thanks!

[mymikemiller]

I haven't needed to optimize for speed yet, but my plan is to quit using MemoryFileSystem and just use a temporary directory to store the files while I work with them.

[renatoathaydes]

This repository seems abandoned, unfortunately. It would be so useful to have a working library for this. I was using it for tests, but I keep running on bugs. I even tried to graps this lib so I could make PRs but the code is harder to follow than I have time to spend on it.

[dnfield]

MemoryFileSystem isn't optimized for performance. Its intended use is in testing frameworks.

There's probably room to optimize it, but it's not something that any of the maintainers of this project are likely very interested in. If you have a patch that improves things please add it.

[renatoathaydes]

This library has so many bugs it's impossible to use for testing. Just look at the bug list, even stuff like listing files in a directory doesn't work properly.

I replaced this library with my own that has a withCurrentDirectory function that lets me make tests run on a system temp dir. If anyone is interested here's my commit: renatoathaydes/dartle@93733ae

[dnfield]

Priority is generally given to bugs that affect the tests run by flutter_tools, which is the original reason this library was written.