Changelog and README for sequential_output/1

CHANGELOG.md

@@ -1,3 +1,10 @@
+## 1.1.0 (unreleased)
+
+Memory blowing up on formatting? Try `sequential_output/2`!
+
+### Features
+* Introduce `sequential_output/2` which you can can call as a function in formatters. Instead of formatting _everything_ first and then writing it out it will format one HTML file and immediately write it out freeing it up for Garabage Collection. This can lead to huge max memory used savings (12 GB --> 7 GB in a bigger benchmark I ran).
+
 ## 1.0.0 (2019-03-28)
 
 Compatibility with benchee 0.99.0 and 1.0.0 as well as benchee_json 1.0.0.

README.md

@@ -68,6 +68,28 @@ When you hover the graphs in the HTML report, quite some plotly.js controls and
 
 Be aware, that currently when too many samples are recorded (> 100_000 usually) rendering might break as plotly can't handle all that data. See [this issue](https://github.com/PragTob/benchee_html/issues/3) on how to quick fix it and what could be done in the future.
 
+### Too much memory consumption?
+
+Due to the way that formatters are designed to first `format/2` everything, which can be done in parallel across formatters, and then `output/2` it the formatter can be quite memory hungry. This is due to the fact, that it means all files need to be held in memory before writing them out. Most times, this should not be an issue - however if you run a benchmark with a lot of scenarios and samples it _can_ be. Hence, there is `sequential_output/2` which produces the same output but formats a file and immediately writes it out.
+
+You can use it as a function:
+
+```elixir
+list = Enum.to_list(1..10_000)
+map_fun = fn i -> [i, i * i] end
+
+Benchee.run(
+  %{
+    "flat_map" => fn -> Enum.flat_map(list, map_fun) end,
+    "map.flatten" => fn -> list |> Enum.map(map_fun) |> List.flatten() end
+  },
+  formatters: [
+    # this is the important bit
+    fn suite -> Benchee.Formatters.HTML.sequential_output(suite, auto_open: false) end
+  ]
+)
+```
+
 ## PNG image export/download
 
 When you hover the graph the controls appear and the left most of those is a camera and says "Download plot as png" - and it does what you'd expect. Refer to the image below if you need more guidance :)

samples/fast_sequential.exs

@@ -1,5 +1,3 @@
-# It is possible to use multiple formatters so that you have both the Console
-# output and an HTML file.
 list = Enum.to_list(1..10_000)
 map_fun = fn i -> [i, i * i] end
 
@@ -8,7 +6,6 @@ Benchee.run(
     "flat_map" => fn -> Enum.flat_map(list, map_fun) end,
     "map.flatten" => fn -> list |> Enum.map(map_fun) |> List.flatten() end
   },
-  # kwargs to map conversions
   formatters: [
     fn suite -> Benchee.Formatters.HTML.sequential_output(suite, auto_open: false) end
   ],