Fun OCaml workshop — Concurrency and Parallelism

We're going to build a distributed ray tracer!

We have written a ray tracer as a library.

We have also written a server that manages the rendering of images.

Your job is to write an "actor" or client that

1. requests rendering jobs from the server,
2. renders the job,
3. respond to server with the rendered images, and
4. repeats

and does that as quickly as possible.

Who completes the most rendering jobs?

0. Background

Concurrent and Parallel Programming with OCaml 5

Also, we are here to help. Feel free to ask any questions!

1. Install

Clone the repository and install dependencies:

$ opam pin add . --with-version=dev

2. Test

You can run the server locally:

$ dune exec funocaml_server

Visit http://localhost:8080/ with your browser.

You can then run some actors with the local server:

$ dune exec -- ./test/test_eio.exe $USER
$ dune exec -- ./test/test_picos_io_cohttp.exe $USER

You can also connect to our server by specifying the server ip:

$ dune exec -- ./test/test_eio.exe $USER $SERVER
$ dune exec -- ./test/test_picos_io_cohttp.exe $USER $SERVER

3. Multiple requests

Modify the actor to take multiple (max 3) requests concurrently and render them.

Tip

You could just fork a few fibers. With Eio you might use a switch or e.g. all. With Picos you could use the sample structured concurrency library, which has e.g. the Flock mechanism and Run.all.

Can you also render the requests in parallel?

Tip

With Eio you'd use a domain manager. Some of the Picos sample schedulers directly support multi-threading, which means that multiple domains are automatically used to run fibers.

4. Subdivide and render in parallel

To utilize all the cores, split the request and render subjobs in parallel.

The "actor" library has operations to split jobs and join images.

Tip

You might want to use a concurrent data structure to manage jobs that you'll then execute in parallel. Saturn and Kcas provide concurrent data structures. Kcas also provides blocking data structures that work with your scheduler either through domain-local-await (supported by Eio) or through Picos.

If you use a non-blocking data structure, you might need to yield.

Which is better for parallelism? FIFO or LIFO?

5. More speedups?

Does your program use all the cores effectively?

If not, figure out the bottlenecks and modify your program to avoid them.

6. Distributed rendering

Let's try to run our actors against the server to render some more complex scenes.

7. More challenge?

Ideas:

• Try other schedulers like Miou, Moonpool, or Riot.

• Use Domainslib or raw domains to parallelize rendering.

• Optimize the ray tracer.

• Go wild, implement a fractal generator for scenes!