Send binary data via ChildProcess.send

[EugenDueck]

Is there a way to send binary data efficiently to a child process using (something as easy to use as) ChildProcess.send? send(buf) does not work as expected:

var cluster = require('cluster');

if (cluster.isWorker)
{
  process.on('message', function messageHandler(msg)
  {
    console.log("msg: " + JSON.stringify(msg) +
        " - type: " + typeof msg +
        " - buf? " + Buffer.isBuffer(msg));
  });
}
else
{
  var buffer = new Buffer(2);
  buffer[0] = 5;
  buffer[1] = 88; 
  var worker = cluster.fork();
  worker.on("online", function() { worker.send(buffer); });
}

The above will log

msg: [5,88] - type: object - buf? false

msg turns out to be a JSONified version of the original buffer.

[rlidwka]

Same thing with dates... I'm not sure it's a bug because send() indeed uses JSON internally, and JSON has no easy way to represent buffers.

[bnoordhuis]

As pointed out by @rlidwka, the control channel speaks JSON. You can only send over things that have a valid JSON representation, something Buffer and Date objects don't really have.

Is there a way to send binary data efficiently to a child process

You can open an additional pipe when spawning the child process:

var args = [ /* ... */ ];
var options = { stdio: ['pipe','pipe','pipe','pipe'] };  // first three are stdin/out/err
var proc = child_process.spawn(cmd, args, options);
var pipe = proc.stdio[3];
pipe.write(Buffer('hello'));

The child can open the pipe like this:

var pipe = new net.Socket({ fd: 3 });
pipe.on('data', function(buf) {
  // do whatever
});

[EugenDueck]

Thanks bnoordhuis, please consider my pull request that exports some of the configurability of child_process.spawn() to cluster.fork(), via child_process.fork(). It can be used similar to your example, only using cluster instead of child_process. The parent part would look something like this, the child/worker part isn't changed:

var cluster = require('cluster');
var worker = cluster.fork(undefined, { addStdio: ['pipe'] });
var pipe = worker.process.stdio[4];
pipe.write(Buffer('hello'));

I wasn't sure how to handle the env parameter in cluster.fork(), my patch essentially makes it "mandatory", hence the need to pass in "undefined" in the example above. As both are of type object, I couldn't just do an if (Array.isArray(argument[0]) style of argument overload detection.

[InfinitiesLoop]

I don't suppose there's been any improvement to this. I would have loved to see the PR for fork options make it, because I cannot seem to figure out a good way of opening this additional IPC channel without having to basically re-implement everything the cluster module is doing for me. Is there a way to set up the IPC channel after the worker has been forked?

[foo123]

So is there a way to pass binary data (e.g typed arrays) to forked (not spawned) child process through process.send?

The use case i have, is an image processing lib (for node, browser and web workers) and i have everything working perfect in all platforms except parallel processing in node (through forked child processes), since typed arrays (i.e image data) are serialised as objects, not even arrays.