Are Ports Fast Enough? – Benchmarking Language Interop with Elixir

Some background: I’m exploring the possibility of using Elixir to drive a game engine. Elixir is a great candidate for a scripting language because of its user-friendly nature, but it’s not fast enough to power a rendering engine by itself. For that, you need a native language like C, Rust, or Go, with some means of communication between the Elixir gameplay logic and the native rendering code.

Ports are one of the main ways that Elixir can communicate with programs written in other languages. When you use a port, the VM spawns another OS process and communicates with it over stdin and stdout. Since this is the simplest way to do IPC between Elixir and other languages, I did some very informal benchmarks to find out if it’s fast enough for my game engine.

TL;DR: ports are definitely fast enough. You can check out the source code for the benchmarks here. #

Methodology #

For each of the languages under test, I wrote a small program that receives \n-delimited messages over stdin and echoes them back over stdout. For example, if you passed the message “foo”, the program would write “responding to foo” to stdout. Since there is no actual processing going on, this will only test raw message throughput with no influence from other factors like parallelism or number-crunching speed.

These programs were then run through an Elixir benchmark which spams as many messages as possible for some duration and measures the amount of messages that were acknowledged. Separate benchmarks were run for large and small messages.

Please note that this is not a formal, complete methodology– if you show this to research scientists at Oxford, they will not be impressed. However, it’s good enough to show whether or not ports are fast enough to be worth exploring for high-performance programs like game engines.

Languages #

In order to get a decent spread of results, I wanted to benchmark communication with a native-compiled language, a VM language, and an interpreted language. I arbitrarily chose Go, Java, and Node.js to fit these roles. benchfella was used as the benchmarking framework.

Messages #

Messages are generated by the following code:

defp gen_complex_term(id), do: %{ id: id,
                    type: :complex_term,
                    name: "Term no. #{id}",
                    children: [%{ foo: "bar" }, %{ bim: ['b', 'a', 'z'] }] }

defp gen_many_complex_terms(num) do
  1..num
  |> Enum.map(&("#{inspect gen_complex_term(&1)}"))
  |> Enum.join
end

Small messages are generated by calling gen_complex_term/1, large messages are generated by calling gen_many_complex_terms/1. All messages are serialized with IO.inspect/1. A small message looks like this:

%{ id: 123,
   type: :complex_term,
   name: "Term no. 123",
   children: [%{ foo: "bar" }, %{ bim: ['b', 'a', 'z'] }] }

The size of a small message comes out to about 102 bytes, while large messages are around 8010 bytes.

Results #

## LargeMsgBench
bench iterations   average time 
go        100000   25.27 µs/op
node       50000   36.81 µs/op
java       50000   38.24 µs/op

## SmallMsgBench
bench iterations   average time 
go        100000   12.48 µs/op
java      100000   17.35 µs/op
node      100000   28.24 µs/op

There were a few surprises here:

1. I expected ports to be much slower in general, but it looks like they should be Fast Enough™ as long as you’re careful about the size of your messages.
2. I expected small messages to be faster than large messages, but large messages are orders of magnitude more efficient. Since most meaningful messages won’t be very large, this tells us that batching communications could be an important performance optimization.
3. The messages/sec throughput for both Go and Java seems to decrease much faster than for Node. Node even beats Java outright in the large message benchmark, which is both interesting and strange.

Conclusion #

So what does this mean for the game engine I described above? Because message passing over a port is so fast, I can write the rendering engine in whatever language I want without having to worry about ports being a bottleneck. It can run in the background and follow commands passed to it by an Elixir program.

This same architecture can be applied to other systems, too. If you want to write a webapp in Phoenix, but need to use some heavy number-crunching code written in C or Java, ports could provide an easy method of calling those functions from Elixir. If you’re interested in interprocess communication from Elixir/Erlang, you should also check out the more complex methods, such as C nodes, which are C programs that act like an Erlang node; NIFs, which are native functions linked into the Erlang runtime; and port drivers, which are like a mix between ports and NIFs.

Overall, I’m very pleased with the results of this benchmark. It’s a great feeling to know that you aren’t shooting yourself in the foot by starting with a particular technology. This means that my (probably terrible) idea for an Elixir-based game engine can proceed without bottlenecks. If you’re interested in following my progress, make sure to join me next time as I explore what a stdin-powered game engine could look like!