Elixir and IO Lists, Part 1: Building Output Efficiently

Also, the fact that we can nest these IO lists makes a huge difference in how quickly we can build them. Normally, appending to a linked list is an O(N) operation: we have to walk through every item in the list, find the last one, and point its tail to a new item. With immutable data, it's even worse: we can't modify the last item, so we have to copy it, which means we have to copy the previous item, and the previous one, all the way back to the start of the list.

However, with nesting, we can append to a list simply by wrapping it in a new list.

names = ["A", "B"]    # => ["A", "B"]
names = [names, "C"]  # => [["A", "B"], "C"]

This operation is O(1) and doesn't require copying anything.

IO lists also have big implications for system calls.

System Calls

Most programs can't work directly with things like files and sockets. Instead, they have to use "system calls" to ask the operating system to do things on their behalf. It's up to the OS to enforce file permissions and handle the details of working with specific hardware.

Consider the following Elixir script:

# Here I'm calling an Erlang function. I'll explain why later.
{:ok, file} = :file.open("/tmp/tmp.txt", [:write, :raw])

foo = "foo"
bar = "bar"

output = [foo, bar, foo]
output = Enum.join(output)

# Another Erlang function call
:file.write(file, output)

It's fairly simple: we open a file, create some strings, concatenate them, and write the result to the file.

When executing the last line, the BEAM makes a system call to write the file. Using Evan Miller's dtrace script (linked from his excellent article), we can see what that looks like:

write:return Write data (9 bytes): 0x00000000146007e2

Here the BEAM uses the system call write and says "please write 9 bytes from memory address 0x00000000146007e2". That 9-byte string contains 3 bytes for "foo", 3 for "bar", and 3 for "foo" again.

Now watch what happens if we comment out the line where we join the strings:

{:ok, file} = :file.open("/tmp/tmp.txt", [:write, :raw])

foo = "foo"
bar = "bar"

output = [foo, bar, foo]
# output = Enum.join(output)

:file.write(file, output)

This time, we're passing an IO list to :file.write/2. It's a small change, but look at the system call:

writev:return Writev data 1/3: (3 bytes): 0x0000000014600430writev:return Writev data 2/3: (3 bytes): 0x0000000014600120
writev:return Writev data 3/3: (3 bytes): 0x0000000014600430

What we're seeing is actually one call to writev asking to write 3 chunks of data: "foo" from one address, "bar" from another, then "foo" from the same address as the first time.

This is very exciting. Nowhere in our program is the final string, "foobarfoo", produced. The only place where the concatenation happens is in the file itself.

When we concatenated in our own code, we took two strings in BEAM memory, copied their contents into a third string, and asked the OS to write that new string to a file.

When we used the IO list, we skipped the work of concatenation, saved ourselves from allocating the full 9-byte string, and removed the need to garbage collect that final string.

All the BEAM had to do was to ask the OS to copy each byte of data to the file.

Caveats for writev

Now, I said that the BEAM doesn't concatenate the strings in an IO list when performing I/O. And if you run this in IEx and trace the system calls, you'll see each item in the IO list as a separate argument to writev.

{:ok, file} = :file.open("/tmp/tmp.txt", [:write, :raw])
:file.write(file, some_iolist_of_your_own)

However, I made two important choices in that snippet to ensure that writev was used.

First, I specified that the file be opened in raw mode. This tells the BEAM not to create a separate process to handle the file, as it normally would. You can read about this in the Erlang docs for file:open/2 and in the "Performance" section at the bottom of that page.

Second, I used two Erlang function calls, not the Elixir code you'd expect:

File.write("/tmp/tmp.txt", some_iodata, [:raw])

That Elixir function delegates to a similar Erlang function, file:write_file/3, which you could call like this:

:file.write_file("/tmp/tmp.txt", some_iodata, [:raw])

This function takes care of opening and closing the file handle for you. In the current release of Erlang/OTP (19.1.2), :file:write_file/3 has a bug: it always combines the IO data into a single string, even if told to use raw mode. I recently fixed that, so once the fix gets released, you'll be able to just use the Elixir code shown above. But for now, if you want to use writev when writing files from Elixir, you'll have to use Erlang's :file.open/3.

Another caveat about writev: you may see strings combined if they are less than 64 bytes long. This has to do with the way the BEAM tracks strings in memory and copies data sent between processes. If your IO list contains larger, "refc" strings (longer than 64 bytes), you will see them show up in the writev vector as their own entries.

Use IO lists for I/O

Caveats aside, here's some simple advice: if you're going to build up some output and then write it to a file, write it to a socket, or return it from a plug, don't concatenate strings. Use an IO list.

It will help you build your output quickly and with minimal memory usage. And by doing this, you allow the BEAM to use writev to reduce copying.

As I mentioned, the Phoenix framework takes advantage of this technique. In my next post, I'll show you how.