Processes, Threads, Goroutines, and BEAM: How Go and Elixir Handle 10,000 Connections

I build things that hold many connections at once. A DNS server in Go called ns. A Redis-like key-value store in Elixir called exstore. A dispatch backend where many drivers and patients are connected at the same time.

Every one of these projects forced me to answer the same question: when 10,000 connections arrive at one server, what actually handles them?

The answer is different in every language. And understanding the difference changed how I write backend code. Let me explain it simply.

1. The 10,000 Connection Problem

Imagine you run a chat server. 10,000 people are connected. Most of them are doing nothing, just sitting there, idle, waiting for a message. Only a few send something at any given second.

The naive approach is obvious: give every connection its own worker. One connection, one thread. When a message arrives, that thread wakes up and handles it.

This is called thread-per-connection. It is easy to write. And it falls over somewhere around a few thousand connections. This exact wall even has a name from 1999: the C10K problem.

Why does it fall over? To answer that, you need to know what a thread costs.

2. What a Process and a Thread Actually Cost

A process is a running program with its own private memory. Your browser tab, your terminal, your database, each is a process. The operating system keeps them isolated so one crashing does not corrupt another.

A thread lives inside a process. Threads in the same process share memory but each has its own stack and its own execution state. The kernel, the core of the operating system, schedules threads onto your CPU cores. When the kernel switches from one thread to another, it must save registers, swap stacks, and sometimes flush CPU caches. That switch is called a context switch, and it is not free.

Here is the math that kills thread-per-connection:

• On Linux, every new thread reserves around 8 MB of stack space. Most of it is virtual, not real RAM, but the kernel still tracks it, and real usage grows fast with deep call chains and buffers.
• 10,000 threads means the kernel scheduler is juggling 10,000 entities. Every context switch burns CPU time doing bookkeeping instead of your work.
• Threads share memory, so every shared counter, queue, or cache needs locks. Locks mean waiting. Waiting means idle CPU. More threads often means slower, not faster.

I learned this the hard way reading about container internals while building ship. A container is just an isolated process, and isolation has a price. Threads are cheap compared to processes, but 10,000 of anything the kernel must schedule by hand is heavy.

So languages invented lighter workers that the kernel never sees.

3. Goroutines: Thousands of Workers the Kernel Never Sees

Go’s answer is the goroutine. Think of it as a thread managed by the Go runtime instead of the operating system.

When you write go handle(conn) in Go, you are not creating an OS thread. You are creating a tiny task, starting at just a few kilobytes of stack, that the Go scheduler multiplexes onto a small number of real OS threads. This design is called M:N scheduling: many goroutines onto few OS threads. The number of OS threads defaults to your CPU core count (controlled by GOMAXPROCS).

Three things make this work:

Tiny stacks that grow. A goroutine starts at a few KB and grows or shrinks as needed. An OS thread reserves megabytes. So 100,000 goroutines use roughly the memory of a few hundred threads. That alone removes the memory wall.

The scheduler lives in your program. Switching between goroutines does not require entering the kernel. It is just the Go runtime swapping pointers. A goroutine switch costs tens of nanoseconds; a kernel thread switch costs microseconds. At 10,000 connections, that difference is everything.

Blocking code doesn’t block the machine. This is the clever part. When a goroutine waits on network I/O, Go’s runtime parks it and runs other goroutines on the same OS thread. There is a component called the netpoller that watches all the idle connections efficiently using OS facilities like epoll. You write simple blocking-looking code, and the runtime makes it non-blocking underneath.

Here is what my DNS server’s core loop looks like in spirit:

func main() {
    conn, _ := net.ListenPacket("udp", ":5353")
    buf := make([]byte, 512)

    for {
        n, addr, _ := conn.ReadFrom(buf)
        // copy the bytes, then handle each query
        // in its own goroutine, cheap, ~KBs each
        go handleQuery(conn, append([]byte(nil), buf[:n]...), addr)
    }
}

func handleQuery(conn net.PacketConn, data []byte, addr net.Addr) {
    resp := resolve(data) // parse packet, look up record
    conn.WriteTo(resp, addr)
}

Each DNS query gets its own goroutine. With UDP there is no persistent connection, but the principle is identical for TCP: one goroutine per connection, tens of thousands of them, and the program stays calm. Writing this felt like thread-per-connection, simple, direct code, except it actually scales.

The catch with goroutines: they still share memory. If two goroutines touch the same map, you need a mutex or a channel. Go’s slogan is “share memory by communicating,” which means channels, pipes that carry values between goroutines. Channels are good, but a forgotten lock or a blocked channel can still deadlock your program. The runtime gives you cheap workers, not freedom from thinking about shared state.

One sentence: you write blocking-looking code, and the runtime makes it non-blocking.

4. BEAM Processes: Lighter Still, and They Don’t Share Anything

Elixir runs on the BEAM, the virtual machine originally built for Erlang at Ericsson to run telephone switches. Phone switches must handle insane numbers of concurrent calls and must never go down. That history shaped everything.

A BEAM process is not an OS process and not an OS thread. It is a unit of execution managed by the virtual machine, even lighter than a goroutine, roughly a couple of kilobytes to start. A single BEAM node routinely runs millions of them.

But lightness is not the interesting part. Isolation is. Three ideas make BEAM processes special:

No shared memory at all. Every BEAM process has its own private heap. Goroutines and threads share memory with each other; BEAM processes physically cannot. They communicate only by sending messages, copies of data, not references. There are no locks because there is nothing shared to lock. A whole category of bugs (data races) simply cannot exist.

Preemptive scheduling by counting work. The BEAM counts units of work called reductions. Every function call costs reductions, and when a process uses up its budget, it is paused and another runs. No process can hog the CPU by accident, even an infinite loop in one process won’t freeze the other million. Go’s scheduler is cooperative in comparison: a tight loop with no function calls can starve other goroutines (newer Go versions improved this, but the BEAM was designed preemptive from day one).

Let it crash. This is the famous one. Since processes share nothing, a crashing process cannot corrupt anyone else. So instead of defensive programming everywhere, you organize processes into supervision trees: a supervisor watches worker processes and restarts them when they die. My key-value store uses exactly this shape, each piece of state lives in a supervised process, and if one dies, the supervisor brings it back with a known-good state.

Here is the shape of a supervised key-value holder, simplified from my exstore project:

defmodule Exstore.KV do
  use GenServer

  # Start one supervised process holding a map
  def start_link(_opts) do
    GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
  end

  def put(key, value) do
    GenServer.call(__MODULE__, {:put, key, value})
  end

  def get(key) do
    GenServer.call(__MODULE__, {:get, key})
  end

  # Callbacks: each runs inside the process,
  # one message at a time, no locks needed
  def init(state), do: {:ok, state}

  def handle_call({:put, key, value}, _from, state) do
    {:reply, :ok, Map.put(state, key, value)}
  end

  def handle_call({:get, key}, _from, state) do
    {:reply, Map.get(state, key), state}
  end
end

Notice what is missing: no mutex, no lock, no atomic. Messages arrive one at a time in the process mailbox, so the state is never touched concurrently. If this process crashes, its supervisor restarts it. Compare that to a Go map guarded by a sync.RWMutex, where one sloppy access from any goroutine panics the whole program.

The catch with the BEAM: per-message copying costs CPU, and the ecosystem is smaller. Raw number-crunching throughput per core usually loses to Go. You pay for isolation with copying overhead. For I/O-bound systems with oceans of connections, it is worth it. For crunching bytes as fast as possible, often not.

One sentence: processes share nothing, so a crash means a restart, not a catastrophe.

5. Side by Side

No row wins everything. That is the point most tutorials skip. Threads are the baseline the kernel gives you. Goroutines keep the familiar shared-memory model and remove the cost. BEAM processes remove shared memory itself and give you failure isolation in return.

6. What I Pick and When

After building in both, here is my honest rule:

• Go when the job is raw throughput on bytes and packets: DNS servers, proxies, CLI tools, API backends doing JSON and SQL. My ns DNS server is Go because UDP packet handling wants speed and the standard library networking is superb. Goroutines plus channels are enough when I can reason about the shared state.
• Elixir when the job is stateful fan-out with many fragile moving parts: chat, presence, dispatch boards, anything where “restart just that one thing” is the correct failure strategy. An SOS dispatch system, drivers connecting, disconnecting, reconnecting, jobs timing out, maps beautifully onto supervision trees. I would reach for Elixir there without hesitation.

And I avoid thread-per-connection everywhere. The one time I need OS threads is when the runtime asks for them, CPU-heavy work in Go gets real parallelism through GOMAXPROCS automatically, and the BEAM has dirty schedulers for the same reason. I never manage threads by hand.

7. The Lesson That Stuck

Learning concurrency taught me something bigger than syntax: the best model is the one you can debug at 2am.

Goroutines are easy to start and easy to leak, a blocked channel in a forgotten goroutine sits there forever, and I have stared at pprof goroutine dumps counting exactly those. BEAM processes are easy to start and easy to observe, the Observer tool shows every process, its mailbox length, its memory, live. A mailbox growing without bound tells you precisely where the jam is.

Both beat the old world where 10,000 connections meant 10,000 kernel threads and a prayer. The industry solved C10K two decades ago, but every new backend developer rediscovers it the first time their server falls over under load. I would rather you read this than live it.

If you take one thing from this post: count your concurrent entities, know what each costs, and know what happens when one dies. Everything else is details.