🌐 The C10K Problem — The challenge that changed the internet

[marcelschmall@infosec.exchange]

The C10K Problem — The challenge that changed the internet

1999. The web is booming. Servers are struggling. Dan Kegel asks one simple question:

“Why can’t a web server handle 10,000 simultaneous connections?”

Not a bandwidth problem. Not a CPU problem. A design problem.

️ The old model was simple but deadly:
→ 1 connection = 1 thread
→ 10.000 connections = 10.000 threads
→ 80GB RAM just for thread stacks
→ Kernel spends more time scheduling than actually working

The server wasn’t busy doing work. It was busy managing the chaos.

C10K forced a complete rethink:
→ 1 thread handling thousands of connections
→ Non-blocking sockets
→ Event loops instead of thread pools

The ripple effects were massive:
→ epoll landed in Linux 2002
→ nginx was born with this model in mind
→ Node.js made it mainstream
→ Redis, HAProxy — all children of C10K

One blog post in 1999 rewired how the entire industry thinks about network servers. We’re still building on those ideas today.

#Linux #Networking #SystemsProgramming #WebDev

[ammarfaizi2@social.gnuweeb.org]

@Suiseiseki @marcelschmall

> Is it really that impressive [...]

Yes. Because spawning a thread is much heavier than waiting for `POLLIN | POLLOUT`.

Spawning a thread just to handle a single fd is definitely not resource-wise; you need a dedicated stack, a tid, scheduling parameters, I/O prio, a dedicated PCB, and so many other scheduling-related resources.

[ammarfaizi2@social.gnuweeb.org]

@Suiseiseki @marcelschmall

Before epoll was invented, there were already select() and poll() to handle multiple fds with a single process, but poll()'s interface wasn't very efficient because you had to traverse all the `struct pollfd` array members for each sysret.

If you have 10K connections and one connection sends you data, with poll(), you need to traverse up to 10K `struct pollfds` to find which connection sends you data.

select() is even worse; it can only handle up to `FD_SETSIZE` number of fds which is defined as 1024. The traversal is pretty much the same as poll().

epoll solves the problem by avoiding wasted iterations in the caller for iterating over connections.

[ammarfaizi2@social.gnuweeb.org]

@Suiseiseki @marcelschmall

Additionally, since Linux 5.1, we have io_uring, which offers advantages over epoll, such as better performance and more I/O operations.

#io_uring

[ammarfaizi2@social.gnuweeb.org]

@Suiseiseki @marcelschmall

But I think you should use io_uring with kernel 6.x+ to actually see how useful it is for various workloads. I'm not confident using io_uring on 5.x kernels.

[yuka@s.umeyashiki.org]

@ammarfaizi2@gnuweeb.org @Suiseiseki@freesoftwareextremist.com @marcelschmall@infosec.exchange the retval from poll can help optimize the traversal as it indicates the number of ready file descriptors.

I know it is still not as efficient as epoll, but if you find all ready fds earlier, you can break the iteration earlier too:

    ready_fds = poll(fds, 10000, -1);
    if (ready_fds < 0) {
        handle_error(ready_fds);
        return;
    }

    for (i = 0; i < 10000; i++) {
        if (ready_fds == 0) {
            // All ready fds have been handled.
            // Break out of the loop early to
            // avoid unnecessary iterations.
            break;
        }

        if (fds[i].revents & (POLLIN|POLLOUT)) {
            handle_events(i, fds[i]);
            ready_fds--;
        }
    }

If you are lucky: that one client is not in the last index, you can break earlier.