I was playing again with my #CommonLisp HTTP implementation and I've made my peace with its blocking nature. It is irrelevant with a buffering reverse proxy (HAProxy or NGINX), performances are excellent and the code of the server stays simple.
Here, 210k+ req/s on 64 connection handling threads with 390µs P90 latency and sub-ms P99 (Linux amd64, SBCL 2.4.1).
Of course I can't use the same approach for my SMTP server (too many parallel connections that stay alive and no buffering proxy possible), but not all software have to use the same language.
@louis No, there are 64 threads handling connections, i.e. reading requests and writing responses. However these read and write operations are blocking, meaning that they only return once data have been read/written. Without buffering, if a client opens 64 connections and does not send anything, the server will lose the ability to process requests.
With a buffering reverse proxy, e.g. NGINX, the proxy manages connections in a non-blocking way (usually with epoll or kueue), without requiring one thread per connection, and accumulates requests until they are complete, then only forward them to the real server. No risk of blocking. Of course in theory I could write the HTTP server in a non-blocking way in Common Lisp, but then we're back to the lack of green threads, callback hell, etc.
Add comment