Resources on Multiplayer Server Development using Godot 4.0

emr, 8 months ago

It’s not specific to Godot 4, but I found this couple of videos really useful for understanding how to build interpolated multiplayer in Godot:

youtube.com/watch?v=w2p0ugw3afs (and the one after on extrapolation.)

Pantoffel, 8 months ago

Thanks 🙏

o11c, 8 months ago

You’ve clearly thought about the problem, so the solutions should be relatively obvious. Some less obvious ones:

• It is impossible to make TCP reliable no matter how hard you try, because anybody can inject an RST flag at any time and cut off your connections (this isn’t theoretical, it’s actually quite common for long-lived gaming connections). That leaves UDP, for which there are several reliability layers, but most of them are not battle-tested - remember, TCP is most notable for congestion-control! HTTP3 is probably the only viable choice at scale, but beware that many implementations are very bad (e.g. not even supporting recvmmsg/sendmmsg which are critical for performance unlike with TCP; note the extra m)
• If you don’t encrypt all your packets, you will have random middleware mess with their data. Think at least a little about key rotation.
• To avoid application-centric DoS, make sure the client always does “more” than the server; this extends to e.g. packet sizes.
• Prefer to ultimately define things in data, not code (e.g. network packet layouts). Don’t be afraid to write several bespoke code-generators; many real-world serialization formats in particular have unacceptable tradeoffs. Make sure the core code doesn’t care about the details (e.g. make every packet physically variable-length even if logically it is always fixed-length; you can also normalize zero-padding at this level for future compatibility. I advise against delta-compression at this level because that’s extra processing you don’t need).
• Make sure the client only has to connect to a single server. If you have multiple servers internally, have a thin bouncer/proxy that forwards packets appropriately. This also has benefits for the inevitable DDoS attacks.
• Latency is a bitch and has far-ranging effects, though this is highly dependent on not just genre but also UI. For example “hold down a key to move continuously through the world” is problematic whereas “click to move to a location” is not.
• Beware quadratic complexity, e.g. if every player must send a location update to every player.
• Think not only about the database, but how to back up the database and how to roll back in case of catastrophe or exploit. An append-only flat file has a lot going for it; only periodic repacking is needed and you can keep the old version for a while with a guarantee that it’ll replay to identical state to the initial version of the new file. Of course, the best state is no state at all. You will need to consider the notion of “transaction” at many levels, including scripting (you must give me 20 bear asses for me to give), trading between players, etc.
• You will have abuse in chat. You will also have cybersex. It’s possible to deal with this in a privacy-preserving way by merely signing chat, not logging it, so the player can present evidence only if they wish, but there are a lot of concerns about e.g. replays, selective message subsets, etc.
• There will be bots, especially if the official client isn’t good enough.
• It’s $CURRENTYEAR; write code for IPv6 exclusively. There are sockopts for transparently handling legacy IPv4 clients.
• Client IP address is private information. It is also the only way to deal with certain kinds of abuse. Sometimes, you just have to block all of Poland.
• Note that routing in parts of the world is really bad. Sometimes setting up your own dedicated connection chain between datacenters can improve performance by orders of magnitude, rather than letting clients use whatever their ISP says. If nesting proxies be sure to correctly validate IPs.
• Life is simpler if internal stuff listens on a separate port than external stuff, but still verify your peer. IP whitelisting is useless except for localhost (which, mind, is all of 127.0.0.0/8 for IPv4 - about the only time IPv4 is actually useful rather than a mere mirage).

Pantoffel, 8 months ago

Thanks for your reply :) This grew quite detailed and more thorough than I had expected. But I’m thankful to now have something to read up on. It will take me a while though.

There are two follow-ups:

1. I’d be surprised if there weren’t libraries for the networking parts already. You briefly mentioned there would be for at least data types for the interfacing and wrote that I should instead use my own solution instead for said reasons. Well, are there libraries I can use for networking? What you mentioned sounds very low level.
2. What do you mean with point (3), where the client “should do more” than the server? The server would be authoritative and the client predicting. But, do you mean that the server shouldn’t do so much heavy lifting like working with game physics?

Thanks!

o11c, 7 months ago

1. Sure, there are libraries and tools for some parts. But the question is: do they actually add value, or do they subtract it due to the cost it imposes to integrate them? There are only a couple parts where it is likely worth it (but even there, you’ll need to understand what they’re doing): the UDP reliability layer, the encryption layer, and possibly the event loop (but I say that mostly because io_uring is weird; previously I wouldn’t include this. On the client side it isn’t needed, but that depends on how much code you share between server and client). Packet framing and serialization are really easy to do yourself and most existing tools (which usually do generate code anyway) have weird limitations or overhead.
2. “should do more” is a general rule to prevent easy DoS attacks. It can mean packet sizes, it can mean computation, it can mean hard-coded timer delays, it can mean all sorts of things. Don’t make it easy for a malicious client to waste shared resources. This is mostly relevant for early in the connection … but keep in mind that it’s possible for a malicious client to spy on a legitimate client and try to take over - that is, connections aren’t actually a real thing (this applies even for TCP).