Well, there's a bit to unpack here. When you download something in seconds that's your whole connection to a server that is on a super fast connection.
Most people running instances are on a much more modest combination of hardware and connection and even the bigger ones (kbin.social etc) are not going to have a connection as fast as dedicated download CDNs can offer. I would expect they probably have a gigabit, and at most 10gbit. That's shared between everyone on the site downloading cat pictures, posting, refreshing AND the federation of all the new content to and from other instances.
But that's really not the problem here at all. Far more of a load is the processing of the incoming and outgoing messages to and from many instances. This takes CPU load (and to a lesser extent memory), and this is shared between the message queue for these inter-instance messages, the web server and database.
When you look at how the fediverse of kbin and lemmy is laid out. You will see there's a handful of larger instances with most of the popular magazines/communities. This means that they're doing the lion's share of this processing. Couple the fact that the population has exploded over the last month or so, even these larger instances might be struggling with hardware and/or infrastructure layout (maybe running all on one box, and needing to split the load for example). That's speculation though.
At any rate, for whatever reason (maybe the lemmy problems backing up message queues with errors) things were MUCH slower last night.