@wimpy I see it as an issue on both sides. A #cdn should handle huge spikes in traffic. I would look at more details of how the CDN is being leveraged. There are some settings to tweak to improve performance.
On the other side, #Mastodon should fix this issue. It's sounds like a bad design smell. Each instance should cache the link preview at the minimum. It shouldn't hit the source for per follower. #twocents
Hey #journalists if you're going to use an open source project as an example of a hack, can you at least let them know in advance you're about to do something reckless so they can prepare?
Não uso muito streaming, mas no início da pandemia, dando suporte às aulas/eventos remotos percebi que nitidamente as soluções Google funcionam mais fluidas.
Não só em relação às soluções on-premisse (Jitsi, ConferênciaWeb, BigBlueButton), mas também em comparação com Zoom, Teams, Skype.
O que Google tem diferente que explicaria isso?
Redes de entrega de conteúdo mais próximas?
Parcerias com provedores de internet, tipo aquilo que o Netflix faz (uma espécie de cache local)?
Algum privilégio nas infraestruturas de transporte de dados?
I enabled Brotli compression on the CDN which serves the main BBC websites (www.bbc.co.uk. www.bbc.com etc.) outside the UK this morning.
Over ~4 hours, we're seeing a mean of ~20% better compression (smaller responses) via Brotli & ~95% of responses being Brotli now.
I've not had time to look in detail at performance but there doesn't look to be a significant change (LMK if you see diferent!).
(the spikes are breaking news events linking to a large "live" pages) #Brotli#WebDev#BBC
A little update on our enabling of Brotli for www.bbc.co.uk, www.bbc.com etc.
We're seeing compression improvements of roughly 15-40% over gzip. 15% is for HTML only, 40% is the overall. The caveat is that some clients which don't support Brotli request unusual content so this may be skewed to some degree.
I'll cover an issue which has cropped up in the next post. #WebDev#Brotli#CDN#Compression#BBC
Our stack is: Fastly -> GTM (BBC CDN) -> Belfrage (BBC routing) -> origins for most of our modern web pages.
Currently, only Fastly supports Brotli, the others do gzip, deflate & no compression.
Fastly strips gzip,deflate from the accept-encoding header sent to origin so our layers all return uncompressed content which means they're using more egress bandwidth. It's not a huge problem for us but something I thought might be useful for others to know. #WebDev#Brotli#CDN#Compression#BBC
Non so quanto senso abbia onestamente, ma probabilmente se potete è meglio firmarla che non. Un mesetto fa ho già parlato della questa questione anche qui sopra, e le cose sono solo peggiorate… non ho una timeline completa sotto mano ma insomma, i #blocchi sono stati più profondi e #problematici, sono stati bloccati siti legittimi ed economicamente e moralmente danneggiate persone che hanno quindi anche sporto querela, tramite un nuovo #blocco di #CDN anche di #CloudFlare (più incidenti minori). Quest’ultima si sta anche giustamente incazzando e ora sta iniziando pure a inviare mail ai suoi utenti chiedendo di lamentarsi con le autorità dei #problemi che questa #merda sta causando e potrà continuare a causare. Ma io ripeto, passino loro che anche se sono una parte (ahinoi) fondamentale dell’infrastruttura di #Internet non sono conosciuti dai normie… ma io ora aspetto che venga fatto un bel torto magari ad Amazon, a Microsoft, a Google, insomma altre aziende ben conosciute anche da chi non mastica tecnologia… me ne farò di risate quando tutti loro si coalizzeranno per boicottare #AGCOM e compagnia bella! 😍
E comunque rimango dell’idea che ‘sta cosa non si doveva proprio fare, perché nel nostro paese abbiamo il #vizio di fare sempre tutto e male finché ci sono gli interessi privati di mezzo (mentre quando ci sono gli interessi del popolo, le cose non si fanno e basta). Tutti i nostri parlamentari sono dei #cretini e noi come popolo siamo dei #coglioni; vi ricordo che la legge che ha portato a #PiracyShield era stata approvata all’unanimità, una cosa assurda, sicuramente un po’ per le #mazzette date dalle #lobby del #calcio ai nostri politici, e sicuramente un altro po’ per peer pressure tra quei signorotti sia al #governo che all’opposizione. Perché il colmo definitivo è che questa #cagata non funziona! Io continuo a seedare (caricare via torrent) decine e decine di GB al giorno di materiale #pirata, e il fatto che ci siano la nuova #legge e la #piattaforma scassata non mi ha impattata di una virgola; letteralmente, magari sarà che io ho a che fare con materiale che non è sport, però davvero pirato tanto bene quanto prima senza alcun accorgimento. #Soldi pubblici semplicemente bruciati, e nessuno che se ne vuole prendere la responsabilità, la #questione alla fine è anche questa, non sono solo i #danni causati e prospettati al #web. 🤮
Though the plugin still has some rough edges and missing features, overall I think it is working pretty well for what I wanted. When I post on joelencioni.com, I can see my post in my Mastodon feed pretty quickly—though sometimes the formatting isn’t the best but I suppose that’s just the way it is. And, commenting is bi-directional: replies from Mastodon show up as comments on my blog, and replies to those comments from my blog show up as replies on Mastodon. Great!
The ActivityPub plugin watches for when the client sends a request HTTP header that is asking for content with the mime type of “application/activity+json”. If that type of content is requested, then instead of responding with the web page, it will respond with some JSON data meant for machine consumption instead of human consumption. This is how the syndication works, and that all seems fine.
However, I have been bumping into an issue due to the way this all works together with the CDN I chose for page caching, CloudFlare.
This is what one of my blog posts looks like when the ActivityPub JSON is served instead of the web page.
The problem is that CloudFlare will cache the first version of the page that is requested and serve that up to everyone going forward, regardless of the type of content being requested.
Thankfully, I found a new approach to try. Using CloudFlare workers, I can program the CDN to vary the content based on this header with this bit of code:
export default { async fetch(req) { const acceptHeader = req.headers.get('accept'); const url = new URL(req.url); if (acceptHeader?.indexOf("application/activity+json") > -1) { url.searchParams.append("activitypub", "true"); } return fetch(url.toString(), { cf: { // Always cache this fetch regardless of content type // for a max of 5 minutes before revalidating the resource cacheTtl: 300, cacheEverything: true, }, }); }}
This tells CloudFlare to look at the accept header, and if it has “application/activity+json”, it will add “activitypub=true” to the request query string (the part of the URL after the question mark) behind the scenes, which effectively makes it a different URL. This allows the different content to be cached and served up differently, which I think should solve the issue for me for good. If you still see this problem, please let me know!
For a very small instance with only a couple of concurrent users a CDN might not make much difference. But if you take a look at your web server logs you’ll quickly notice that every post / like / vote triggers a storm of requests from other instances to yours, looking up lots of different things. It’s easy to imagine how quickly this would overwhelm an instance once it gets even a little busy.
One of the first web performance tools people reach for is to use a CDN, like Cloudflare. But how much difference will it make? In this video I show you my web server logs before and after and compare them.
The short answer is – before CDN: 720 requests. After CDN: 100 requests.
Usually just turning on a CDN with default settings will not help very much, you’ll need to configure some caching rules or settings. By watching your server logs for a while you’ll get a sense for what needs to be cached but check out mine for a starting point:
Beware of caching by URI Path because often fediverse software will return different data depending on the Accept header that the requester sets. For example, on PieFed and Lemmy instances a request by a web browser to /post/123 will return HTML to show the post to someone. But when that same URL is requested with the Accept: application/ld+json header set, the response will be an ActivityPub representation of the post! You don’t want people getting activitypub data in their browser and you don’t want to be serving HTML to other instances. Once you spot a URL you want to cache, use a tool like Postman to set the Accept header and make a fake ActivityPub request to your instance and see if you get back HTML or JSON.
Another problem that can happen is that often a response will vary depending on whether the viewer is logged in, or who is logged in. If you can figure out how to configure the CDN to pay attention to cookies or whatever headers are used for Authentication by your platform then you might be able to cache things like /post/*… I couldn’t.
The things I’ve chosen to cache by URI Path above are ones that I know don’t vary by HTTP header or by authentication.
Although we can’t use URI Path a lot of the time, we can cache ActivityPub requests by detecting the Accept: allocation/ld+json header:
https://join.piefed.social/wp-content/uploads/2024/02/caching_activity2-1024x811.pngThis will cache all ActivityPub requests, regardless of URL. People browsing the same URLs as those used by ActivityPub will be unaffected as their requests won’t have the special HTTP header. I used a short TTL to avoid serving stale data when someone quickly edits a post straight after creating it.
There seems to be a deep vein of optimization here which I’ve only just started to dig into. These changes have made a huge difference already and for now my instance is under very little load so I’ll leave it there for now…
🪡 🧵 Would you like to learn a new skill while doing something good for the environment? The éco-quartier Côte-des-Neiges and the FabLab at the Bibliothèque Interculturelle are joining forces to give you the chance to learn how to use digital sewing machines! 🪡 🧵
If Substack is perfect for your needs then use that. Your problem with substack prolly isn't who else uses it, but rather, that you yourself are calling a proprietary, privacy disrespecting deprecated monolithic silo a "Perfect solution".
Instead of doing what's right, and for the right reasons, you eschew dogfooding on #FOSS when you should be championing it, and call a professional data mining haven perfect, when it is anything but.
Well, you're already on the Fediverse, so you should know better, but I'll dispense with the lecture now and point out a few good FOSS solutions that are Fediverse powered (and one that isn't, but still rocks as a publishing platform) for you:
Option #1, #WriteFreely, which you can find over at its git repo under https://gitHub.com/writefreely/writefreely.
Option #2, deploy yourself a #WordPress site, Then install the #ActivityPub plugin - the latest release publishes into the Fediverse and allows any Fediverse account to reply/comment threads natively - like I'm responding now. It also allows anyone on the Internet to join the discussions as well. WordPress has many options for subscriber lists, Etc., as well as #paywalled#digital_downloads, if you like.
Option #3, #Mitra is a Fediverse publishing platform that currently supports paid subscriptions for Authors: https://mitra.fediverse.observer/list - pick one that has open registrations or self-host yourself, like all of the other solutions here :)
If you're really talking about maintaining subscribers lists, but especially Having a subscriber list and building it up, then most ignorant folks would recommend HubSpot - but they would be wrong, because you can get the same powerful inbound marketing solution / #CRM, only better, for #FREE (That's a bare minimum savings of over $500/month)!!! So install #Mautic and let it do what it does, which you can get here: https://www.mautic.org/download/source-code and then after that, use it in conjunction with the following FOSS application that was tailor made for exactly what you're asking for...
#Ghost is FOSS, and in conjunction with an inbound marketing platform like Mautic is the perfect dynamic duo - like Batman and Robin. But even better, is that I'm going to point you towards a #HowTo that is an actual cookbook #tutorial written by someone expressing the same lamentations as yourself, and here's the exact solution they've provided for you:
By the way, your Mautic server also integrates directly with#MailGun (or Sendgrid, SendinBlue, SparkPost, etc.) to complete your transactional email system that will tell you when each and every recipient received, viewed (and or how long) your emails, as well as how many times they looked at those emails, with a bunch of other tools as well.
I hope that helps, and I'm very glad that you came to your senses about not using a privacy disrespecting, proprietary closed source solution like Substack - besides, registering your own domain name would have hidden the fact that you were using substack anyway, so it's about YOU doing the right thing the right way. Please choose your software in the future based upon the freedoms and ethics it offers in serving you and your customers. There's evil people everywhere, and the smart ones are using FOSS too - not substack.
It's good to receive feedback that helps people determine information that has value to others. It helps us focus on topics with merit.
There are a couple of additional things I'd like to address though, as briefly I can, considering I'm a rather loquacious sort ;)
I think it was @frogzone that brought up the general controversies that typically do follow #Cloudflare around. I have privacy conscious friends on both sides of that widening chasm...
In general it tends to be the developer sorts that although are cautious, reserved usually, when passing around compliments where Cloudflare is concerned, they're also the pragmatists where performance and dare I say security is concerned, and are often quite willing to turn to Cloudflare (specifically, as a #CDN).
With respect to security concerns, it is true that incorporating a CDN does provide a level of obfuscation of the IP spectrum, that is often cited as a major reason by hosting providers for the customer to incorporate/subscribe to CDN services (more often than not, Cloudflare - because they offer better kickbacks (er.... incentives) to hosting providers.
Then there's the hard core privacy concerned folks. #Last_Mile delivery performance considerations typically being much less of a compelling reason to use, let alone pay, for a CDN like Cloudflare to be injected into the website admin's #DNS. This is because, and let's be real here folks, most websites don't generate anywhere near the levels of traffic that their Nginx or Apache Servers can easily serve up, and for folks on the other side of the world from the particular website, a few milliseconds on a clear day is negligible.
Now, if you're running a very busy site, like... Etsy, or even really popular sites with thousands of requests per minute then you can really benefit by spreading your cache around the globe on super fast CDN services. Even a site that receives on average 1 request per second (60 per minute - and that's pretty respectable traffic) doesn't really benefit enough from the #caching related benefits of a CDN to mark a compelling case - the Last Mile Delivery, however, to Oslo, Norway, from a website in Melbourne, Australia... that can indeed improve perceived response by 250ms (2.5 seconds) or so.
So, just like these so-called VPN services, like NordVPN, etc., there needs to be an effort to educate the consumer as to the actual benefits expected for specific matters - some may be important considerations for the consumer, while others may just be a tech support person in a boiler room trying to reach that bonus number for the month... I've seen waaaay too many people purchase services they really didn't need or would receive much benefit from, and many support desk personnel upselling customers with things they probably shouldn't have.
Now, there's another thing I didn't mention - #Denial_of_Service attacks... Good ole #DDoS campaigns. Well, first of all, one should check with their hosting provider - whether they have the benefit of protections against such attacks, and then, weigh the added benefit of using something like Cloudflare to do the same job (are you paying for protection that you might need twice?).
I personally would probably not have included Cloudflare as part of the #HowTo. It can be added at anytime, but some folks swear by it, so it's not that I'm on the fence about Cloudflare, it's just that I look at it more from the engineering and security perspective, with an eye specifically focused on the veracity of any perceived needs by the customer. And I'm not super fond of turning all of that DNS control (and valuable #metadata) to some third party.
I realize that may have only served to raise more questions, so I'll just say that this is why you pay your trusted IT support professionals who make all of their money on labor they've billed you for, to sit down and discuss what you may or may not need, and especially, why 👍
Brenden Eich was invoked by @marathon - and I too, concur that It is only right to measure technology based on it's own merit and capability - without regard to superfluous and unrelated matters of personal politics.
When haters start fomenting hatred, disparaging everyday, average people for their informed choice of technologically capable software relevant to the task at hand, I like to remind those vile, adolescent, sniveling children that they're literally denigrating things like Brave Browser and Soapbox (the platform I'm authoring this post on), while at the same time availing themselves of the full compliment of features that #JavaScript's technology affords them - JavaScript, invented by#Brenden_Eich...
And they have my blessings to completely swear off and forgo ever using JavaScript again - but they won't, will they? Why? Because they're filthy, hateful, hypocrites consumed by their own criminal commiserations.
Hands up if you caused a global outage today...
Just me?
Sorry!
I was making a change to our "outside the UK" CDN config today for www.bbc.co.uk & www.bbc.com & the change included 2 bugs which pre-testing didn't spot:
#Discord will switch to temporary file links to block #malware delivery
Discord's #CDN is regularly abused by threat actors to host and drop malware such as information stealers. Discord has always struggled with mitigating this.
The hope is that temporary file links will make it that much harder for threat actors to host malware using Discord's CDN... but we will see. Supposed to roll-out by end of the year.
@tayledras@noiq As a matter of fact, I did help a fmr. client/employer to move their #Servers and #VMs inhouse again amidst escalating costs...
Turns our it's cheaper to invest 8 digits once to reduce the hosting bills from 6 to 4 digits espechally since they ain't having short-term & highly fluctuating loads that justify the cost.
I mean, I know cases where that does apply and in these cases it's worth paying €€€€ to do #livestreaming on a #CDN like #dacast / #Akamai, but that's an exception.
Cloudflare. A #cybersecurity service that "protects" you from bad actors, #hackers, #DDoS and other online threats.
What seemingly no one talks about is that 80% of #CDN market is owned by Cloudflare. They move more traffic than #FAANG companies combined!
They hold such obscene amount of power and control over the #internet, its hard to describe.
They are the gatekeepers, they are enforcers of whom to block, and what to allow to exist.
Google holds no power when compared to Cloudflare.
#Cloudflare often blocks my ISP connection on websites that use it, so why would I trust #CloudflareFonts? This #CDN gets zero approval from me. If anything, this #web#fonts service just creates yet another pocket for cache alongside#GoogleFonts (that majority of websites were brainwashed to use already). If it's what to #DeGoogle is, then I suggest avoiding extra hops and using #BunnyFonts instead.
Dear friends of #BSDCafe and #Fediverse, pgbouncer is now up and running. You probably won't notice any changes as we didn't have performance issues related to the database, but this is a more suitable and efficient approach.
In recent days, I duplicated the jail containing Mastodon (the operational one) to support sidekiq and serve as a backup for the rest. For this reason, you shouldn't have noticed any disruptions because while I was making changes to the primary jail, BSD Cafe was still operational using the secondary one (on another host).
I'm seriously considering conducting an experiment and replicating this jail (along with the reverse proxy) on a different continent to speed up user access in that location. Currently, everything is in Europe (Helsinki), so I might replicate somewhere in the Americas.
I'm not sure if this would yield tangible benefits since the database, Redis, etc., would remain in Europe. Multimedia files are already on a CDN, so they are georeplicated.
It looks like the CDN files.mastodon.online for #MastodonOnline is currently broken and redirecting everything to https://(null)/ from files.mastodon.online.
If you're wondering why some pictures and media aren't loading right now, this is why.