On multiple occasions I've listened to instance admins speak about high S3 costs. The sheer amount of data absolutely balloons the more activity your server sees, I get it.
What I don't get is whether there's some unknown fedi ethical reason everybody insists on setting up an S3 cache (followed immediately by complaining about it).
Y'all want to know what the rest of the web does? Hosts their own uploaded media, and links out to the rest...
It's worthwhile to expand on a point to @devnull that I made: "preventing the sending server from seeing the IP" is a mostly* BS justification for local caching of media.
Broadly speaking:
Inconsistency around security policies is a recipe for dramatic, consequential failures.
Users are not notified if this is a feature, and clients and servers can both override it.
Kennt sich jemand mit dem ObjectStorage von IONOS aus? Was kommen da für Traffic-Kosten, bei einer durchschnittlichen Mastodon Instanz, auf einen zu? #mastoAdmin#admins#s3 :boost_ok:
The whole S3 charging for unauthorized/denied accesses to #S3 shows exactly the culture of #Amazon . Just because they reversed this policy (TBD if they actually do) doesn't mean that other similar policies will be changed. That the support person couldn't raise concerns, that the middle managers didn't care enough about the customers to realize how bad/stupid/damaging it is.
Sooo dann möchte ich den Interessierten mal den S3 Anbieter vorstellen, den ich entdeckt habe. Ich dachte bei dem Namen "Xaweho" erst an eine chinesische Firma, jedoch ist es ein deutsches Unternehmen mit Sitz in Xanten (das liegt bei Essen / Düsseldforf)
Die Vorteile sind fast schon einzigartig:
-keine Traffickosten egal welche Richtung
-nur 3,79€ pro TB im Monat
-so viele Buckets, User, ACL wie man möchte
-keine Bandbreitenlimitierung.
Damit ist das Angebot sogar noch besser als das von Contabo
Das Ganze wird sehr einfach erreicht: Man bekommt seine eigene MinIO Instanz auf deren Servern. Alles ist redundant angebunden.
Ein extrem faires Angebot und ich nutz es nun für meine Backups. Ich möchte noch Mastodon darauf umstellen. Interessant dürfte das für jeden größeren Instanzadmin sein, da ja auch Firefish, Lemmy und auch Misskey ein S3 Backend anbieten.
I need a Github Action template "upload-to-S3-provider-who-is-not-AWS-for-dumb-bimbo" because damn that overcomplicated devops ecosystem is gatekeeping simple babes like me 😭💅
(look at this rocket-science shit called AWS documentation… what the hell)
#S7 does just what I want, but dare I use it in anything that needs to be maintainable? OO R implementations get replaced so quickly. Is it better to just pretend the only options are #S3 and #S4? #rstats
Hey @arch you might find this interesting, I just got this in my weekly “Interesting Tools" newsletter. It is an #S3 compatible distributed file system that even has #kubernetes drivers and much much more. https://github.com/seaweedfs/seaweedfs it is called #SeaweedFS
It’s been a pretty successful week so far. Got a #lambda function to update our #S3 bucket policies and #sql backups direct to an S3 bucket accomplished too. Off to a good start!
#AWS and the #cloud is a fun place to be right now.
The last few days I've been contemplating just how "chicken and egg" my digital #DisasterRecovery plan is.
I #SelfHost#Bitwarden for passwords. It's backed up to #S3 - but the credentials for getting access to those backups are stored in... you guessed it, Bitwarden.
Additionally my TOTP codes are in Aegis. If my house burns down and I lose the phone I can just restore them from google. Except my Google account is in Bitwarden - and that's protected with TOTP.
People may want to reconsider using #AWS#S3 for static web hosting, or at the bare minimum come up with convoluted names and treat their S3 bucket name as sensitive information. If your S3 bucket name comes up in any web search (for example because it's literally in a public GitHub repo), that's a potential attack vector.
I'm planning to write a guide on how to switch your Mastodon instance from Scaleway S3 to Self hosted Minio S3. There are some "lessons learned" that I would like to share.
... But as always, time is limited... I hope I'll get this done before all my thoughts are vanished 💭😬
So S3 Select assumes that an S3 object is a CSV/JSON/Parquet blob, parses it to generate a database table on the fly, and then runs a query against it in a SQL subset dialect.
I think that's what I'll dress up as for Halloween.
I briefly looked at AWS S3 Glacier storage, thinking maybe having a second cloud host for my backups would be good.
I can't actually figure out how much this will cost me because they charge per operation (you know, like PUT, GET, etc.) in addition to the storage costs (which I easily figured out).
I just came across a great article by Antonia Langfelder on #ApacheTika's tika-pipes module and the /async handler, enabling reading from and writing to #s3.
The point about setting 'OMP_THREAD_LIMIT=1' to limit tesseract is interesting.
Hey, #Fediverse
My #mastodon instance stopped recognising updates from my #pixelfed instance that I follow from it about a week ago. Both appear to be otherwise federating okay.
When I open the followed pixelfed account from mastodon, it shows no updates since a week ago.
Where to start looking, please? I did a cleanup and moved to #S3 storage last week but have done many pixelfed posts since then.
Storj DCS isn't suitable for Pixelfed or Mastodon instances because of the segment limit.
Also the pricing is pretty complicated 😕..
Now I want to migrate to Scaleway. Any thoughts?