I briefly looked at AWS S3 Glacier storage, thinking maybe having a second cloud host for my backups would be good.
I can't actually figure out how much this will cost me because they charge per operation (you know, like PUT, GET, etc.) in addition to the storage costs (which I easily figured out).
People may want to reconsider using #AWS#S3 for static web hosting, or at the bare minimum come up with convoluted names and treat their S3 bucket name as sensitive information. If your S3 bucket name comes up in any web search (for example because it's literally in a public GitHub repo), that's a potential attack vector.
The whole S3 charging for unauthorized/denied accesses to #S3 shows exactly the culture of #Amazon . Just because they reversed this policy (TBD if they actually do) doesn't mean that other similar policies will be changed. That the support person couldn't raise concerns, that the middle managers didn't care enough about the customers to realize how bad/stupid/damaging it is.
I haven’t tested this myself, but it seems this may be a very nasty way to inflict targeted or random harm against anyone with #AWS#S3 buckets. #infosec
I need a Github Action template "upload-to-S3-provider-who-is-not-AWS-for-dumb-bimbo" because damn that overcomplicated devops ecosystem is gatekeeping simple babes like me 😭💅
(look at this rocket-science shit called AWS documentation… what the hell)
Hey, #Fediverse
My #mastodon instance stopped recognising updates from my #pixelfed instance that I follow from it about a week ago. Both appear to be otherwise federating okay.
When I open the followed pixelfed account from mastodon, it shows no updates since a week ago.
Where to start looking, please? I did a cleanup and moved to #S3 storage last week but have done many pixelfed posts since then.
#S7 does just what I want, but dare I use it in anything that needs to be maintainable? OO R implementations get replaced so quickly. Is it better to just pretend the only options are #S3 and #S4? #rstats
After some weeks of silence, there was some free time for a little blog post once again - after discovering that my very small instance took almost 1,8 TB of #S3-storage:
On multiple occasions I've listened to instance admins speak about high S3 costs. The sheer amount of data absolutely balloons the more activity your server sees, I get it.
What I don't get is whether there's some unknown fedi ethical reason everybody insists on setting up an S3 cache (followed immediately by complaining about it).
Y'all want to know what the rest of the web does? Hosts their own uploaded media, and links out to the rest...
It's worthwhile to expand on a point to @devnull that I made: "preventing the sending server from seeing the IP" is a mostly* BS justification for local caching of media.
Broadly speaking:
Inconsistency around security policies is a recipe for dramatic, consequential failures.
Users are not notified if this is a feature, and clients and servers can both override it.
In a recent announcement, Pixelfed creator Daniel Supernault (@dansup), shared exciting news for Pixelfed instance administrators. A forthcoming feature is set to empower admins by allowing the storage of imported media from Instagram directly on S3 Storage.
The development is part of a pull request (PR) on GitHub, where Supernault detailed the functionality of the feature. Admins will soon have the ability to opt-in to store Instagram-imported media on S3 filesystem driver. This marks a significant enhancement for Pixelfed instances, providing a seamless integration for media management.
Key Configuration Details:
To enable or disable the feature, admins can set PF_IMPORT_IG_CLOUD_STORAGE to true or false. Notably, this can only be activated if Cloud Storage (PF_ENABLE_CLOUD) is enabled. However, admins have the flexibility to disable this feature and retain Instagram-imported media locally, even with Cloud Storage enabled.
Existing local media will be seamlessly migrated without requiring any action from admins. A cron job will automatically handle the migration of both existing and new Instagram media. While the process may take some time for instances with substantial media content, Pixelfed assures administrators that the system is designed to efficiently manage the transition.
Migration Process:
During the migration, Pixelfed has chosen to silently update media URLs to avoid sending unnecessary “Update” activities. This careful approach ensures a smooth experience for users, with local media URLs gracefully redirecting to their corresponding S3 URLs when appropriate.
Pixelfed’s commitment to user experience and efficient media management is evident in this upcoming feature. Admins can anticipate enhanced control over media storage, providing a more seamless and scalable solution for Pixelfed instances.
The Pixelfed community eagerly awaits the official release of this feature, anticipating its positive impact on the platform’s media management capabilities.
After some great discussion here yesterday on the topic of hijacked S3 buckets, I wrote up this blog post covering how I've combatted this in the past at multiple organizations.
In the post we explore how S3 bucket takeover occurs and how you can prevent it for buckets you own. Ultimately this is a software supply chain attack and should be a addressed as a security issue. #security#aws#s3
I just came across a great article by Antonia Langfelder on #ApacheTika's tika-pipes module and the /async handler, enabling reading from and writing to #s3.
The point about setting 'OMP_THREAD_LIMIT=1' to limit tesseract is interesting.
Version 8.7.1 is now available with bug fixes 🐛 https://cyberduck.io/changelog/. Resolves interoperability using AWS CLI configuration for #S3 connections.
So S3 Select assumes that an S3 object is a CSV/JSON/Parquet blob, parses it to generate a database table on the fly, and then runs a query against it in a SQL subset dialect.
I think that's what I'll dress up as for Halloween.
Kennt sich jemand mit dem ObjectStorage von IONOS aus? Was kommen da für Traffic-Kosten, bei einer durchschnittlichen Mastodon Instanz, auf einen zu? #mastoAdmin#admins#s3 :boost_ok: