After some great discussion here yesterday on the topic of hijacked S3 buckets, I wrote up this blog post covering how I've combatted this in the past at multiple organizations.
In the post we explore how S3 bucket takeover occurs and how you can prevent it for buckets you own. Ultimately this is a software supply chain attack and should be a addressed as a security issue. #security#aws#s3
@toxi I have some experience with this. I recommend never relinquishing S3 buckets after using them for public facing production services. You are only charged for S3 storage so if you camp on an empty bucket name for the life of your account, you incur no charges. Also, add a zero length file for future people (possibly yourself) named something like DO_NOT_DELETE_S3_BUCKET_PERSISTENT or adding a tag like PERSISTENT=true.
I call this technique "S3 bucket takeover neutralization". #s3#aws
I just came across a great article by Antonia Langfelder on #ApacheTika's tika-pipes module and the /async handler, enabling reading from and writing to #s3.
The point about setting 'OMP_THREAD_LIMIT=1' to limit tesseract is interesting.
I have ses receive mail and put it directly into an s3 bucket.
Bucket has a notification to topics for creates into the report and forensic subfolders to a sns/sqs that feeds the lambda to process them. Then I can batch them.
Then lifecycle policy on bucket to clean up reports.
Why would anyone choose to use AWS? What are the killer features that keep you using it? It seems like there are so many better comparable or even better services out there.
@BraveLilToaster It depends on if storage is the only thing you need. I don't think Backblaze offers static website hosting or event notification support, like AWS S3 does? How about multiregion availability?
Version 8.7.1 is now available with bug fixes 🐛 https://cyberduck.io/changelog/. Resolves interoperability using AWS CLI configuration for #S3 connections.
Testing out OVH Cold Archive (LTO tape backed)
11.02TB stored came out to ~$23 post-tax.
This data is immutable and cannot be accessed or deleted without recovery, plus requires a 180 day minimum storage time (if you delete it before then you are charge the remaining time).
Comparing to Scaleway C14, this is definitely more affordable for the same amount of data, but is however less flexible; C14 has no min-time and is easier to restore and push to. #Datahoarder#OVH#Scaleway#S3#LTO
So S3 Select assumes that an S3 object is a CSV/JSON/Parquet blob, parses it to generate a database table on the fly, and then runs a query against it in a SQL subset dialect.
I think that's what I'll dress up as for Halloween.
Kennt sich jemand mit dem ObjectStorage von IONOS aus? Was kommen da für Traffic-Kosten, bei einer durchschnittlichen Mastodon Instanz, auf einen zu? #mastoAdmin#admins#s3 :boost_ok:
I tried firing up a #Windows instance in #AWS#EC2. Super-easy, fast, and doesn't cost anything when not in use (an $0.12/hr when in use). You can snapshot the #EBS volume to #S3 and delete/restore it later to save more. If you’re like me and only need access to Windows periodically, it's a great way to go.
It’s been a pretty successful week so far. Got a #lambda function to update our #S3 bucket policies and #sql backups direct to an S3 bucket accomplished too. Off to a good start!
#AWS and the #cloud is a fun place to be right now.
🤔 Armon Dadgar announces #HashiCorp's #BSL future
🏆 Matt Rickard on why #TailwindCSS won
🕴️ WarpStream is like #Kafka directly on top of #S3
🧩 Vadim Kravcenko’s guide to managing difficult devs
📢 Russ Cox gives an update on #golang 2
🎙 hosted by @jerod