Kbin and NoSQL

Hi there ! (First post ever yay!)

I’m, like a lot of users lately, been migrating from Reddit to Kbin and Lemmy.

I finally had a good reason to contribute by writting HelmCharts to run Lemmy and Kbin on Kubernetes ( hopefully some of you might be interested !)

Anyways, i was wondering whrn analysing the docker stack of Kbin if instead of running postgresql, it could be replaced by a NoSQL service such as MongoDB.

I felt like the concept of the fediverse and the data structure seem pretty compatible.

And since there have been some performance issues ( probably for database) lately, it might be an interesting way to deal with large volumes of data.

Let me know what you think !

Cheers!

BiggestBulb,
BiggestBulb avatar

I think there's quite a bit to say about cost. I'm not sure how Mongo is, but I'm quite familiar with DynamoDB and it gets quite expensive at scale. I'd be worried it'd be unsustainable, especially since postgres is very cheap most of the time (aka, unless you go with RDS).

Also, I'm sure Postgres can be optimized to work well at kbin's scale (since it has worked on sites with millions of users for years). I'm hesitant to say it's an issue with postgres itself, it may be an issue with the queries to the database (maybe the lock?) or even with database auto-scaling (or possible lack thereof).

Also, kbin's general data seems to be highly relational - articles will always have an author, a title, how many votes it has, how many boosts, and all of the comments on it. I think (personally) SQL makes more sense than NoSQL here for that reason, among the others I've listed.

Burn1ngBull3t,

A very interesting take ! They might have been a way to structure the data, but indeed it might make more sense to use a relational DB.

Thanks for the answer ^^

piotrsikora,
@piotrsikora@pol.social avatar

@Burn1ngBull3t There are many transaction in Fedi, that why Mastodon and Kbin use SQL for standard operation. But in future maybe will be some place to something like Elasticsearch for full text search

Burn1ngBull3t,

Thanks for the reply @piotrsikora ! I see, so relational db is pretty much mandatory as i understand, and NoSQL has its place as a complement instead of a replacement

Thanks

birlocke_,
birlocke_ avatar

I do believe that Mastodon supports Elasticsearch/Opensearch OOTB, so you're not wrong on NoSQLs being fediverse compatible.
Personally I haven't looked into the kbin stack so I can't give an opinion as to why psql is used over X.

Burn1ngBull3t,

As i understand, psql has pretty much it's use for all that relational data, so it makes sense using it. Also since it's Symfony, it might be easier to work on relational DB such as psql instead of NoSQL where it might not be an easy task.

And yeah if Mastodon supports Elasticseach/Opensearch, i suppose it would be a complement for the full text search as @piotrsikora said.

Thanks for the answer tho !

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • kbinMeta
  • InstantRegret
  • ngwrru68w68
  • everett
  • mdbf
  • modclub
  • rosin
  • khanakhh
  • DreamBathrooms
  • thenastyranch
  • magazineikmin
  • Youngstown
  • GTA5RPClips
  • slotface
  • kavyap
  • JUstTest
  • ethstaker
  • osvaldo12
  • normalnudes
  • tacticalgear
  • cisconetworking
  • cubers
  • Durango
  • Leos
  • anitta
  • tester
  • megavids
  • provamag3
  • lostlight
  • All magazines
    •