Loss of niche and technical information

Hey All,

Tl;Dr: Save Reddit technical expertise before it’s lost to the sands of time and corporate shutdown.

Just wanted to get a discussion started on what would be, in my opinion, the greatest loss that we, as a society, would experience with the loss of Reddit. Proposed solutions welcome. And please don’t take this as me supporting Reddit in any way shape or form.

For years Reddit acted as the Internet’s foremost discussion board. As part of that it became host to a slew of subreddits which house niche information and technical expertise. I can’t tell you how often I’ve struggled to find a solution to a particular problem only to stumble across it on a years old reddit thread.

That information is, frankly, invaluable. Reddit may not be tailored to providing technical advice akin to the likes of stack overflow but, nevertheless, it is home to some of the hardest to find answers. If Reddit was to disappear tomorrow, so would that information.

As such I think that information should be treated as a goldmine, and just like a goldmine, excavated. If Lemmy is to play host to the great Reddit migration then it might well play host to these valuable tidbits. Exactly how such an excavation could be done without a blanket copy of all data on Reddit I don’t know. But I think it’s definitely something worth discussing and promoting amid Reddits recent mishaps. Who knows what the future holds for the site? But it’s downfall shouldn’t lead to the loss of decades worth of troubleshooting efforts and technical expertise.

density, (edited )
density avatar

I think what you are looking for is the Archiveteam's Reddit project.

Archiveteam (AT) is a group which according to their website,

is in no way affiliated with the fine folks at ARCHIVE.ORG

However, the goals and philosophy of archive.org, aka The Wayback Machine, aka The Internet Archive do have significant overlap with AT. AT is coordinated by a staff member of archive.org, and the products of their work are typically donated to archive.org.

They do missions to save particular collections of internets which are under imminent or generalized threat of deletion. One way to participate is by installing their custom Warrior VM software on your computer and it will use your home internet connection to pretend to be a user and systematically crawl/save the material in a coordinated fashion which evades detection. There can be other tasks if you can't or don't want to run that software. For example if the Warriors are triggering captchas, they can forward the captchas to users who sit around solving them. So you can solve captchas on other people's computers so those computers can proceed unattended.

Here is the tracker showing the moment to moment progress. At the moment it display 13.58 billion items weighing in at 3.06 petabytes (3,060,000 GB).

Here is a reddit post from a month ago going over the project.

You can find more comprehensive info on their website. They coordinate via chat.

I would also like to add that while I share the sentiment about technical information, and I have already been stumped with some issues I know I could have solved with reddit, we shouldn't keep a narrow perspective. The nerd stuff is the most resilient because we are the people most comfortable with technology who a) have a lot of other established forums, and b) have the best ability to adapt to a new forum and regroup. I am really deeply sad about all the non-techy groups who I fear will now be dissolved forever. All that knowledge.... is much more gone. I can find another place to resolve issues with linux or ask about mechanical keyboards. Much of the rest of it I have no idea without resorting to instagram or tiktok which I think are revolved around fundamentally different relationships and communications.

lvxferre,
@lvxferre@lemmy.ml avatar

The first step would be to discourage people from making the gold mine even deeper, as it makes the problem worse. They should be adding gold elsewhere - preferably a decentralised, widely accessible source of information, more resilient against information loss and easier to sort out. From the top of my head, a wiki is perhaps the best approach.

Note that I don’t recommend Lemmy as a repository of this info. Lemmy shares a bunch of cons with Reddit in this aspect - if the server is gone, there’s a good chance that the info is gone too. And forums in general are prone to add too much noise around the info, as they’re made for users to interact with each other.

With that step ready, people could be migrating the info already in Reddit to the new platform, and sorting it out from all the noise (all those posts and comments that are not information). Easier said than done because this would need to be done manually. But it would be basically the online equivalent of mining that gold and transforming it into ingots, because a small gold vein inside a big chunk of rock is useless.

I believe that this should be easier to do for content up to March/2023, since it’s archived through Pushshift.

DisappointingIntro,

I had a brief read through the article you linked. I’m frankly shocked at how relevant it is. I never considered how widespread the problem is in reality. It goes so much further than just reddit. An extremely interesting article for anybody curious, I’ll be dedicating more time to reading it in its entirety this weekend.

Thanks very much for linking it!

HughJanus,

Twitter killed their API entirely and with zero notice (among other horrific things) and their traffic has only fallen by 5%. Reddit isn’t going anywhere. People can’t be bothered to move.

DisappointingIntro,

That’s interesting - can I ask where you’re seeing those stats?

HughJanus,

cnbc.com/…/twitter-traffic-is-nosediving-as-metas…

lvxferre,
@lvxferre@lemmy.ml avatar

On the very least Reddit is becoming a walled dumpster fire (“walled garden” is a bit too much). Even if the info stays there, I think that “we” (people in general) should be migrating it elsewhere.

Haui,
@Haui@discuss.tchncs.de avatar

Thanks for pointing that out. But as much as numbers don’t lie, the situation is very different on reddit, at least for the technical/niche forums on there.

A lot of subreddits that drove massive expertise to the place have shut down and moved to other places. It’s not about the api alone.

It’s about giving the finger to the foss community and other dick moves against „the people“. Twitter has been a dumpster fire before felon husk took over. Reddit not that much. As soon as 1%ers (in members) start moving, the floor is gonna shift in reddits corporate halls.

I moderate a couple subs there, one of those is top 5% or something. We‘re discussing moving atm. If moderators keep it low and organize, reddit is being hit hard.

Sanctus,
@Sanctus@lemmy.world avatar

Wouldn’t web archive already have a lot of these? And if not couldn’t it just be added to it? Its an already existing well known archive. Tho I’m not sure what that would do to internet searches.

DisappointingIntro,

It’s something I thought about. But as you said the exposure to search engines is key.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • snoocalypse@lemmy.ml
  • DreamBathrooms
  • mdbf
  • ethstaker
  • magazineikmin
  • GTA5RPClips
  • rosin
  • thenastyranch
  • Youngstown
  • osvaldo12
  • slotface
  • khanakhh
  • kavyap
  • InstantRegret
  • Durango
  • provamag3
  • everett
  • cisconetworking
  • Leos
  • normalnudes
  • cubers
  • modclub
  • ngwrru68w68
  • tacticalgear
  • megavids
  • anitta
  • tester
  • JUstTest
  • lostlight
  • All magazines
    •