The Mali government taking control of the .ml tld probably has something to do with the fact that hundreds of thousands of US military emails have been accidentally sent to Mali by users who type .ml instead of .mil in the address field.
It’s just that the Fediverse now has enough global attention being paid to it that they’re probably actually cracking down on enforcement. Probably something under the “Insults” or “Racism” content policy, since those are the most vague and poorly defined and highly likely to be “obvious” primarily to the country who is operating them, Mali.
This is why we host our instance on a .org. Honestly another huge blow for Lemmy. It doesn’t really inspire confidence in the platform. Hopefully after enough time passes smaller instances like us and the bigger ones left will have help up a good track record to inspire confidence again.
It’s not a huge blow for Lemmy, I barely notice I’m not on Reddit anymore. I realize there are small niche subs that they don’t have here, but that’s not where Reddit makes their money anyway. People keep trying to ddos Lemmy, that alone should tell you that it’s a threat.
Well I think some instances like lemmy.ml interpreted the domain as marxist-leninist, but over the whole web this will be the minority. Many websites covering machine learning use(d) that domain.
this is why instances should be abstracted away as underlying infrastructure and the users don’t have to think about “instances”. accounts and communities are replicated across servers.
This was my thought as well. Before learning more about the fediverse, I thought things are distributed and are replicated across servers (much like how distributed storage and computing works). But apparently they’re not. You still have to choose which instance you want to use as your “home”, and your data and your contents stays in your home. Others get to look at your profile and contents thanks to ActivityPub.
I understand the needs for multiple instances (i.e., preferences for moderating concents, governance, etc.) But shouldn’t the users and the user generated contents (arguably fediverse’s valuable resources) should be safe-guarded by having redundancies in place across multiple instances?
I thought things are distributed and are replicated across servers (much like how distributed storage and computing works)
yes, exactly! when you use the internet, you don’t manually choose which ISPs to route through. you can pick which DNS servers to use but you don’t have to. when you use youtube, netflix, or facebook, you don’t choose which CDNs to use.
There are a few technical problems with that. First of all, the cost of each instance would become quickly unbearable since everyone has all the duplicated data.
Second problem, a malign entity could just come, create its own instance, spam everything and everyone with ads or whatever and suddenly every instance is full of that stuff. Also, how do you handle defederating in that case?
What has been proposed before instead was to make some kind of mega communities that gather all posts from communities with the same name across instances
everyone does not have all the duplicated data. they only have the data that they need – the data requested by a user who happens to be using some instance.
handling defederating is a good point. there could be malicious nodes that would be damaging to the network. i suppose there could be a community-mainted ledger of known malicious nodes (similar to minecraft usernames of known hackers), and the admins of the servers would maintain a blacklist. (obviously you configure that your instance’s blacklist would be automatically synced with this ledger)
the mega community idea could be good. where is this being discussed?
This is not the solution! Being able to pick a server to trust your data and content moderation with is a feature, not a bug.
What we do have to do is make this feature more resilient and easier to use. Like adding the ability to easily transfer accounts and communities between instances, or even change the domain name of an entire instance.
no, you’re misunderstanding. that shouldn’t be how it works. there shouldn’t be any difference between the software on each instance such that it make your data insecure. this is how bitcoin works. this is why anyone can spin up a bitcoin instance and have it start contributing to the bitcoin blockchain and you as a user don’t have to “trust” that particular node. trust is built into the distributed software architecture. you don’t “choose” a set of bitcoin nodes. you don’t “choose” your CDN or DNS servers.
Cryptocurrencies and social platforms are completely different beasts. In crypto I want no moderation/censorship, I want anonymity, and there is a payout system so nodes can compete for something. This is all different when building a social network, so you can’t just use the same architecture. Building social structures and trust is desirable in a public forum, not something you want to get rid of.
This is all different when building a social network
wait you want censorship in a social network? also, the architecture i’m describing does not do away with moderation and social structure. what about it makes you think that to be the case?
Of course! Moderation is censorship. There is certain content I don’t want to see, and I don’t want to have to filter it myself so I join a community of seemingly likeminded people who censor content based on rules I generally agree with. They ban users who break the rules, keep spambots out, block malicious instances and so on, and if they are doing their job right then it builds trust and attracts more people.
what about it makes you think that to be the case?
Because you want to strip all that out and abstract it away. Who do you think would do the moderating and spam blocking? Who aggregates posts from all over the world and presents a sorted list to a user on their smartphone? It would be the wild west with users having to do everything themselves. I know it’s tempting to think about building a Fediverse without instances, but afaik you need these social structures for the system to work.
Crypto for example only works because you can define the rules mathematically beforehand, and then hand out money for computers to check them. That’s just not possible with a public forum, at least not yet imo.
i do not want to strip out the functionality of communities having mods that moderate the discourse and ban malicious users etc. it sounds like you misunderstood what i was proposing.
you connect to some lemmy instance on your web browser
the client application (lemmy web app) authenticates your login credentials by first checking its own user database, if it doesn’t find you (which it should because by default you’d be connecting to an instance that you’ve already used, and if done through a mobile app for example it would automatically find the best instance to use by lowest latency), it send out a message to the nodes(instances) that it knows about, searching for your user, recursively, when found, sent back and stored in each node that was part of the searching. (there’d be some threshold of tree depth so the unsuccessful branches don’t keep going forever, and some other algorithmic details to prevent redundant network activity)
you navigate to your subscribed communities feed, lemmy shows you the posts that are already on the node that you are directly connected to, then asynchronously sends out a request to the surrounding nodes to pull more posts from those communities, recursively reaching out to adjacent nodes, again avoiding repeatedly hitting the same node via algorithmic details which we can discuss further if you wish, sending back the info up the tree to your primary node. now a bunch of servers have duplicated community data, like a distributed storage system, but you, the user, don’t know about all that stuff that just happened behind the scenes. your GUI is updated accordingly
now you can interact with these posts, make new posts, and each interaction will be sent out to all the relevant nodes in a reverse process.
another user on the other end can visit some community that you just posted to, and a request will again be propagated through the network, but starting from his node, and eventually reaching some node that has your new post.
the advantages of this:
if a node goes down, not all of the community and user data is lost, because its neighbor nodes have replicated the data
if i am hosting a node, and have limited bandwidth and storage, i can specify limits so that my network is not unintentionally DoSed. so this implies that when the prior-described processes are occurring, some instances will not store the data they are pushing through, which is fine, and one of the intended features of this distributed architecture
similar to previous point, each instance can have a whitelist or blacklist of communities (for either storage and/or data passing), defined by the admin, if he/she wishes to tailor the content for example to keep it related to content they are interested in rather than being forced to serve everyone on the network. it’s like if someone wants to help a little bit but they don’t have all the bandwidth and storage in the world, they can, instead of having to handle traffic for a bunch of irrelevant-to-them communities.
There is so much wrong with this that I don’t even know where to begin.
I don’t intent to be rude, but this is just not how you build a decentralized/distributed system. The network would grind to a halt if every user app had to search recursively through a portion of the network, and aggregate & rank posts by itself. Aggregate values (communities, votes and so on) would never be right, because you’d never be able to acually gather all events for a particular entity in time. This might work in a local network of 10 nodes, but not on a global scale.
On top, who would pay for those nodes you are querying? There is no relationship between the users and the nodes, so why would anyone just run a node for others or be willing to pay anyone else in this scenario? Servers cost money and stuff. And your spam filtering and moderation solution would be the exact same as with instances, so nothing is gained here.
Maybe have a look at the Session messenger and their Oxen network. They go to great length to make sure the work is equally distributed among nodes and they are compensated fairly. This doesn’t just happen magically by itself, and there are many bad actors who will try to exploit any weakness they can find.
So I just think it’s impossible to create something like lemmy in an anonymous way, because content moderation is a human decision. There is no one correct mathematical solution, and I also can’t send some kind of filter query to a server to do it for me. All I can do is read the general rules that another human being has wrote up, subscribe to their moderation “service”, see how they are doing, and decide to stay or switch to another.
Similarly, if I don’t want to aggregate all the posts in the world by myself (as you are suggesting), then I’ll have to fine someone to do it for me, and somehow pay that someone for their service. This part is actually kind of solvable (again look at Session), but it is not straight forward at all! It would involve crypto currencies, mining/staking, and some kind of client-side monetization. For this part I think trusted instances are just a much better solution, because we are building a social structure here anyway.
ok, you make good points, but i feel like the algorithm could work to not have the system grind to a halt. i’d have to look at other examples where this has been done. but maybe i am overly-optimistic and it’s not possible.
who would pay for those nodes you are querying
the people who are already running nodes, like lemmy.world, lemmy.ml, me, etc. i run some services on my home server that i let anyone use, because i have the hardware and the bandwidth to be able to afford it. there are enough people who have the necessary hardware and bandwidth to contribute to it at minimal detriment to them. it’s already an open-source project where people volunteer their time to code it.
i’ll read up on oxen network.
in an anonymous way
wait who said anything about anonymous? what are talking about being anonymous? there would still be user accounts.
if I don’t want to aggregate all the posts in the world by myself (as you are suggesting), then I’ll have to fine someone to do it for me
this is already what is done, except that the data is not stored in a replicated and distributed manor. you get all the posts in the world of a community of an instance. it is one server, with all the data stored on its harddrive, like a traditional website. in what i’m proposing, this is also what would happen in many cases, because the thing wouldn’t requery the entire network every time you request posts, there would be a time threshold, like how posts are cached on your local mobile device for most social media apps. posts would be cached on the server.
now, yes, this architecture would in fact result in more network traffic occurring between each and every node, as they receive updates about events on other nodes. so that would be extra burden upon the hosts. but i believe it is something we can work through.
the tankie instance or the nutballs on the fascist instance
here you reveal a conceptual misunderstanding, or rather, a part of the lemmy architecture which i disagree with. there shouldn’t be a concept of a “interest X instance” etc. it should be similar to a distributed storage model. so the concept of a community is not per-instance, it’s just an abstract thing that exists in conceptual space.
I’m aware of how you think it should work (Usenet, basically) but how does moderation actually work on Lemmy? Can someone be banned from a sublemmy on one instance and not banned from it on another?
you already share water with them though. how is this any different? more seriously though, you already share internet infrastructure with them. the packets you just sent to make that comment could have been sandwiched between a “tankie” and a “fascist nutball”. that’s just the way it is man, there have always been crazy humans.
When I was talking my cyber security / ethical hacking class, we learned how to do zone transfer. The concept never stuck and I basically “copy” from my friend. So what exactly is a DNS Zone Transfer?
Friday I was doing a zone transfer! What are the odds?
A zone transfer is like moving houses, except for an authoritative zone.
In DNS, we have what’s called an authoritative zone. That means the device hosting the “resource records” (all the data that DNS passes around) is the “ultimate” answer. I.e, it’s not cached data. It’s not a hosts file. It’s not a recursive answer. It’s the real deal.
When you want to move the authoritative zone to another server, you do a “zone transfer” that means the new server will copy all the resource records over TCP from current authoritative zone. The reason you may want to do this instead of manually hand-jamming it is that many large organizations have, sometimes, hundreds of resource records (last month I coordinated a zone transfer that was over 1000 records!).
Why would a hacker want to conduct a zone transfer? In otherwords, what is the utility or usefulness of a zone transfer for a hacker (black or white hat)?
If you initiate a zone transfer, you can now claim to be authoritative for a zone. That means you can be a ‘bad actor’ DNS server that serves fake records. In practice, this means that you can redirect people to an attack site.
Let’s say you’re Joe the Random Internet User and you want to go to lemmy.world This is what happens in a non-attack (we’re skipping caching & non-authoritative answers for brevity):
You type “lemmy.world” into your browser
Your computer initiates a stub resolution for lemmy.world. (the trailing dot here isn’t a period. It’s the “true” FQDN)
Computer looks at hosts file and doesn’t see anything
DNS packets are sent to your configured DNS server. If you don’t have one configured, DHCP already configured it for you
Your DNS server performs a recursive search for world by asking the root zone where the “world” Name Serer is
root zone resolves world as:
world. 3600 IN NS v0n0.nic.world.
world. 3600 IN NS v0n1.nic.world.
world. 3600 IN NS v0n2.nic.world.
world. 3600 IN NS v0n3.nic.world.
world. 3600 IN NS v2n0.nic.world.
world. 3600 IN NS v2n1.nic.world.
Your DNS server reaches out to one of those Name Server’s (That’s what the NS record is for) and asks it where “lemmy” is
world Name Server responds with:
lemmy.world. 300 IN A 172.67.218.212
lemmy.world. 300 IN A 104.21.53.208
Your DNS server contacts your computer and serves it those IP addresses. (A record’s are domain name to IP Address)
Now lets say there’s a DNS spoof attack:
Before the “world” server can get back to your DNS server, the hackers server interjects with it’s own authoritative claim that lemmy is here:
lemmy.world. 300 IN A [attack site IP]
Your DNS server contacts your computer and serves it that IP address. Your computer then contacts the attack site and you get a virus.
Could users set a temporary entry in their hosts file pointing the .ml domains to public IPs in order to regain access to their account if they needed to?
Can Lemmy federate to an IP address directly or will the settings only accept an fqdn?
Will a Lemmy instance work behind a reverse proxy.
Yes. Unless there’s some kind of crazy domain-level hi-jinks involved with Lemmy (I am not versed in Lemmy), pointing directly to the IP will work if you bypass it by spoofing your DNS (Hosts file, for example).
I don’t know how Lemmy federation works, sorry :(
See #2
Sorry that I couldn’t answer more of your questions.
There are several problem with this including total lack of SSL without the proper cert for that other domain, also Lemmy.ml’s IP seems to be running a reverse proxy so the internal IP that we would want to connect to is not visible to the world this is common for web security, the owners must set allowed domains and ports in their config file.
If none of that was a problem Lemmy itself does not do well with changing domains, as highlighted here: lemmy.nrd.li/comment/190200
So, how some companies get right to sell TLDs? Can I start selling TLDs nowdays? It’s just that they were there first and get all top level domains and now we have to pay for it?
Companies don’t/can’t sell TLD’s. Only IANA can decide those. When the internet first started, .org, .net, .com etc. were handed out to non-profit organizations and the costs were purely to keep the servers running. Eventually though, when IANA decided to hand out country codes like .io (Indian Ocean), .cat (Catalonia) or .tv (Tuvalu), those countries rent their “desirable” names to private organizations that sell domain registrations for lots of money. In 2013, IANA decided to enact the gTLD auctions to help raise more money. Basically, if you wanted to (and had a lot of money & DNS engineers on staff), you could register any TLD you want provided you were willing to make a large donation to IANA. If someone else wanted it, they had to go into an action war over it. That’s how we ended up with things like .party or .sport or .world cough Now-a-days, if you want a TLD, you’d have to convince IANA to give you one… But good luck with that. They won’t give you one unless you’re some major corporation that can actually handle it. They also just don’t give them out. Usually it’s only when they really feel like more TLD’s are needed. It’s a very serious responsibility and mismanagement could accidentally DDOS a DNS root zone & impact the internet.
Well that sounds like my dream job, unfortunately this issue in particular is more of a Lemmy problem, not a DNS problem. See: lemmy.nrd.li/comment/190200 for the explanation of why you cant just transfer domains with Lemmy.
Also, if you’re genuinely interested in this field, first you should enter the world of enterprise network engineering. Get Security +, CCNA, and PCNSA. With those certs in hand (and knowledge in your brain), apply to jobs as a network support engineer. Do the work for a few years. Learn BIND. Learn Infoblox. Focus on learning DHCP and subnetting. Learn DNSSEC & IPv6. Experiment with a Pi Hole. Set up a home lab. Apply to jobs with DNS. Start living the good life. This takes about 10 years if you learn fast and are good at interviews.
I only just now saw this post, the last month i have already been going all out to learn everything that i need for my Security+ (then CySec+) i have a 30hr video course im part way thorugh, and ive set up a few VMs with various servers like OWASP Security Shepherd and Dam Vunurable Web App for some more hands on experience as well as testing on my personal production Nextcloud and Jellyfin servers and ive been having alot of fun with it all, i think im pretty solid with DHCP and subnetting already through my home networking adventures. I will look into each of those other Certs and each thing you mention to learn thank you! Ive been deep into various Linux systems since about 2008 and im hoping to leverage that as much as i can(although its left me with a lack of modern Windows experience).
Thank you so much for all the tips! I feel some good things coming as im getting into this as work.
To answer your other question: most likely, www.cakefarts.com is now accessible from cakefarts.com for one of three reasons:
Your web browser automatically checks the A record “www” if “cakefarts.com” doesn’t have an A record. A records are the records in a DNS server that says "this domain goes here"
The site cakefarts.com put their website on cakefarts.com and placed a CNAME record called “www” that points to cakefarts.com
For the ‘record’, www.cakefarts.comlly common record name. There’s nothing special about it. You could have dudebro.cakefarts.com or www.cakefarts.com.com. It’s up to the domain owner.
The “.com” and “.org” and all other Top Level Domains are owned/controlled by some organization.
Com and org are your original TLDs, so since they were around first you see them everywhere. At some point countries got their own TLDs so Mali got “ml” for example but Tuvalu got “tv”. (Yes, technically “.tv” has nothing to do with television.) And a few years back there was open bidding for a bunch of new TLDs which is where “.sport” or “.dentist” come from.
Anyone some entity owns/controls them and then can sell any word or domain under it. So if you want “greatgatsby.com” you have to talk to the “.com” owners. If you want “greatgatsby.sport” you talk to the “.sport” owners. Usually there is another company or agreement that groups these together so you can manage all your domains in one place.
So anyways now you own a domain like “greatgatsby.sport”, what do you want to host? Mail at “mail.greatgatsby.sport”? A website at world wide web aka “www.greatgatsby.sport”? Up to you.
Over time, largely by convention “www” became where you put your website.
So say I want a “.travel”, who actually makes and sells these? Is it a private company? A country? An independent entity who’s sole purpose it is to keep domains and the interwebs alive?
The last one, ICANN is the name of the organization. It’s reasonable to argue they are actually the first one. Also they are based in the US, so technically the country answer also apply. HOWEVER they are suppose to be independent.
Also since you want “.travel” that’s a common enough word that it is probably already owned by an entity, so you would probably have to buy it from them.
However let’s say you wanted “.tchotchony” which I feel confident saying doesn’t exist yet. As far as I know ICANN is not regularly taking applications for new TLDs, so you probably can’t have it. Although realistically if you have enough money, you can.
Well, it’s not just a money issue. There’s also the “are you knowledgeable, responsible, and have DNS engineers on staff” problem. If you own your own TLD, it means you can talk directly to the root zone. You could theoretically DDOS the root zone servers and cause them to crash. They would, of course, just revoke your TLD permanently & it wouldn’t really cause any noticeable disruption to the rest of the internet. You could also allow attack domains or shady websites. Maybe it could be used to pretend to be another site. Imagine owning “.conn” that would be a premium attack site TLD because it looks like “com”. There’s lots of other issues too.
They don’t know unless the DNS server tells them. For example, a very popular webhost Akamai uses a complex DNS + web hosting suite (DNS edgesuit to be exact) to send that type of data to the web servers. It can also allow for many many other features.
We had a situation at a shared space here where an OpenWRT client device accidentally somehow managed to announce itself into the network in a way that its v6 local link address (fe80::) got inserted into /etc/resolv.conf as a third DNS option (with the first two being the ones from DHCP) and then served incorrect records when queried. What mechanism is that and were the engineers who designed that feature on drugs? Also, how can I tell my Linux system to not accept such announcements?
Because it’s the least-likely position to be staffed by a company. It’s the “least important” person to have… until it breaks. Often a company relies on routing-switching engineers to do DNS instead of hiring a dedicated DDI engineer (DNS, DHCP, IPAM). It saves money in the short term, but when shit hits the fan… no one knows how to fix it because DNS is really easy until it’s not. DNS is super simple at a basic level. But it goes way deeper than most people realize.
Because DNS is the user-facing part of the whole system. There is plenty of trouble with everything else, but you usually don’t see that as a user. Also it’s a hierarchical system with big providers/governments giving and taking names as they see fit, so there is always the possibility to get screwed.
How does the TLD get reclaimed? I’m assuming whoever was previously the “owner” of the .ml tld was on board and Mali didn’t just come along and snatch it away?
So here’s the thing about TLD’s, ownership of them is determined by IANA (Internet Assigned Numbers Authority). They’re basically my career’s gods. If they tell me to jump, I ask “how high”. They control the DNS root zone. Effectively, that’s the actual top-level of ALL domains. If they decide to remove a TLD or reassign it, all you can do is lodge a complaint straight to their shredder. They’re owned and operated by ICANN, a non-profit organization.
Back in 2013, Mali allowed a private Netherlands company to “manage” (rent) their TLD, .ML Recently, that company (Freenom) got sued by Meta. Even though I don’t really like Meta, as a network engineer, I don’t like Freenom even more. They turn a blind eye to bad actors on the internet, refuse to investigate hackers/scammers/DDOSers, and generally refuse to play ball. They are a huge pain in the ass. Due to the lawsuit, IANA reassigned ML to Mali since they asked for it. At the end of the day you “cant” sell a country-level TLD. Mali was renting it to Freenom under the table. This happens a lot and IANA usually just looks the other way. .io for example is the freakin’ Indian Ocean.
So yeah, Mali didn’t “snatch” it. They just asked IANA to reassign it and there isn’t shit Freenom can do about it since they never “really” owned it in the first place.
Hi! When DNS servers are launched, they have to be purchased, correct? So in this case, did Mali file for the domain to be reclaimed somehow? Do you have an idea how that might work?
I can answer this. The organization that says mali owns .ml gives the ownership country a lot of sway.
So if the country of mali were to reach out formally to the organization and say “hey this domain violates our laws” they would take that very seriously and then work with the registrar & authoritative nameserver owner to handle the situation.
I’m sure this isn’t 100% accurate but 90-95 based on my work in a web hosting company
It’s a little stronger than that. The country gets the final say on where the root zones point to when it comes to their assigned country code. Many countries employ private organizations to handle their TLD. They aren’t supposed to be paid for that though. (But it 1000% happens under the table)
I’m new to the fediverse and not sure how it works just yet. Can someone help me understand? My account was created on Lemmy.ml, will it no longer work and I’ll have to make another?
I would make a lemmy world account personally, it’s tough to say what will happen long term with .ml domains. Even if your account still works, it may by hampered getting posts from the rest of the fediverse. Worst case scenario you have account on two popular lemmy instances.
Are you aware that ml in lemmy.ml stands for marxism-leninism and that the admins of your instance don’t support any critique of the chinese government? I’m asking because I think a lot of new users chose lemmy.ml randomly - mostly because it was big - and if they knew this, many of them would have chosen differently.
I had no clue. I was just tired of Reddit and when looking into Lemmy the .ml one came up first in the search. Guess I get to make a new one somewhere else.
They chose .ml because it was free. They don’t delete comments just because they don’t agree with them. I don’t get why some people feel the need to spread lies about the lemmy devs/lemmy.ml admins just for being in favour of communism.
The claim about .ml meaning that is about as dumb as people saying AC/DC means anti-christ devil-child…
But lemmy.ml absolutely removed posts criticizing china. They also made their beliefs very clear. It wasn’t lemmygrad… but they would absolutely censor anti-china rhetoric, and had many brainwashed or Chinese troll accounts.
They even started censoring certain words on lemmy.ml, including “bitch.” I’m not okay with that…
I think the majority of users were normal people who randomly ended up there. But just because this one claim is silly, it doesn’t detract from very real issues that existed there.
I’m not sure that’s true. Couldn’t it just automatically broadcast your server’s current correct IP to all servers it federates with each time it (the IP) changes (and if a server fails to find a federated server by the most recent IP in its records, have it query other federated servers for a more up to date IP.)
Add comment