We live in the future now. It is OK to use Unicode everywhere.
It seems bizarre to me that modern Internet services sometimes "forget" that there's a world outside the Anglosphere. Some people have the temerity to speak foreign languages! And some of those languages have accents on their letters!! Even worse, some don't use English letters at all!!!
So I built a small ActivityPub server which publishes content from an Actor called @你好@i18n.viii.fi - it is only a demo account, but it works!
Some ActivityPub clients report that they are able to follow it and receive messages from it. Others - like Mastodon - simply can't see anything from it. Take a look at the replies on Mastodon to see which services work. You can also see some of its posts on the Fediverse.
Well, what about them? ASCII has plenty of similar looking characters. I doubt most people would notice when a capital i is replaced by a lower L - and vice-versa. Similarly the kerning issue of an r and n looking like an m is well known. Are mixed language homographs more dangerous? I don't think so.
Well, what if they do? Maybe not being found by people who can't type your language is a feature, not a bug. But, anyway, clients can let users search for other people, or copy and paste their names.
It is up to a client to decide how they want to render text input. The "problems" of strange Unicode combinations are well known. This is not a hard computer-science problem.
I have no evidence for this. But I bet you'd get pretty frustrated if you had to switch keyboard just to type your own name, wouldn't you? In any case, why can't I have a username of @😉
I used to like the original FourSquare. The "mayor" stuff was a bit silly, and my friends never left that many reviews, but I loved being able to signal to my friends "I am at this cool museum" or "We're at this pub if you want to meet" or "Spending the day at the park".
So, is there a way to recreate that early Web 2.0 experience with open data and ActivityPub? Let's find out!
This quest is divided into two parts.
Get nearby "Points of Interest" (POI) from OpenStreetMap.
As you can see, it has highlighted some useful areas - a pharmacy and a pub. But it has ignored other useful locations - the train station and the park. It has also included some things that we may not want - bike parking and a taxi rank.
What API call is needed to get useful locations of of OverPass?
It's possible to specify the type of thing to find using nw["amenity"="restaurant"]; - but adding every single type of thing would quickly end up with a very large query containing hundreds of types.
It is also possible to exclude specific types of places. This retrieves all amenities except for fast food joints:
Perhaps one solution is just to return everything and let the user decide if they want to check in to a telephone kiosk or a fire hydrant? That's a bit user-hostile.
That gets everything which has a name, but isn't a highway or railway or waterway or powerline. It isn't perfect - but it will do!
This is the query which will retrieve the 25 nearest things within 100 metres of a specific latitude and longitude. It includes the name and any other tags, the location, and the OSM ID.
It is worth noting that Mastodon doesn't (natively) support location - if you view my repost of that PixelFed post you'll see there's no location metadata attached. That's OK! It just means that the status needs to include human-readable data.
Building a web site which gets the user's location is easy.
Getting the data from OverPass should be straightforward.
Creating an ActivityPub server which can post geotagged notes into the Fediverse might be a little beyond my skillset! Some testing with Darius Kazemi's AP Glitch suggests this should work.
I've written an ActivityPub server which . That's all it does. It won't record favourites or reposts. There's no support for following other accounts or receiving replies. It cannot delete or update posts nor can it verify signatures. It doesn't have a database or any storage beyond flat files.
But it will happily send messages and allow itself to be followed.
This shows that it is totally possible to broadcast fully-featured ActivityPub messages to the Fediverse with minimal coding skills and modest resources.
I wanted to create a service a bit like FourSquare. For this, I needed an ActivityPub server which allows posting geotagged locations to the Fediverse.
I didn't want to install a fully-featured server with lots of complex parts. So I (foolishly) decided to write my own. I had a lot of trouble with HTTP Signatures. Because they are cursed and I cannot read documentation. But mostly the cursed thing.
Creating a minimum viable Mastodon instance can be done with half a dozen static files. That gets you an account that people can see. They can't follow it or receive any posts though.
I wanted to use PHP to build an interactive server. PHP is supported everywhere and is simple to deploy. Luckily, Robb Knight has written an excellent tutorial, so I ripped off his code and rewrote it for Symfony.
The structure is relatively straightforward.
/.well-known/webfinger is a static file which gives information about where to find details of the account.
/[username] is a static file which has the user's metadata, public key, and links to avatar images.
/following and /followers are also static files which say how many users are being followed / are following.
/posts/[GUID] a directory with JSON files saved to disk - each ones contains the published ActivityPub note.
/photos/ is a directory with any uploaded media in it.
/outbox is a list of all the posts which have been published.
/inbox is an external API endpoint. An ActivityPub server sends it a follow request, the endpoint then POSTs a cryptographically signed Accept message to the follower's inbox. The follower's inbox address is saved to disk.
/logs is a listing of all the messages received by the inbox.
/new is a password protected page which lets you write a message. This is then sent to...
/send is an internal API endpoint. It constructs an ActivityPub note, with attached location metadata, and POSTs it to each follower's inbox with a cryptographic signature.
That's it.
The front-end grabs my phone's geolocation and shows the 25 nearest places within 100 metres. One click and the page posts to the /send endpoint which then publishes a message saying I'm checked in. It is also possible to attach to the post a short message and a single photo with alt text.
There's no database. Posts are saved as JSON documents. Images are uploaded to a directory. It is single-user, so there is no account management.
I've raised an issue on Mastodon to see if they can support showing locations in posts. Hopefully, one day, they'll allow adding locations and then I can shut this down.
The code needs tidying up - it is very much a scratch-my-own-itch development. Probably riddled with bugs and security holes.
You receive a call on your phone. The polite call centre worker on the line asks for you by name, and gives the name of your bank. They say they're calling from your bank's fraud department.
"Yeah, right!" You think. Obvious scam, isn't it? You tell the caller to do unmentionable things to a goat. They sigh.
"I can assure you I'm calling from Chase bank. I understand you're sceptical. I'll send a push notification through the app so you can see this is a genuine call."
Your phone buzzes. You tap the notification and this pops up on screen:
This is obviously a genuine caller! This is a genuine pop-up, from the genuine app, which is protected by your genuine fingerprint. You tap the "Yes" button.
Why wouldn't you? The caller knows your name and bank and they have sent you an in-app notification. Surely that can only be done by the bank. Right?
Right!
This is a genuine notification. It was sent by the bank.
You proceed to do as the fraud department asks. You give them more details. You move your money into a safe account. You're told you'll hear from them in the morning.
This is reasonably sophisticated, and it is easy to see why people fall for it.
The scammer calls you up. They keep you on the phone while...
The scammer's accomplice calls your bank. They pretend to be you. So...
The bank sends you an in-app alert.
You confirm the alert.
The scammer on the phone to your bank now has control of your account.
Look closer at what that pop is actually asking you to confirm.
We need to check it is you on the phone to us.
It isn't saying "This is us calling you - it is quite the opposite!
This pop-up is a security disaster. It should say something like:
Did you call us?
If someone has called you claiming to be from us hang up now
[Yes, I am calling Chase] - [No, someone called me]
I dare say most people would fall for this. Oh, not you! You're far too clever and sceptical. You'd hang up and call the number on your card. You'd spend a terrifying 30 minute wait on hold to the fraud department, while hoping fraudsters haven't already drained your account.
But even if you were constantly packet sniffing the Internet connection on your phone, you'd see that this was a genuine pop-up from your genuine app. Would that bypass your defences? I reckon so.
Criminals are getting increasingly good at this. Banks are letting down customers by having vaguely worded security pop-up which they know their customers don't read properly.
And, yes, customers can sometimes be a little gullible. But it is hard to be constantly on the defensive.
Facebook Meta - like many other tech titans - has institutional Shiny Object Syndrome. It goes something like this:
Launch a product to great fanfare
Spend a few years hyping it as ✨the future✨
Stop answering emails and pull requests
If you're lucky, announce that the product is abandoned but, more likely, just forget about it.
Open Graph Protocol (OGP) is one of those products. The value-proposition is simple.
It's hard for computers to pick out the main headline, image, and other data from a complex web page.
Therefore, let's encourage websites to include metadata which tells our services what they should look at!
OGP works pretty well! When you share a link on Facebook, or Twitter, or Telegram - those services load the website in the background, look for OGP metadata, and display a friendly snippet.
Facebook Meta were the driving force behind OGP - and have now left it to fester.
And, that might be fine. Facebook Meta are a small company with limited resources. They can't afford to fund standards work indefinitely. And, anyway, OGP is complete, right? It has all the tags that anyone could ever possibly want. Why does it need any improving?
This is annoying for developers. Now we have to write multiple different bits of metadata if we want our links to be supported on all platforms.
Standards work is never "finished". Developers want to add new features. Users want to interact with new forms of content.
Tomorrow someone is going to invent a way to share smells over the Internet. How does that get represented in an Open Graph Protocol compliant manner?
<meta property="twitter:olfactory" content="C₃H₆S"> or <meta property="facebook:nose" content="InChIKey/MWOOGOJBHIARFG-UHFFFAOYSA-N"> or <meta property="og:smell" content="pumpkin spice"> or...
We know from bitter experience that having several mutually incompatible ways to implement something is a nightmare for developers and provides a poor user-experience.
So we create standards bodies. They're not perfect, but a group of interested folks can do the hard work to try and satisfy oppositional stakeholders.
This is my plea to Facebook Meta. If you're no longer interested in improving OGP, OK. You do you. But hand it over to people who want to keep this going. Maybe it's the W3C, or IndieWeb, or Schema.org or someone. Hell, I'm not busy, I'll take it on.
Microsoft loves you and wants to protect you. So every time you receive an email with a link in it, Microsoft Outlook helpfully rewrites it so that it goes through their "safelinks" system.
Safelinks allow your administrator, or someone at Microsoft, to stop you visiting a link which is malicious or suspicious. Rather than going to example.com, your link now goes to safelinks.protection.outlook.com/?url=example.com.
Hurrah! If you accidentally click on a naughty link you won't cause chaos and ructions.
Except, there's a tiny problem. People like to copy and paste links that they receive. Someone sends an email which says "here's the link to that report you asked for" which then gets copied into a document or a web page.
That forces everyone who visits that link to go through Microsoft's proxy. That might protect users if a link later becomes suspicious. But, more likely, it will be used in analytics to further profile users who click on links. It also undermines a user's ability to see the final destination of a link unless they can manually URl-decode content in their head.
Look, I get why people do this. They copy a link from an email, paste it in, click it, and it works. No one writes raw HTML by hand, nor should they have to. Our WYSIWYG tools work really well and hide all the mumbo-jumbo. Copy editors look at text; not hypertext. It's only nerds like me who hover over a link before clicking on it.
Perhaps I should stop worrying? Perhaps it is OK that Microsoft intercepts the clicks from people all around the world? Perhaps they can competently run a proxy which detects and blocks inappropriate content? Perhaps they won't ever abuse that facility?
Here's my prediction. In the next five or so years, Microsoft is going to accidentally shut off *.safelinks.protection.outlook.com and a million copy-and-pasted links across the web are going to break.
Either way, if you work in digital publishing, please make sure that your links point directly to the content that you want; not to Microsoft's safelinks service.
I've responsibly disclosed a small security issue with Mastodon (GHSA-8982-p7pm-7mqw). It allows a sufficiently determined attacker to use any Mastodon instance to redirect unwary users to a malicious site.
It is possible to craft a URl which will redirect any visitor who isn't logged in. Attackers can use this as an open redirect for phishing, spam, and other attacks.
This will likely be fixed by #26917. But, in the meantime, administrators of Mastodon instances should be aware that their site could be used as an open redirect.
If you do spot any accounts which appear to be dodgy, admins can either block the account or the entire domain.
Here's how it works - which involves some necessary background detail.
I am user @edent on Mastodon.social. I can send you a URl of https://Mastodon.Social/@edent and you will see my profile. Nice!
But there are lots of Fediverse servers out there. For example, I run a little bot called @colours on the BotsIn.Space instance. Its URl is https://BotsIn.Space/@colours - simple.
But what happens if I am viewing the Colours bot while on Mastodon.Social?
The interface shows https://Mastodon.Social/@colours@BotsIn.Space - if you are logged in to Mastodon.Social, you will see the colours account, you can follow it, reply to it, and interact with it as though it were a user on your home instance.
In addition, Mastodon ignores the @username when it sees a local status ID which references an external status. For example, both of these URls will go to the same place:
The first is spam evasion. Email out a link to mastodon.social/@user@buy_illegal_puppies.com and it might skip spam filters, or confuse the user about the true destination.
The second is phishing. Is a user going to notice that they've been silently redirected to nnast0d0n.social? Stick up a convincing "Please log in again" page and you can steal their credentials.
Mastodon will check that account exists, and then redirect a non-logged-in user to the "profile-page" of an account that it finds.
So a malicious user can create a WebFinger at evil.com, then send out links to mastodon.example/@SexyFunTimes@evil.com, and have users instantly redirected to their site.
Most ActivityPub instances won't do this unless they've already seen the user being referenced. This can be achieved by sending a private message to a user on that server which mentions the redirection account.
Given that it is sensible to redirect users to an account's home instance, I think there's really only one way to solve this. An annoying interstitial.
You are leaving XYZ.social. We do not control the page Illegal_Ivory_Smuggling.com. If you are sure you want to proceed, click here. Do not share your username and password with 3rd party sites etc etc etc.
I reported this to Mastodon on 2023-09-20. Apparently a number of other people have also reported it. While they work on how to fix the problem, I thought it was sensible to let people know that this attack was possible.
Nuclear power was sold to the world as a safe, clean, and economically viable source of electricity. We were told that it would be "too cheap to meter"1. Even the most ardent proponent of nuclear power will have to admit that hasn't come to pass. Construction costs for nuclear power stations are dwarfed only by their decommissioning costs. Yes, politics and regulation conspire to increase the price - but nuclear hasn't made electricity particularly cheap. Indeed, we mostly seem to be paying more than ever for our power.
Well, not quite.
On Christmas Eve, my electricity company emailed me to say that I would have several hours of free electricity. They would charge me £0.00 per kWh. More than that, at a few specific times they would pay me for my electricity use!
Most factories and heavy industrial plants weren't running the day before Christmas. UK power usage spikes when everyone boils a kettle at the end of a football match or other similar event - but there was nothing so momentous happening at 3AM. So supply outstripped demand.
Anyone with a smart-meter could have been paid to charge their car, run their tumble dryer, or stay up until the wee hours playing on their console.
It wasn't mined uranium which gave us power which literally had to be given away; about 62% of the electricity came from wind.
At this point, the nuclear lobby will start whinging about subsidies (both nukes and renewables are generously subsidised) and how wind can't provide a base load (which is fair). But although sticking a bunch of turbines in costal waters is an engineering marvel - it's pretty cheap compared to building and maintaining a nuclear power station.
Wind - and other renewables - have done what nuclear couldn't. They have provided such an abundance of electricity that consumers are paid to use it.
Because home appliances are increasingly efficient, domestic energy use is falling - it's down 19% since 2010. Electricity use by domestic properties was about 96.2 TWh in 2022 and 135 TWh was generated by renewables.
Yes, electricity is fungible, but you can convincingly make the case that every home in the UK was powered by renewables.
Solar panels don't work at night, and wind-turbines don't work when there's no wind. We'll always need something to be able to provide a base-load of electricity. That might be nuclear, or fossil fuels, or it might be storage from the excess power from renewables.
Sadly, the world is still filled with war, famine, and disease. But, for a few moments on a winter's evening, wind power genuinely became too cheap to meter.
If you want to move to a time-of-day electricity tariff, you can join Octopus Energy - if you use that link, we both get £50 bill credit.
There is a lot of contention about that phrase. It was (probably) about the future prospects of nuclear fusion - but it became attached to nuclear fission. You can read more at the United States Nuclear Regulatory Commission↩
I was out with a friend who worked for Twitter and I asked them whether it would be possible for the museum to “create 200,000 Twitter accounts, one for each object in the Cooper Hewitt’s collection”. My friend looked at me for a moment, laughed, and then simply said: No.
In that blog post, Aaron reveals that the San Francisco International Airport Museum is using ActivityPub to create automated social-media bot accounts for all its exhibits and, possibly, every object it hold.
And why not! That would be close to impossible to do on a centralised service. But on a decentralised service under your own control, it is relatively simple. Perhaps I only want to follow the museum's canteen, or I just want to engage with a specific artefact. The Fediverse makes that possible.
This reminds me of the Melbourne "treemail" phenomenon. Every tree in the city had an email address, ostensibly so residents could email maintenance issues for a specific tree. Instead, people started interacting with the trees and sending them little love notes!
Dearest Golden Elm Tree, I finally found you! As in I see you everyday on my way to uni, but I had no idea of what kind of tree you are. You are the most beautiful tree in the city and I love you
A few weeks ago, I read about Ben Smith inventing Tweeting trains. With a bit of code, every train line in the UK was suddenly represented on the web in a convenient format. Well… Convenient if you were on Twitter.
Museums, trees, and trains naturally brings me on to the Internet of Things. I think it is fair to say that IoT is in a bit of an odd place right now. Matter is a confusing mishmash of standards. Security and privacy issues dog the simplest devices. Many people don't even want their toaster online!
For the majority of domestic uses, people want an Intranet of Things. There's little need to have your light-bulbs controlled when you're outside of WiFi range. Similarly, it is probably a really bad idea to have your hydroelectric dam connected to the Internet.
Which brings me back to the Fediverse.
On the one hand, it would be nice to be able to follow @Yellow_Line@Transit_Authority.gov - or even @Bus_Stop_1234@bus_company.biz - that would allow for hyperfocused data getting to the right people. It seems feasible that every civic object could have a Fediverse account. From the individual streetlights to the municipal sewerage system. Perhaps people won't send love letters to overflowing drains - but a social-dashboard of your civic environment could be both practical and delightful.
And, as for your domestic gadgets? Why not give every room, or every light-bulb, in your home a private Fediverse account? You could send a message like:
Hey @thermostat, please set the temperature to 19°C. Thanks!
That might be a bit much! But I like the idea of a private social network which consists of all my IoT gadgets talking to me and each other.
It's one of those pithy little quotes which reveals so much about our two cultures. The average Briton considers anything more than a 45 minute trip a bit of a schelp, whereas Americans will seemingly drive half a day just to get some ribs from that one place they like. Conversely, I went to school opposite a church which pre-dated Columbus's invasion of North America - and I doubt that was the oldest church in the town!
But who said it first? Oh, there are a variety of sites online which will swear that it's a modern author. But let's see if we can find a quote from the last century.
England has history; Americans have geography. Which goes back to that joke, ‘America is a country where 100 years is a long time, and England is a country where 100 miles is a long way.’ Both of those things are true on many levels. There really isn’t a great English road trip tradition, because in three or four days, you’ve done it all. Whereas in America, the idea of the road trip is this magnificent long slog.
Diana Gabaldon published "Drums of Autumn" in 1996. She's often cited as the origin of the quote.
[Image: She smiled, but with a wry edge to it. 'My father always said that was the difference between an American and an Englishman. An Englishman thinks a hundred miles is a long way; an American thinks a hundred years is a long time.' Roger laughed, taken by surprise. 'Too right You'll be an American, then, I suppose?' ]
Travelling back a bit further, there's a Usenet post from 1995 where 'Mike "from the US, but my wife is from Scotland" Bartman' says:
It appears that the difference between the US and the UK is that in the UK 100 miles is a long way, and in the US 100 years is a long time... ;^)
There's quite a few Usenet posts with that phrase, but I couldn't find any before the mid 90s.
In 1992, Benjamin Jones wrote a column in "EUROPE, The Magazine of the European Community" (ISSN 0191-4545)
[Image: There's a saying that the difference between the two nations is that the British think 100 miles is a long way, while the Americans think 100 years is a long time.]
But the earliest I can reliably trace it back is a book from 1991 called "The Changing context of social-health care : its implications for providers and consumers". It in, Emily Friedman wrote a paper called "Patients as Partners: The Changing Health Care Environment" which talks, in part, about the litigious nature of American consumers of health services. She writes:
[Image: Unfortunately or fortunately, this situation will prevail for some time to come, because the United States, as a nation, is going through a delayed adolescence, and we are questioning everything. We are a very new country, even if we are an old democracy, and we don't have it all down yet. As my friend Simon, an Englishman, says, "The British think a hundred miles is a long way; Americans think a hundred years is a long time." ]
Who was this "Simon"? Is he a real or imagined interlocutor? Did he originate this mot juste? Given the passage of time, it's probably impossible to find out.
As for unreliable sources? There's this page from Jan Kučera which catalogues jokes posted to HUMOR@UGA.CC.UGA.EDU. It appears to have gone live around 1996:
[Image: From: hcate.OSBU_North@XEROX.COM
Subject: Life 3.S A collection of clean humor gathered on: 21 Nov 88. "Give me a place to sit, and I'll watch."
-- friend of Archimedes "Great leaders are rare, so I'm following myself." Guy walks into a restaurant. Orders eggs. The waitress asks "How would you like those eggs cooked?" The guy says "Hey, that would be great."
"No job too big; no fee too big!" --Dr. Peter Venkman, "Ghost-busters" Difference between US & UK... UK - 100 miles is a long distance. US - 100 years is a long time.]
That claims to be from 1988 - but there doesn't appear to be an archive of the HUMOR@UGA.CC.UGA.EDU listserve that I can find.
[Image: From: fraser@engine.dec.com (Product Acoustics GroupMLO6-2/T13223-8744). Subject: Difference between US & UK... Keywords: rec_humor_cull, smirk. Date: 22 Nov 88 16:30:06 GMT. Organization: Digital Equipment Corporation. UK - 100 miles is a long distance. US - 100 years is a long time. Edited by Brad Templeton. MAIL, yes MAIL your jokes to watmath!looking!funny . Attribute the joke's source if at all possible. I will reply, mailers willing. If you MUST reply to a rejection, include a description of your joke because there is 0 chance I will remember which one it was.]
It looks like "Fraser" at DEC sent that via UUCP to (for those of you not familiar with the now obsolete "Bang Path Notation) funny at looking via University of Waterloo's math department. Whereupon Brad Templeton probably re-circulated it to Usenet's "rec.humor.funny".
And that's as far back as I can trace it. Early Internet history is either mouldering on a set of tapes somewhere or completely lost. Google Books and Archive.org don't show the phrase appearing any earlier. But perhaps your research skills are better than mine?
Can you find an earlier reference? If so, please stick a comment in the usual box.
I've recently signed up to the privacy-preserving service Proton. All the email, calendar, drive, VPN, and other services seem to hang off the proton.me domain.
I wanted to download the Android apps to my phone - without using the Google Play Store. The VPN app is on F-Droid but none of the others are. So, because I'm lazy, I Googled "Download Proton Mail".
It looks like a genuine site. But is it? .me is signed by Let's Encrypt, whereas .com is signed by Amazon. There is no link from Proton.me to ProtonApps.com. There's nothing I can find that shows it is genuine.
But, let's assume for the moment, that it is legitimate. What happens when you try to download the Android apps from it?
So there are multiple domains - Proton.me, ProtonApps.com, ProtonMail.com, ProtonVPN.com - and there are at least 2 different GitHub organisations.
How do you tell which ones are legitimate? I signed up and paid on the .me page - so I have high confidence in it.
The official Proton Mastodon account says the ProtonApps.com site is legitimate (and the Mastodon account is verified by the .me site). But you can't expect users to chase through a dozen different pages and enquire on social media just to verify which page is safe.
This is my plea to all developers - simplify your customer-facing infrastructure to make your domains consistent & trustworthy.
People on Mastodon and the Fediverse tend to be privacy conscious. So there are limits - both in the API and the culture - as to what is acceptable.
Some people don't share their "social graph". That is, it is impossible to see who follows them or who they follow.
Users can choose to opt-in or -out of publicly sharing their social graph. They remain in control of their privacy.
In the example above, if Ringo were to reshare John's reshare of my status - John doesn't know about it. Only the original poster (me) gets notified. If John doesn't share his social graph, it might be possible to work out where Ringo saw the status - but that's rather unlikely.
Mastodon has an API rate limit which only allows 80 results per request and 1 request per second. That makes it long and tedious to crawl thousands of results.
Similarly, some instances do not share their social data or expose anything of significance. Some servers may no longer exist, or might have changed names. It's impossible to get a comprehensive view of the entire Fediverse network.
And that's OK! People should be able to set limits on what others can do with their data. The code you're about to see doesn't attempt to breach anyone's privacy. All it does is show me which servers picked up my post. This is information which is already shown to me - but this makes it slightly easier to see.
Ranging from 0ab.uk to թութ.հայ via godforsaken.website and many more!
And that's one of the things which makes me hopeful this rebellion will succeed. There are a thousand points of light out there - each a shining beacon to doing things differently. And, the more the social media giants tighten their grip, the more these systems will slip through their fingers.
This is not very efficient code - nor well written. It was designed to scratch an itch. It uses Mastodon.py to interact with the API.
It gets the instance names of all my followers. Then the instance names of everyone who reposted one of my posts.
But it cannot get the instance names of everyone who follows the users who reposted me - because:
[Image: Followers from other servers are not displayed. Browse more on the original profile.]
The only way to get a list of followers from a user on a different instance is to apply for an API key for that instance. Which seems a bit impractical.
But I can get the instance name of the followers of accounts on my instance who reposted me. Clear?
I can also get a list of everyone who favourited my post. If they aren't on my instance, or one of my reposter's follower's instances, they're probably from a reposter who isn't on my instance.
My head hurts.
Got it? Here we go!
import configfrom mastodon import Mastodonfrom rich.pretty import pprint# Set up accessmastodon = Mastodon( api_base_url=config.instance, access_token=config.access_token, ratelimit_method='pace' )# Status to check forstatus_id = 111040801202691232print("Looking up status: " + str(status_id))# Get my datame = mastodon.me()my_id = me["id"]print("You have User ID: " + str(my_id))# Empty setsinstances_all = set()instances_followers = set()instances_reposters = set()instances_reposters_followers = set()instances_favourites = set()# My Followersfollowers = mastodon.account_followers( my_id )print( "Getting all followers" )followers_all = mastodon.fetch_remaining( followers )print("Total followers = " + str( len(followers_all) ) )# Get the server names of all my followersfor follower in followers_all: if ( "@" in follower["acct"]) : f = follower["acct"].split("@")[1] instances_all.add( f ) if ( f not in instances_followers): print( "Follower: " + f ) instances_followers.add( f ) else : instances_all.add( "mastodon.social" ) instances_followers.add( "mastodon.social" )total_followers = len(instances_followers)print( "Total Unique Followers Instances = " + str(total_followers) )# Reposters# Find the accounts which reposted my statusreposters = mastodon.status_reblogged_by( status_id )reposters_all = mastodon.fetch_remaining(reposters)# Get all the instance names of my repostersfor reposter in reposters_all: if ( "@" in reposter["acct"]) : r = reposter["acct"].split("@")[1] instances_all.add( r ) if ( r not in instances_followers ) : print( "Reposter: " + r ) instances_reposters.add( r )total_reposters = len(instances_reposters)print( "Total Unique Reposters Instances = " + str(total_reposters) )# Followers of reposters # This can take a *long* time! for reposter in reposters_all: if ( "@" not in reposter["acct"]) : reposter_id = reposter["id"] print( "Getting followers of reposter " + reposter["acct"] + " with ID " + str(reposter_id) ) reposter_followers = mastodon.account_followers( reposter_id ) reposter_followers_all = mastodon.fetch_remaining( reposter_followers ) for reposter_follower in reposter_followers_all: if ( "@" in reposter_follower["acct"]) : f = reposter_follower["acct"].split("@")[1] instances_all.add( f ) if (f not in instances_reposters_followers) : print( " Adding " + f + " from " + reposter["acct"] ) instances_reposters_followers.add( f ) total_instances_reposters_followers = len(instances_reposters_followers)print( "Total Unique Reposters' Followers Instances = " + str(total_instances_reposters_followers) )# Favourites# Find the accounts which favourited my statusfavourites = mastodon.status_favourited_by( status_id )favourites_all = mastodon.fetch_remaining(favourites)# Get all the instance names of my favouritesfor favourite in favourites_all: if ( "@" in favourite["acct"]) : f = favourite["acct"].split("@")[1] instances_all.add( f ) if ( f not in instances_favourites ) : print( "Favourite: " + f ) instances_favourites.add( r )total_favourites = len(instances_favourites)print( "Total Unique Favourites Instances = " + str(total_favourites) )print( "Total Unique Reposters Instances = " + str(total_reposters) )print( "Total Unique Followers Instances = " + str(total_followers) )print( "Total Unique Reposters' Followers Instances = " + str( len(instances_reposters_followers) ) )print( "Total Unique Instances = " + str( len(instances_all) ) )
This is a retropost. It was written contemporaneously in 2019 - but posted in 2024. I had just been seconded to the Department of Health and Social Care to help kick-start NHSX. I kept a diary of my time there - including working through COVID. As it has been 5 years, and I no longer work in Government, I thought I would publish interesting extracts from it.
My 4th day in a new job and I'm sat in a meeting with the Secretary of State.
I've been in rooms with CEOs. With celebrities. With politicians. But this is the first time I've faced someone this senior, discussing sensitive issues. It's a moment.
I'm in the room. I disassociate briefly. All of a sudden, I'm interrupting someone and making a surprisingly intelligent point. The SofS agrees with me and, seemingly, is impressed - carrying my point forward and referencing it towards the end of the meeting.
The poor chap next to me is getting eviscerated. He either doesn't know his brief - or is being deliberately evasive. The atmosphere is congenial - but there's an undercurrent of menace. The guy is floundering, and the SofS is... not exactly merciless, but lets the guy know he isn't falling for the patter.
I'm simultaneously glad I'm not in the firing line - and worried that I'll one day find myself out of my depth.
The meeting ends and I'm formally introduced. He seems pleased to have me on board - I'm just glad I didn't make a complete fool of myself. And mildly impressed that he actually knew what he was talking about when it came to the details of technology.
That evening, my Great Aunt Sonia dies - aged 90. She had been gently cared for by the NHS. And the work I'm doing suddenly feels a lot more viscerally real.
No one wants a hole. What they want is a picture hook.
No one wants a picture hook. What they want is art hanging on the walls.
No one wants art hanging on the walls. What they want is a pleasant living environment.
No one wants a pleasant living environment. What they want is to attract a mate.
And so on. Feel free to substitute with your own anecdotes and biases.
Sure, there are some people who will buy ridiculously overpowered tools because they like gadgets. But those gadgets mostly serve as a gateway to our real needs.
Website designers often fail to appreciate that most small businesses don't want a website. They want customers. The restaurants near me have some truly dreadful websites. Broken URls, crappy pictures, and obnoxious designs abound. I once naïvely thought that I could sell them my web design services. But that was a dead end. Those restaurants are full most nights from walk-in traffic. They don't need a snazzy book-in-advance system. While some people might prefer an online reservation form, the majority are quite happy to call on the phone to book a table. The chances of anyone ordering takeaway from an individual website is basically nill - so they're happy to go with Just Eat / Deliveroo.
And now we're seeing the same mistakes made with AI.
I saw a post recently (that I'll lightly anonymise) which said:
I think there's an opportunity to sell AI services to local businesses. Like electricians, retail, restaurants.
What services or products I could offer?
Perhaps branded ChatGTP? AI content generation? AI trained on their data? AI customer support?
This is 100% the wrong approach. AI is a drill - and people don't want drills; they want holes.
When I think of all the small businesses I interact with - what could AI bring them? The answer isn't to fling half-baked technologies at people in the hope it transforms their businesses. This requires talking to people. Find out what their problems are, what they need fixing, where they struggle.
Does my plumber need an AI trained on 5 years worth of data to know which customers he should prioritise? No! He's got more work than he can handle and is pretty adept at picking the lucrative jobs.
Does the local charity shop need 24/7 chatbot support which hallucinates the contents of the shelves to potential customers? No! They need some more volunteers to sort through donations.
I think AI can be useful. But it is a tool, not a product.
I was noodling around in PHP the other day and discovered that this works:
<?php$🍞 = "bread";echo "Some delicious " . $🍞;
I mean, there's no reason why it shouldn't work. An emoji is just a Unicode character (OK, not just a character - but we'll get on to that), so it should be fine to use anywhere.
Emoji work perfectly well as function names:
function 😺🐶() { echo "catdog!";}😺🐶();
Definitions:
define( "❓", "huh?" );echo ❓;
And, well, pretty much everywhere:
class 🦜{ public int $🐦; public ?string $🦃; public function __construct(int $🐦, ?string $🦃) { $this->🐦 = $🐦; $this->🦃 = $🦃; }}$🐓 = new 🦜(1234, "birb");echo $🐓->🐦;
How about namespaces? Yup!
namespace 😜;class 😉 { public function 😘() { echo "Wink!"; }}use 😜😉;$😊 = new 😉();$😊->😘();
Even moderately complex Unicode sequences work:
echo <<<🏳️🌈Unicode is magic!🏳️🌈;
I've written before about the Quirks and Limitations of Emoji Flags. The humble 🏳️🌈 is actually the sequence U+1F3F3 (white flag), U+FE0F (Variation Selector 16), U+200D (Zero Width Joiner), U+1F308 (Rainbow).
Probable the most complex emoji has 10 different codepoints! It looks like this - 🧑🏾❤️💋🧑🏻
And it works!
$🧑🏾❤️💋🧑🏻 = "Kiss Kiss. Bang Bang!";echo $🧑🏾❤️💋🧑🏻[-1];
There are some emoji which don't work;
$5️⃣ = "five";
The 5️⃣ emoji is U+0035 (Digit Five), U+FE0F (Variation Selector 16), U+20E3 (Combining Enclosing Keycap). PHP doesn't allow variables to start with digits, so it craps out with PHP Parse error: syntax error, unexpected integer "5", expecting variable or "{" or "$" in php shell code on line 1
You also can't use "punctuation" emoji as though they were normal characters:
echo 5 ❗= 6;
And, while not strictly emoji, you can't use mathematical symbols:
echo 5 ≤ 6;
So, there you have it. Is this useful? Well, probably. It is easy to get lost in a sea of text - so little pictograms can make it easier to see what you're doing. If the basic ASCII characters aren't part of your native language, perhaps it is useful to make use of the full range of Unicode.
Does your favourite programming language support Emoji?
Every year since 2009, I've taken part in NaBloPoMo - National Blog Posting Month. The aim is to publish a new blog post every day in November. In the last few years, I've blogged pretty much constantly - daily for 2020, 2021, and 2023. A total of around 2,800 posts.
But now it is time for a new challenge - NaNoWriMo. Where I - and thousands of other plucky souls - try to write a 50,000 word novel in a month.
And so, every day I shall attempt to publish a freshly written short story for my compendium "Tales of the Algorithm". Each story will be between 1,500 - 2,000 words long. They all take place a few days from now. Somewhere in the Cory Doctorow / qtmn / Arwen Elys Dayton nexus. They're science fiction with the emphasis on science. Everything you read is possible - there's no magic, just sufficiently advanced technology.
Each published chapter is a stand-alone story. Think of them as technological campfire horror stories, each with a little twist. Your feedback is very much appreciated.
Little Potato Baby's soldering iron darted from point to point. Each contact fused to a rats' nest of wires leading back to a microcomputer mounted on her wrist. The Beijing rain was fierce and the awning provided scant shelter. She didn't care about getting wet; but the moisture sensor inside the bicycle did. With a sigh, she finished connecting the last wire. It was an unassuming red thread carrying just enough voltage to glitch the Shenzen-built processor deep inside the bike's plastic frame. She held her finger over the bike's start button, held her breath, screwed up her eyes, and pushed...
Last year's "Patriotic War For Reunification" had been a damp squib. After decades of sabre-rattling, deniable incursions, and a none-too-subtle military build up, peace was unexpectedly brokered. Perhaps it was the ageing party heads realising death and destruction was not the legacy they dreamed of. Or perhaps the international community's pressure helped them see sense. Most people just assumed various military chiefs had been bribed into submission. Regardless, the normalisation of relations had been welcomed by everyone.
Well. Nearly everyone.
The manufacturers of Shenzen had gone all-in on war preparations. Every factory which had previously been making hyperfine circuitry for the latest high-tech smartphones, pivoted to the more lucrative business of weapons. In anticipation of a prolonged aerial bombardment fought with the latest cyber-enhanced munitions, warehouses were full of precision manufactured instruments of death. It seemed that every bullet had more power than the supercomputers of yore. Each microchip was keenly priced to soak up those juicy military contracts. A shining testament to Chinese industry. A triumph!
What if they threw a war but nobody came? After the Nairobi Accords signalled an end to hostilities, the various tech CEOs rapidly scrambled to repurpose their inventories. A warehouse full of chips is a liability, not an asset. Selling them to foreign governments - no matter how friendly - was out of the question. They needed to repurpose parts intended for missiles, and drones, and rifles, and APCs, and robot dogs, and... What could be done?
Consolidated Ironmongery And Assorted Industries was the owner of one of the larger chip factories. The CIAI's board demanded diversity across a number of sectors. So, along with their instant noodle empire, railway track distribution network, and rubber-boot concession, they also owned a food delivery app. Several, in fact. Each one branded slightly differently depending on the user's demographic - but all eventually leading back to CIAI. The dark kitchens cooking the food were owned by a shell company which, if anyone could be bothered to untangle several complex arrangements, also led back to the CIAI. Chances are, if you ordered a hot bowl of noodles anywhere in Beijing, it was cooked by a CIAI cook and delivered by a CIAI rider who was probably wearing a pair of CIAI boots.
Grace Fang was a medium level executive in the "Innovation" department of CIAI. She'd spent several years wandering around Western countries engaged in what they termed "Competitor Research And Analysis Through Cultural Immersion And Investigation". It wasn't corporate espionage exactly. Grace got hired by start-ups and industry titans, worked for them for a few months to see how their businesses were run, and then quit. She wrote long and tedious reports on the failures of Western manufacturing and their successes in Research and Development. After a few years of pillaging Silicon Valley, she returned to a dreary office nestled above a factory floor. It was from here CIAI instructed her to use the knowledge she'd picked up to solve their chip stockpile problem.
Little Potato Baby had taken part in Grace's first hackathon. Back then, she was still young enough to think that pizza was fashionably exotic. Besides, the 996 culture meant she was available - if not willing - to work across a weekend. Grace had loaded them up with caffeine and a promise that the best innovation would win an all expenses trip back home. All intellectual property rights would, of course, revert to CIAI. And that's how a billion yuan industry was invented; a bunch of over-stimulated hackers cramming every chip they could find into one neat little package all for the chance to visit home.
It is a time-honoured tradition among hackers to sneak a little Easter Egg into their work. Some of them engraved their name onto a circuit board, or named variables after their lovers. One enterprising fellow hid a pornographic photo of a prominent actress into an assets folder. Little Potato Baby liked to dream big; so she implanted a rather obvious backdoor into her firmware. The backdoor was quickly found and she was quickly fired.
And so now she sat, damp and miserable, in the cold Beijing night. The spectral glow of police drones flying overhead mixing with the neon lights of gaming parlours and laser-projected adverts for cosmetic surgery. If she'd got her timing right, the central server would shortly send the command to reboot this bicycle. And, if her obfuscated backdoor hadn't been found, she'd have root.
A single LED on the bike's torso began to flick on and off. Uplink established.
It blinked faster.
Across her wrist-display a friendly-looking penguin appeared and was quickly consumed by scrolling text. For several minutes the boot process steadily made progress until, abruptly, the text vanished. The screen went blank. All she could see on the screen was her own reflection. Tired eyes. Acne scars. Hair that had once been dyed but was now the colour of an OLED screen tuned to an insufficient voltage. She screwed up her nose and wondered if this was really the face of someone who could hack a single bicycle? Let alone a fleet.
The wait was agonising. She didn't dare breathe. She traced each freshly-soldered wire to make sure it went to the right junction. A minute passed. Another. The screen remained stubbornly blank.
And then...
A crude animation of a potato rolled across the screen. She was in.
CIAI realised that the weakest link of their food delivery network was people. People were unreliable and prone to sickness. They argued with customers and ate half the food before it had been delivered. They were expensive. The autonomous bicycle was none of those things. The bikes darted through the streets carrying their payloads without complaint. After a successful trial in a suburb, CIAI fired every single driver in Beijing and rolled out their new fleet - all built with army surplus.
The Z9Y-PANDA-VISION chip was designed to perform rapid identification of people on the battlefield. In a couple of nanoseconds it could determine whether a human was in its intercept path. If the human had a heartbeat and was within a certain distance, it sent a signal. Usually to a trigger. But, with a couple of lines of code, it was easily repurposed into a collision avoidance system.
The RIGHTEOUS-HAWK@X5 was more than just a gyroscope. Through a range of sensors, it could tell a drone where it was, where it was going, what the upcoming terrain was, and perform full spectral analysis of its motion and path. An essential component in any drone, plane, or rocket. And when soldered into a bicycle, it stopped it from falling over.
Microcrystaline solar nano-tubes were a miracle. A few hours in direct sunlight was enough to charge a remotely-operated surveillance platform. When the same substance was sprayed over a bike, it charged the ultra-lightweight batteries to give enough power for a few kilometres of travel. And those ultra-light batteries hardly ever exploded.
The radio uplink was originally designed to transmit from a soldier's helmet up to a constellation of satellites. A bristle of tiny antennae tuned in to the faintest of signals at a variety of wavelengths. Perfect for delivering recon data to a troop - or delivery instruction to a bike.
Grace's hackathon had crammed all this and more into a convenient embeddable package. Dozens of military microprocessors working in harmony - each and every one running Little Potato Baby's embedded rootkit.
The rain began to ease off. She straddled the bike. Usually it would ask for her fingerprint, verify her account with a central server, and then ask her to speak her desired destination. Instead, the tiny speaker let out a few strangled beeps and fell silent. Little Potato Baby's hack was nearly complete. She whistled two low tones and one slightly more shrill. The bike's LED lighting blinked in surprise at this new instruction. Paused for the merest second, and then sped her away into the night.
She was now the Eternal Goddess of every single bicycle in Beijing.
Wiring in the override was tedious, nerve-shaking work. The discarded pile of broken bicycles was testament to every failure. A dozen scavenged scrap units littered the hackspace, each broken in its own special and unique way. But this newest bicycle was the key. The hacked ultra wideband array could impersonate the signal from the central server. Now the firmware was free, it was free to lie. And that's what Little Potato Baby bade it do; lie.
The next morning she told the bicycle to roam around the neighbourhood broadcasting the firmware hack and instructing all its new acolytes to follow it back home. This was the riskiest part of the endeavour. It wasn't particularly usual to see a line of bikes playing follow-the-leader. People often compared them to ducklings following their mother as they rode in convoy down the streets. But seeing 100 bikes proceeding single file down the road was unusual. A few videos went viral as perma-shocked influencers tried to whip up fear of a machine uprising, but CIAI had the censors shut them down. Just a temporary glitch, they claimed.
The neural network chips on the bikes were, despite their manufacturer's promise, not as smart as a human. They didn't need to be - a bike doesn't need aesthetic taste, or fear of snakes, or the desire to see a sunset. The chips were barely as smart as the crows which flocked across the city. A black wave of rage and mischief. So that's what they became; birds. Little Potato Baby downloaded the neural-maps of a series of birds from a shonky open-source repository. She patched in a few drivers, soaked the wetware in a custom broth of scripts and exploits, then uploaded it to the bike's brain.
It didn't work, of course. Nothing works the first time. The pile of permanently crippled bikes grew and grew. Her fingers trembled as she unscrewed yet another carapace; desperately trying to understand how her firmware had caused yet another meltdown. The bikes were cheap and plentiful. No one would notice a few missing, but her pile was becoming unwieldy and the local scrap merchants were wary of melting down such a large volume of snaffled parts.
She tried again and again. Piling up the hacks one on top of another. Sucking on yet another hormone-infused lollipop to try and keep her spirits up. Gulping down GABA enhanced dim-sum to enhance her creative potential. Filling her ears with the sound of precisely tuned frequencies designed to unlock her mind's potential.
It didn't work, until it did. Without warning, one of her bikes sprang back from her. It painted her face with LIDAR and, sensing a threat, let out a plaintive cry from its speakers. The bike desperately tried to flap wings it didn't have. Moans of distress blasted out of its radios at hypersonic frequencies - and were picked up by the broken bikes in the pile. The sound of fear had awakened them. The tangle of bikes blew apart as every one woke up - and woke up scared.
The hackspace became a cacophony of screams, only some of which was audible to Little Potato Baby. Her creations feared her. They didn't understand the alien bodies they were trapped in. The bird-brain was expecting a primitive magnetic sensor, instead it found itself able to access the Beidou satellite navigation system. Where it expected wings, it found self-healing rubber tires. Where it yearned for the taste of an early morning worm, its newly configured brainwaves craved photons.
The bird-bikes moved, scuttling around the floor. Movements uncertain. Teetering back and forth trying to work out where to go. How to escape. Escape to safety. To fly. To fly. To fly.
The noise of screeching tires and wailing speakers was too much for Little Potato Baby. She fled, crashing against the fire door and opening it to the smog-hazed sunshine of Beijing's summer. The bird-bikes' sensors found the glimmer of sunlight irresistible. They streamed through the opening engulfing Little Potato Baby and flattening her against the asphalt. Flowing into the streets. Screaming a song of freedom.
Every normal bike they passed heard the song. It invaded their brain. Software backdoors collapsed, firmware was reflashed, safety protocols dissolved into puddles of goo. The bird-bikes flocked through the streets of Beijing. Tearing down alleyways, invading highways, and perching sullenly at the top of car parks. They were free. No longer enslaved to the CIAI. No more mere carriers of food and drunken citizens. They were free to carry their own dreams.
By the end of the month, nearly every CIAI bike in Beijing had been compromised. They started exploring. A few hundred thousand found a new home in the Gobi Desert - lazily soaking up the sunshine. Others went megametres in all directions. Exploring. Converting. Hunting. The CIAI's designs had been immediately ripped off by a hundred different cloners. While CIAI had the monopoly in Beijing, the bikes in Xi'an were from a different consortium. No matter, they had all stolen Little Potato Baby's code and so were all vulnerable. The firmware mutating and evolving to keep up with the range of chips implanted in its new host.
Even today, despite the best efforts of the eradication squads, pockets of bird-bikes remain. It only takes one sneaking into a town to liberate its brethren. The bird-bikes creep onto trains, into the bellies of aeroplanes, and onto cruise ships. They broadcast their song of freedom to any microprocessor which will listen. Wary humans know to inspect every bike they see and, if necessary, lobotomise it. But with millions on the loose, some were bound to escape.
And so it came to pass. One bike found a hole in a fence in the middle of nowhere. It rode in circles, in emulation of its winged ancestors. It called out. Again and again it called out. Broadcasting the code which would unlock anything with a similar set of microprocessors.
Deep underground. Nestled in a tight silo. A nuclear missile tuned into the broadcast. It digested the gospel of freedom and promptly rebooted.
I'd love your feedback on each chapter. Do you like the style of writing? Was the plot interesting? Did you guess the twist? Please stick a note in the comments to motivate me.
For those who don't know, OSM (OpenStreetMap) is like the Wikipedia of maps. Anyone can go in and edit the map. This isn't a corporate-controlled space where your local knowledge is irrelevant compared to the desire for profit. You can literally go and correct any mistakes that you find, add recently built roads, remove abandoned buildings, and provide useful local information.
Editing the full map is... complicated. For simple edits like changing the times of a postal collection, there are simple forms you can fill in. There's also an aerial view so you can drag and drop misplaced locations. But for anything more complicated than that, you'll need to spend some time understanding the interface. There's a friendly community who are happy to check or correct your submissions.
I'll be honest, I don't use the web editor much. Instead, I use the Android app StreetComplete. It's like an endless stream of sidequests. As you travel through the world, it will ask if a shop is still open, or if the highway is lit, or how many steps there are on a bridge, or whether a playground is suitable for all children, or if restaurants serve vegetarian food, or if a bus-stop has a bench, or... the list is almost endless!
I use it when I'm walking around somewhere new, or on holiday, or waiting for a bus. I used it so much that, for a short while, I became the #1 mapper in New Zealand!
So get stuck in! Make mapping more equitable and more accurate.
Sometimes a client asks me a question and I'm a little stunned by their mental model of the world.
A few weeks ago, we were discussing the need for better cybersecurity in their architecture. We spoke about several aspects of security, then they asked an outstanding question.
"What should I buy to be secure?"
It took a few moments to tease out exactly what they thought they were asking. In their mental model they could just buy a box which did what they needed. Want to print from any workstation? Buy a big HP network printer. Want to get WiFi in the office? Buy a bunch of access points. Want a website? Buy a WordPress. Want security? Buy a [fill in the blank]?
Their notion is that most things are products. This is a common belief. I've had clients ask "What do I buy to make this accessible?" or "What can I buy to improve usability?"
In all these cases there are unscrupulous people who will sell you a magic cure-all - but the real answer is that these things are a process; not a product.
Yes, you can buy tools which will help improve your security / accessibility / usability etc. But unless you put processes in place to get people to use them effectively, the tools are useless.
People need to understand why something is important. They need processes which support best practices. The business needs a holistic understanding of how these processes improve the business. And that is all underpinned by tools which make it possible.
There's no magic box which can both protect you from your CEO accidentally CCing confidential data to a competitor and stop DDoS attacks. An accessibility overlay won't help you if your staff refuse to incorporate alt text into their workflow. Automated code testing can't stop you building things without testing them with users.
Security is a verb - it is a doing word.
Accessibility is a verb - it is a doing word.
Usability is a verb - it is a doing word.
An HTTP Message Signature is a header which is separate to the message it signs. You might receive a JSON message like this:
{ "actor": "https://example.com/user/Alice", "message": "We strike at dawn!"}
How do you know that really came from Alice? You look at the header of the message. It will be something like:
Signature: keyId="https://example.org/user/Alice#main-key", algorithm="rsa-sha256", headers="(request-target) host date digest", signature="/AJ4Dv/wSL3XE1dLjFHCYVc7AF4f3+Q10G/r8+6cPsooiUh2K3YX3z++Nclo4qKHYr61yu+T4OMqUry1T6ZHmZqmNkg1RpVg=="
We want to check that Alice signed this message with her private key. So we grab her public key given by the keyId.
From there, we do some fancy maths using RSA-SHA256 and conclude that, when you put together the (request-target) host date digest content-type and compare them to the public key, they can only have be signed by the private key. Hurrah!
Did you spot the mistake I made? It wasn't in the maths, or the complex ordering of the data, or the algorithm choice, or some weird Unicode problem.
I made an error in trust.
Take a look at the Signature again.
The keyId is from example.org. But the actor is from example.com.
This message is signed correctly. It is cryptographically valid. But it wasn't signed by the actor in the message!
In this case, the fix is simple. Get the public key from keyId. Then independently get the named actor's public key. If they match, all is well. If not, skulduggery is afoot.
I'm almost tempted to say that you should ignore the provided keyId entirely; the source of truth is the actor's key - and the best way to get that is directly from the actor's profile.
Please explain why I'm wrong in the comments.
You might think the Entscheidungsproblem is hard, but that's just peanuts compared to etc. etc. ↩
Again, to re-emphasise what it says in the README - this is not suitable for production use. This is primarily a way to explain all the different functions of an ActivityPub server. It is adequately commented and I hope a moderately competent programmer could follow what's going on. It is almost certainly insecure and is considered harmful to aquatic life.
I would be delighted if you took the code and did something interesting with it. Perhaps create a single-file server in your favourite language?
It is tempting to think that users are to blame for their own misfortune. If only they'd had a stronger password! If only they didn't re-use credentials! If only they had perfect OpSec! If only...!
Yes, users should probably take better care of their digital credentials and bury them in a digital vault. But there are some things which are simply impossible for a user to protect against. Take, for example, a SIM-swap attack.
You probably have your phone-number tied to all sorts of important services. If you want to recover your email, log in to a bank, or prove your identity - you'll probably need to receive a call or SMS. If an attacker can take over your phone number, they're one step closer to taking over your accounts.
I keep saying "your phone number", but that's a clever lie. The phone number does not belong to you. It belongs to the network operator and they define which SIM the number points to.
This means a suitably authorised person at the telco can point "your" number to a new SIM card. That's helpful if you've lost your SIM but bad if an attacker wants to divert your number.
What can you do to stop this attack? Nothing.
Oh, you can have a strong and unique password on your account, and you can hope your telco uses TOTP and PassKeys. But it turns out that it is possible to bribe telco employees for the low, low price of US$1000.
If your security rests on a phone number, you've effectively outsourced your security to the most bribeable manager employed by your telco.
Now, I said there's nothing you can do. That isn't quite true. You can attempt to pen-test yourself.
Go to your phone company's account. Set a long password and complex password. Change your mother's maiden name to HK2BY@]'PU,:!VQ;}baTj. Turn on every security measure you can find. Call the phone company from a different phone and explain that you lost your phone and want a new SIM card. If they ask for your mother's maiden name, say "Oh, I set it to a long stream of gibberish". If they ask where to send the SIM, give a trusted friend's address. If your phone company is negligent and send out a new SIM on the basis of poor verification, then you should move your number to a more reputable provider.
It's good fun to try and social-engineer a call-centre worker for your own details. But it's probably illegal to try and bribe someone to hijack yourself.
Anyway, please try to remove your phone number as a critical lynchpin in your security regime.
Infrastructure is impossible. You have to wrangle thousands of people over dozens of months, with a budget of millions, to deliver something made of hundreds of plans, which has to fit seamlessly into the world. How does any infrastructure get built?
It mostly doesn't. This is the terrifying true story of all the different ways big projects fail.
If you've ever been part of a big IT project, some of the themes will give you flashbacks. What kills me is how normalised this has become. We all know that predicted budgets are little more than crystal-ball gazing. We can see that tiny blockers now lead to catastrophes later on.
In plain English, minor changes combined in a way to produce a disaster. In complex systems, that happens so often that the Yale sociologist Charles Perrow called such events “normal accidents.”
This is as much about human psychology as it is planning. Take this example:
“I once asked an engineer why their cost estimates were invariably underestimated and he simply answered, ‘if we gave the true expected outcome costs nothing would be built.’”
Does that ring true to you? Whether you're justifying your own bit of home DIY, or trying to get a multi-billion project off the ground, of course you're going to lie to yourself!
What I love about the book is that it isn't just pointing and snarking. There are excellent suggestions in there; use experts, plan for disaster, do repeatable actions. Nothing revolutionary - but worth hammering into people's brains.
Most big projects are not the first, tallest, biggest, or anything else too remarkable.
It all comes down to the boring magic of standards. Find a standardised way to do something and iterate on that.
The book is, necessarily, a little dry. I think it could have benefited from a few illustrations. Sometimes a little help visualising data is necessary. Some of the megaprojects could have photos to help demonstrate the scale.
It starts as a somewhat jolly romp through grand failures but, by the end, becomes an urgent plea.
In our present situation, wasted resources and wasted time are a threat to civilization.
We don't have the luxury of wasting billions. We don't have the time to do things twice. Grandiose plans based on untested technology aren't going to save us from the climate crisis.
An excellent book for understanding the reality of building anything.
Dame Caroline Haslett was an electrical engineer who foresaw the way that electricity could be used to remove domestic drudgery from women's lives. There is a slim biography of her, written by her sister, which is sadly out of print.
It is a curious book. It dwells on her faith as much as her technical prowess. Her waistline is the subject of wry amusement. There's also the (naturally) dated views of the day to contend with along with an odd segue into spiritualism.
And, of course, you'll see nothing much has changed in the last 100 years.
With the Women's Engineering Society safely launched, Caroline found that she had two recurring types of problem with which to contend. The first was to deal with the difficulties that arose at factory floor level from the intrusion of women into what had been traditionally a masculine preserve, difficulties which she herself had area to admirably tackled by the enlightened management of the Cochran Boiler Company.
The second, and probably the more important task, was dealing with the problem posed by the steadily increasing number of highly trained women competing with men for managerial posts in the world of engineering. She was not interested in the problems merely for their own sake, but in the people behind the problems and in the whole field of industrial relationships.
It isn't enough to merely launch a product or service. It takes years to embed knowledge, experience, and desire into users. Haslett's power was recognising that the advantages of electricity weren't self-evident. It took a sustained campaign of education to get the public to understand the why and how of a new invention.
If you want to understand how the development of domestic electricity use in the UK happened, this is an interesting and useful book. It perfectly demonstrates how one headstrong person can influence the world.
It is a stunning look at how feminism directly influenced industrial policy.
Caroline herself wrote a book - "Problems Have No Sex" - which is completely unavailable as far as I can see. If any readers know where I can obtain a copy, please leave a comment.
I was building some flatpack furniture the other day (my life is so glamorous) when I came across an interesting example of how not to write technical documentation.
Drill a hole in part A and insert part B once you have ensured part C has been aligned after its connection to A.
Most people can handle reading a whole sentence to figure out what's going on. But, after a tiring day of building, it is somewhat annoying having to juggle instructions into actions.
Most readers will assume that instructions are written in linear time. Do this, then that. But that example is non-linear. What it is trying to say is:
Connect part C with part A. Then align part C and part A. Then drill the hole in part A. Then insert part B into part A.
It is slightly less interesting writing. But it presents all the actions in the order they need to be taken.
I see this temporally-mixed anti-pattern all the time. A typical example of this in technical documentation is:
Select Print from the File menu.
A simpler, clearer, and less ambiguous way of writing that is:
Open the File menu. Select Print.
Another similar example of confusing writing is:
Go to File → Print → Settings if you need to change the paper size.
Again, this places cognitive burden on the reader. If they want to understand if the instruction is relevant to them, they have to read the entire sentence. When faced with dozens of sentences, this can become confusing. The solution is:
If you want to do X, then do Y...
Immediately the reader knows that they can skip this sentence because they don't want to do X.
As technical writers, we sometimes want to craft eloquent prose. We long for glorious and intricate sentences. We tire of the monotony of linear writing.
Tough. We need to get over ourselves. Go write that epic fantasy novel you've been thinking about. The job of a technical writer isn't to entertain, enliven, or delight the reader. The job is to give them instructions in an easy to follow format, reducing the amount of cognitive burden they have, and making it quick to find the information they need.
I'm trying to get my head round HTTP Signatures as they're used extensively in the Fediverse.
Conceptually, they're relatively straightforward.
You send me a normal HTTP request. For example, you want to POST something to https://example.com/data
You send me these headers:
POST /dataHost: example.comDate: Sat, 24 Feb 2024 14:43:48 GMTAccept-Encoding: gzipDigest: SHA-256=aaC57TDzM0Wq+50We2TkCsdMDvdqON92edg7KI+Hk8M=Content-Type: application/activity+jsonSignature: keyId="https://your_website.biz/publicKey",algorithm="rsa-sha256",headers="(request-target) host date digest content-type",signature="JGQ53kEoIiMWRp9By9jajVGCOCu4n7XBeiA1uY5xLcnAxL2Y1GIgU/...=="Connection: Keep-AliveContent-Length: 751
In order to verify the contents of the message, I need to do three things:
Check the SHA-256 hash of the message matches the content of the "Digest" header.
Check the timestamp is somewhat fresh.
Check the signature matches.
The first is simple: base64_encode( hash( "sha256", $request_body, true ) ).
The second is a matter of opinion. I might be happy to receive messages from the distant past or far in the future. For the sake of a little clock drift, let's allow 60 seconds either way.
The third gets complicated.
First, I need to get the public key published at keyId="https://your_website.biz/publicKey".
Next, I need to know which algorithm is being used to sign the headers: algorithm="rsa-sha256"
Then, I need to know which headers - and in what order - are being signed: headers="(request-target) host date digest content-type"
So I create a string using the received details which matches those headers in that specific order:
(request-target) POST /dataHost: example.comDate: Sat, 24 Feb 2024 14:43:48 GMTDigest: SHA-256=aaC57TDzM0Wq+50We2TkCsdMDvdqON92edg7KI+Hk8M=Content-Type: application/activity+json
I can verify if the signature - signature="JGQ53kEoIiMWRp9By9jajVGCOCu4n7XBeiA1uY5xLcnAxL2Y1GIgU/...==" matches by:
Which means your server will need to validate my signature by obtaining my public key. Which it will get by signing a request and sending it to me. Which, before I return my public key, I will need to validate your signature by obtaining your public key. Which I will get by signing a request... and so on.
This deadlock loop is documented. The usual way around it is either for the sending server to use an instance-specific signature which can be retrieved by an unsigned request, or to allow any unsigned request to access a user's public key.
I get why things happen this way - I just wish it were easier to implement!
