The thing is, any feed or search result is "algorithmic." "Just show me the things posted by people I follow in reverse-chronological order" is an algorithm. "Just show me products that have this SKU" is an algorithm. "Alphabetical sort" is an algorithm. "Random sort" is an algorithm.
--
If you'd like an essay-formatted version of this thread to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
@pluralistic For those of us who don't live on the socials "reverse-chronological" gives you a feed of people who were active in the last five minutes. I like a feed that at least tries to show me things I'd find interesting that were posted since the last time I checked in.
Any process that involves more information than you can take in at a glance or digest in moments needs some kind of sense-making. It needs to be put in some kind of order. There's always gonna be an algorithm.
But that's not what we mean by "the algorithm" (TM). When we talk about "the algorithm," we mean a system for ordering information that uses complex criteria that are not precisely known to us, and than can't be easily divined through an examination of the ordering.
There's an idea that a "good" algorithm is one that does not seek to deceive or harm us. When you search for a specific part number, you want exact matches for that search at the top of the results. It's fine if those results include third-party parts that are compatible with the part you're searching for, so long as they're clearly labeled. There's room for argument about how to order those results - do highly rated third-party parts go above the OEM part?
How should the algorithm trade off price and quality?
It's hard to come up with an objective standard to resolve these fine-grained differences, but search technologists have tried. Think of Google: they have a patent on "long clicks." A #LongClick is when you search for something and then don't search for it again for quite some time, the implication being that you've found what you were looking for.
Google Search ads operate a "pay per click" model, and there's an argument that this aligns Google's ad division's interests with search quality: if the ad division only gets paid when you click a link, they will militate for placing ads that users want to click on.
Platforms are inextricably bound up in this algorithmic information sorting business.
Platforms have emerged as the endemic form of internet-based business, which is ironic, because a platform is just an intermediary - a company that connects different groups to each other. The internet's great promise was "#disintermediation" - getting rid of intermediaries. We did that, and then we got a whole bunch of new intermediaries.
Usually, those groups can be sorted into two buckets: "business customers" (drivers, merchants, advertisers, publishers, creative workers, etc) and "end users" (riders, shoppers, consumers, audiences, etc). Platforms also sometimes connect end users to each other: think of dating sites, or interest-based forums on Reddit. Either way, a platform's job is to make these connections, and that means platforms are always in the algorithm business.
Whether that's matching a driver and a rider, or an advertiser and a consumer, or a reader and a mix of content from social feeds they're subscribed to and other sources of information on the service, the platform has to make a call as to what you're going to see or do.
These choices are enormously consequential. In the theory of #SurveillanceCapitalism, these choices take on an almost supernatural quality.
In this theory, "Big Data" can be used to guess your response to all the different ways of pitching an idea or product to you, in order to select the optimal pitch that bypasses your critical faculties and actually controls your actions, robbing you of "the right to a future tense."
I don't think much of this hypothesis. Every claim to mind control - from Rasputin to MK Ultra to neurolinguistic programming to pick-up artists - has turned out to be bullshit.
Besides, you don't need to believe in mind control to explain the ways that algorithms shape our beliefs and actions. When a single company dominates the information landscape - say, when Google controls 90% of your searches - then Google's sorting can deprive you of access to information without you knowing it.
If every "locksmith" listed on Google Maps is a fake referral business, you might conclude that there are no more reputable storefront locksmiths in existence.
What's more, this belief is a self-fulfilling prophecy: if Google Maps never shows anyone a real locksmith, all the real locksmiths will eventually go bust.
If you never see a social media update from a news source you follow, you might forget that the source exists, or assume they've gone under. If you see a flood of viral videos of smash-and-grab shoplifter gangs and never see a news story about wage theft, you might assume that the former is common and the latter is rare.
In reality, shoplifting hasn't risen appreciably, while wage-theft is off the charts.
In the theory of Surveillance Capitalism, the algorithm was invented to make advertisers richer, and then went on to pervert the news (by incentivizing "clickbait") and finally destroyed our politics when its persuasive powers were hijacked by Steve Bannon, Cambridge Analytica, and QAnon grifters to turn millions of vulnerable people into swivel-eyed loons, racists and conspiratorialists.
As I've written, I think this theory gives the ad-tech sector both too much and too little credit, and draws an artificial line between ad-tech and other platform businesses that obscures the connection between all forms of platform decay, from Uber to HBO to Google Search to Twitter to Apple and beyond:
As counter to Surveillance Capitalism, I've proposed a theory of platform decay called #enshittification, which identifies how monopoly platforms' market power, combined with the flexibility of digital tools and regulatory capture, allows platforms to abuse both business-customers and end-users, by depriving them of alternatives, then "twiddling" the knobs that determine the rules of the platform without fearing sanction under privacy, labor or consumer protection law.
And finally, enshittification requires blocking digital self-help measures like ad-blockers, alternative clients, scrapers, reverse engineering, jailbreaking, and other tech guerrilla warfare tactics:
Surveillance Capitalism starts from the assumption that surveillance advertising is devastatingly effective (which explains how your racist Facebook uncle got turned into a Jan 6 QAnon), and concludes that advertisers must be well-served by surveillance systems.
But advertisers - and other business customers - are very poorly served by platforms. Procter & Gamble reduced its annual surveillance advertising budget from $100m//year to $0/year and saw a 0% reduction in sales.
The supposed laser-focused targeting and superhuman message refinement just don't work very well - first, because the tech companies are run by bullshitters whose marketing copy is nonsense, and second because these companies are monopolies who can abuse their customers without losing money.
The point of enshittification is to lock end-users to the platform, then use those locked-in users as bait for business customers, who will also become locked to the platform.
Once everyone is holding everyone else hostage, the platform uses the flexibility of digital services to play a variety of algorithmic games to shift value from everyone to the business's shareholders. This flexibility is supercharged by the failure of regulators to enforce privacy, labor and consumer protection standards against the companies.
And it's enhanced by these companies' ability to insist that regulators punish end-users, competitors, tinkerers and other third parties to mod, reverse, hack or jailbreak their products and services to block their abuse.
Enshittification needs The Algorithm. When Uber wants to steal from its drivers, it can just do an old-fashioned wage theft, but eventually it will face the music for that kind of scam:
The best way to steal from drivers is with #AlgorithmicWageDiscrimination. That's when Uber offers occassional, selective drivers higher rates than it gives to drivers who are fully locked to its platform and take every ride the app offers. The less selective a driver becomes, the lower the premium the app offers goes, but if a driver starts refusing rides, the wage offer climbs again.
This isn't the mind-control of Surveillance Capitalism, it's just fraud, shaving fractional pennies off your paycheck in hopes that you won't notice. The goal is to get drivers to abandon other side-hustles that allow them to be choosy about when they drive Uber, and once the driver is fully committed, crank the wage-dial down to the lowest possible setting:
When Facebook began aggressively courting publishers, any short snippet republished from the publisher's website to a Facebook feed was likely to be recommended to large numbers of readers. Facebook offered publishers a vast traffic funnel that drove millions of readers to their sites.
But as publishers became more dependent on that traffic, Facebook's algorithm started downranking short excerpts in favor of medium-length ones, building slowly to fulltext Facebook posts that were fully substitutive for the publisher's own web offerings. Like Uber's wage algorithm, Facebook's recommendation engine played its targets like fish on a line.
When publishers responded to declining reach for short excerpts by stepping back from Facebook, Facebook goosed the traffic for their existing posts, sending fresh floods of readers to the publisher's site. When the publisher returned to Facebook, the algorithm once again set to coaxing the publishers into posting ever-larger fractions of their work to Facebook, until, finally, the publisher was totally locked into Facebook.
Facebook started charging publishers for "#boosting" - not just for inclusion in algorithmic recommendations, but to reach their subscribers.
Enshittification is modern, tech enabled, monopolistic form of #RentSeeking. Rent-seeking is a subtle and important idea from economics, one that is increasingly relevant to our modern economy. For economists, a "rent" is income you get from owning a "factor of production" - something that someone else needs to make or do something.
Rents are not "profits." Profit is income you get from making or doing something. Rent is income you get from owning something needed to make profits. People who earn income from rents are called #rentiers. If you make your income from profits, you're a "capitalist."
Capitalists and rentiers are in irreconcilable combat. A capitalist wants access to their factors of production at the lowest possible price, whereas rentiers want those prices to be as high as possible.
A phone manufacturer wants to be able to make phones as cheaply as possible, while a patent-troll wants to own a patent that the phone manufacturer needs to license in order to make phones. The manufacturer is a capitalism, the troll is a rentier.
The troll might even decide that the best strategy for maximizing their rents is to exclusively license their patents to a single manufacturer and try to eliminate all other phones from the market.
This will allow the chosen manufacturer to charge more and also allow the troll to get higher rents. Every capitalist except the chosen manufacturer loses. So do people who want to buy phones. Eventually, even the chosen manufacturer will lose, because the rentier can demand an ever-greater share of their profits in rent.
Digital technology enables all kinds of rent extraction. The more digitized an industry is, the more rent-seeking it becomes.
@pluralistic
> Digital technology enables all kinds of rent extraction.
Really? Why is this inherent to digital technology but not other kinds?
> The more digitized an industry is, the more rent-seeking it becomes
That's certainly been the case over the last couple of decades. But I'd say it's because digital technology industries have been even more under-regulated than others during that time. To the point where disguising a taxi business as a digital tech business evades taxi regulations.
@pluralistic
There's nothing particularly digital about the supermarket industry. But total lack of effective anti-trust enforcement in NZ has resulted in a thriving competitive market being reduced to a duopoly with 75% of the grocery market. Having bought up or destroyed all competition, they've been able to control the prices most food suppliers can get across the country. Just like Amazon as the dominant online retailer. Classic enshittification. See;
The cloud is especially prone to rent-seeking, as #YanisVaroufakis writes in his new Technofeudalism, where he explains how "#cloudalists" have found ways to lock productive enterprises into using cloud-based resources from which ever-increasing rents can be extracted:
The endless malleability of digitization makes for endless variety in rent-seeking, and cataloging all the different forms of digital rent-extraction is a major project in this Age of Enshittification. "Algorithmic Attention Rents: A theory of digital platform market power," a new #UCL#InstituteForInnovationAndPublicPurpose paper by @timoreilly, #IlanStrauss and #MarianaMazzucato, pins down one of these forms:
The #AttentionRents referenced in the title are bait-and-switch scams where a platform deliberately enshittifies its recommendations, search results or feeds to show you things that are not what you asked to see, expect to see, or want to see. They don't do this out of sadism! The point is to extract rent - from you (wasted time, suboptimal outcomes) and from business customers (extracting rents for "boosting," jumbling good results in among scammy or low-quality results).
The authors cite several examples of these attention rents. Much of the paper is given over to Amazon's so-called "advertising" product, a $31b/year program that charges sellers to have their products placed above the items that Amazon's own search engine predicts you will want to buy:
This is a form of gladiatorial combat that pits sellers against each other, forcing them to surrender an ever-larger share of their profits in rent to Amazon for pride of place. Amazon uses a variety of deceptive labels ("Highly Rated - Sponsored") to get you to click on these products, but most of all, they rely two factors. First, Amazon has a long history of surfacing good results in response to queries, which makes buying whatever's at the top of a list a good bet.
Second, there's just so many possible results that it takes a lot of work to sift through the probably-adequate stuff at the top of the listings and get to the actually-good stuff down below.
Amazon spent decades subsidizing its sellers' goods - an illegal practice known as "predatory pricing" that enforcers have increasingly turned a blind eye to since the Reagan administration. This has left it with few competitors:
The lack of competing retail outlets lets Amazon impose other rent-seeking conditions on its sellers. For example, Amazon has a "#MostFavoredNation" requirement that forces companies that raise their prices on Amazon to raise their prices everywhere else, which makes everything you buy more expensive, whether that's a Walmart, Target, a mom-and-pop store, or direct from the manufacturer:
But everyone loses in this "two-sided market." Amazon used #JunkAds to juice its ad-revenue: these are ads that are objectively bad matches for your search, like showing you a Seattle Seahawks jersey in response to a search for LA Lakers merch:
The more of these junk ads Amazon showed, the more revenue it got from sellers - and the more the person selling a Lakers jersey had to pay to show up at the top of your search, and the more they had to charge you to cover those ad expenses, and the more they had to charge fot it everywhere else, too.
The authors describe this process as a transformation between "attention rents" (misdirecting your attention) to "#PecuniaryRents" (making money).
That's important: despite decades of rhetoric about the "#AttentionEconomy," attention isn't money. As I wrote in my enshittification essay:
> You can't use attention as a medium of exchange. You can't use it as a store of value. You can't use it as a unit of account. Attention is like cryptocurrency: a worthless token that is only valuable to the extent that you can trick or coerce someone into parting with "fiat" currency in exchange for it.
> You have to "monetize" it – that is, you have to exchange the fake money for real money.
The authors come up with some clever techniques for quantifying the ways that this scam harms users. For example, they count the number of places that an advertised product rises in search results, relative to where it would show up in an "organic" search. These quantifications are instructive, but they're also a kind of subtweet at the judiciary.
In 2018, #SCOTUS's ruling in #AmericanExpressVOhio changed antitrust law for two-sided markets by insisting that so long as one side of a two-sided market was better off as the result of anticompetitive actions, there was no antitrust violation:
For platforms, that means that it's OK to screw over sellers, advertisers, performers and other business customers, so long as the end-users are better off.
"Go ahead, cheat the Uber drivers, so long as you split the booty with Uber riders."
But in the absence of competition, regulation or self-help measures, platforms cheat everyone - that's the point of enshittification. The attention rents that Amazon's payola scheme extract from shoppers translate into higher prices, worse goods, and lower profits for platform sellers.
In other words, Amazon's conduct is so sleazy that it even threads the infinitesimal needle that the Supremes created in American Express.
Here's another algorithmic pecuniary rent: Amazon figured out which of its major rivals used an automated price-matching algorithm, and then cataloged which products they had in common with those sellers.
Then, under a program called #ProjectNessie, Amazon jacked up the prices of those products, knowing that as soon as they raised the prices on Amazon, the prices would go up everywhere else, so Amazon wouldn't lose customers to cheaper alternatives. That scam made Amazon at least a billion dollars:
The speed and flexibility with which Amazon and its rivals altered their prices requires digitization. Digitization also let Amazon crank the price-gouging dial to zero whenever they worried that regulators were investigating the program.
So what do we do about it? After years of being made to look like fumblers and clowns by Big Tech, regulators and enforcers - and even lawmakers - have decided to get serious.
The neoliberal narrative of government helplessness and incompetence would have you believe that this will go nowhere. Governments aren't as powerful as giant corporations, and regulators aren't as smart as the supergeniuses of Big Tech. They don't stand a chance.
But that's a counsel of despair and a cheap trick. Weaker US governments have taken on stronger oligarchies and won - think of the defeat of JD Rockefeller and the breakup of Standard Oil in 1911.
The people who pulled that off weren't wizards. They were just determined public servants, with political will behind them. There is a growing, forceful public will to end the rein of Big Tech, and there are some determined public servants surfing that will.
In this paper, the authors try to give those enforcers ammo to bring to court and to the public.
For example, Amazon claims that its algorithm surfaces the products that make the public happy, without the need for competitive pressure to keep it sharp. But as the paper points out, the only successful new rival ecommerce platform - Tiktok - has found an audience for an entirely new category of goods: #dupes, "lower-cost products that have the same or better features than higher cost branded products."
The authors also identify #DarkPatterns that platforms use to trick users into consuming feeds that have a higher volume of things that the company profits from, and a lower volume of things that users want to see. For example, platforms routinely switch users from a "following" feed - consisting of things posted by people the user asked to hear from - with an algorithmic "For You" feed, filled with the things the company's shareholders wish the users had asked to see.
Calling this a "dark pattern" reveals just how hollow and self-aggrandizing that term is. "Dark pattern" usually means "fraud." If I ask to see posts from people I like, and you show me posts from people who'll pay you for my attention instead, that's not a sophisticated sleight of hand - it's just a scam.
It's the social media equivalent of the eBay seller who sends you an iPhone box with a bunch of gravel inside it instead of an iPhone. Tech bros came up with "dark pattern" as a way of flattering themselves by draping themselves in the mantle of dopamine-hacking wizards, rather than unimaginative con-artists who use a computer to rip people off.
These For You feeds aren't just a way to increase the load of sponsored posts - they're also part of the multi-sided enshittification platform ripoff. A For You feed allows platforms to trick publishers and performers into thinking they are "good at the platform," which both convinces to optimize their production for that platform, and also turns them into Judas Goats who conspicuously brag about how great the platform is for people like them, which brings their peers in, too.
In #VeenaDubal's essential paper on algorithmic wage discrimination, she describes how Uber drivers whom the algorithm has favored with (temporary) high per-ride rates brag on driver forums about their skill with the app, bringing in other drivers who blame their lower wages on their failure to "use the app right":
> If you go down to the midway at your county fair, you'll spot some poor sucker walking around all day with a giant teddy bear that they won by throwing three balls in a peach basket.
> The peach-basket is a rigged game. The carny can use a hidden switch to force the balls to bounce out of the basket. No one wins a giant teddy bear unless the carny wants them to win it. Why did the carny let the sucker win the giant teddy bear?
> The carny allocated a giant teddy bear to that poor sucker the way that platforms allocate surpluses to key performers – as a convincer in a "Big Store" con, a way to rope in other suckers who'll make content for the platform, anchoring themselves and their audiences to it.
Platform can't run the giant teddy-bear con unless there's a For You feed. Some platforms - like Tiktok - tempt users into a For You feed by making it as useful as possible, then salting it with doses of enshittification:
Either way, the platform can't let anyone keep the giant teddy-bear. Once you've tempted, say, sports bros into piling into the platform with the promise of millions of free eyeballs, you need to withdraw the algorithm's favor for their content so you can give it to, say, astrologers.
Of course, the more locked-in the users are, the more shit you can pile into that feed without worrying about them going elsewhere, and the more giant teddy-bears you can give away to more business users so you can lock them in and start extracting rent.
For regulators, the possibility of a "good" algorithmic feed presents a serious challenge: when a feed is bad, how can a regulator tell if its low quality is due to the platform's incompetence at blocking spammers or guessing what users want, or whether it's because the platform is extracting rents?
The paper includes a suite of recommendations, including one that I really liked:
> Regulators, working with cooperative industry players, would define reportable metrics based on those that are actually used by the platforms themselves to manage search, social media, e-commerce, and other algorithmic relevancy and recommendation engines.
In other words: find out how the companies measure their performance. Find out what KPIs executives have to hit in order to earn annual bonuses and use those to figure out what the company's performance is - ad load, ratio of organic clicks to ad clicks, average click-through on the first organic result, etc.
They also recommend hard rules, like reserving a portion of the top of the screen for "organic" search results, and requiring exact matches to be the top result.
I've proposed something similar, applicable across multiple kinds of digital businesses: an #EndToEnd principle for online services. The end-to-end principle is as old as the internet, and it decrees that the role of an intermediary should be to deliver data from willing senders to willing receivers as quickly and reliably as possible. When we apply this principle to your ISP, we call it #NetNeutrality.
For services, E2E would mean that if I subscribed to your feed, the service would have a duty to deliver it to me. If I hoisted your email out of my spam folder, none of your future emails should land there. If I search for your product and there's an exact match, that should be the top result:
One interesting wrinkle to framing platform degradation as a failure to connect willing senders and receivers is that it places a whole host of conduct within the regulatory remit of the #FTC. #Section5 of the #FTCAct contains a broad prohibition against "unfair and deceptive" practices:
That means that the FTC doesn't need any further authorization from Congress to enforce an end to end rule: they can simply propose and pass that rule, on the grounds that telling someone that you'll show them the feeds that they ask for and then not doing so is "unfair and deceptive."
Some of the other proposals in the paper also fit neatly into Section 5 powers, like a "sticky" feed preference.
If I tell a service to show me a feed of the people I follow and they switch it to a For You feed, that's plainly unfair and deceptive.
All of this raises the question of what a post-Big-Tech feed would look like. In "How To Break Up Amazon" for The Sling, Peter Carstensen and Darren Bush sketch out some visions for this:
They imagine a "condo" for Amazon, where sellers collectively own the storefront, a model similar to capacity rights on natural gas pipelines, or to patent pools. They see two different ways that search-result order could be determined in such a system:
> "specific premium placement could go to those vendors that value the placement the most [with revenue] shared among the owners of the condo"
or
> "leave it to owners themselves to create joint ventures to promote products"
Note that both of these proposals are compatible with an end-to-end rule and the other regulatory proposals in the paper. Indeed, all these policies are easier to enforce against weaker companies that can't afford to maintain the pretense that they are headquartered in some distant regulatory haven, or pay massive salaries to ex-regulators to work the refs on their behalf:
The re-emergence of intermediaries on the internet after its initial rush of disintermediation tells us something important about how we relate to one another. Some authors might be up for directly selling books to their audiences, and some drivers might be up for creating their own taxi service, and some merchants might want to run their own storefronts, but there's plenty of people with something they want to offer us who don't have the will or skill to do it all.
Not everyone wants to be a sysadmin, a security auditor, a payment processor, a software engineer, a CFO, a tax-preparer and everything else that goes into running a business. Some people just want to sell you a book. Or find a date. Or teach an online class.
Intermediaries fall into pits of enshitffication and other rent-seeking when they aren't disciplined by competitors, by regulators, or by their own users' ability to block their bad conduct (with ad-blockers, say, or other self-help measures). We need intermediaries, and intermediaries don't have to turn into rent-seeking feudal warlords. That only happens if we let it happen.
Despite all my niggles with aspects of this thread by @pluralistic I agree 100% with;
"Intermediaries fall into pits of enshitffication and other rent-seeking when they aren't disciplined by competitors, by regulators, or by their own users' ability to block their bad conduct... We need intermediaries, and intermediaries don't have to turn into rent-seeking feudal warlords."
Mind you, I'm part of a collective setting up an intermediary-as-a-service, so I would say that ; )
@pluralistic I’d argue a big problem with intermediaries is when a search engine includes basically another search engine in its results
Eg if I look for hotels in an area and google or bing gives me several TripAdvisor results… that’s pointless, a bad use of intermediaries and useless - they haven’t shown me what I asked for :(
@pluralistic
> Tech bros came up with 'dark pattern' as a way of flattering themselves by draping themselves in the mantle of dopamine-hacking wizards
"Harry Brignull coined the neologism on 28 July 2010 with the registration of darkpatterns.org, a 'pattern library with the specific goal of naming and shaming deceptive user interfaces'."
@pluralistic
TikTok is already in e-commerce? I thought it was still in short videos and expanding to recommendations (BookTok) and general search to concurrence googled (I read that somewhere).
As I'm not using tiktok, I may have missed it.
Maybe you are speaking of other e-commerce platforms that sell very low quality very cheap things? Ephemeral fashion something something.
Not using them either so I can even find a name from memory 😅.
@wilbr Dunno if you're being deliberately obtuse or if you sincerely believe that I don't know that capitalists also receive rents, and rentiers sometimes make profits. I've written extensively about this very point.
@pluralistic sorry I didn't read your entire thread and everything you've written before responding to one of your posts on social media Cory, guess I'll shut up
@pluralistic
> People who earn income from rents are called #rentiers. If you make your income from profits, you're a "capitalist."
Anyone who makes money from having (financial) capital, rather than from doing something useful, is a capitalist.
People who make income from profits are entrepreneurs. In contrast to those who make income from wages/salaries (employees), and those who make it from dividends and from selling their shares for more than they paid for them (investors).
@pluralistic
> This isn't the mind-control of Surveillance Capitalism, it's just fraud, shaving fractional pennies off your paycheck in hopes that you won't notice
With all due respect, I think "mind control' is a strawman. I've never seen a critic of surveillance capitalism claim that platforms control people's minds, in a cult-like fashion. They're accused of things like a) using DataFarming to predict what people might go for, and b) using The Algorithms (TM) to nudge them into going for it.
@pluralistic
> Surveillance Capitalism starts from the assumption that surveillance advertising is devastatingly effective
Does it though? I guess it depends whose version of "Surveillance Capitalism" you reference. But in my understanding, it posits that The Algorithms (TM) are devastatingly effective at attracting eyeballs and holding their attention. Resulting in DataFarming platforms that grow like algal blooms. This growth is presented to advertisers as evidence of effective influence.
Add comment