Algorithms, amplification & regulation

Four distinctions this page refuses to collapse

The same discipline as the rest of this section’s own “four distinctions” device — kept apart because most public argument about algorithms fails by treating these as one thing.

1. 1

   Engagement-optimised is not the same as biased

   A feed ranked to predict what you will engage with is a business-model choice, not evidence of a political thumb on the scale. The academic record below shows engagement-optimisation systematically favours emotionally arousing and out-group content over neutral content — a real distortion, but a different one from partisan bias, and one that cuts across every political direction at once.

2. 2

   Personalisation is not the same as manipulation

   Ranking content by predicted relevance to you is what every recommender system is built to do, including ones nobody objects to (a shopping site, a music app). The question worth asking is not whether a feed is personalised — it always is — but what it is being optimised for, and who gets to know.

3. 3

   Virality is not the same as representativeness

   What spreads furthest on a platform is not a random or typical sample of opinion — it is the content that survived a filter for what gets shared. Out-group animosity is the single strongest documented predictor of social-media sharing found in this research; treating the most-shared post as "what people think" mistakes the filter’s output for the input.

4. 4

   Algorithmic amplification is not the same as editorial endorsement

   A newspaper that leads on a story has decided the story matters. A platform whose ranking system boosts a post because it is predicted to hold attention has made no equivalent judgement about its truth, importance or quality — which is exactly why the Online Safety Act regulates recommender systems as a harm-and-risk mechanism, not as an editorial one.

Five mechanisms, and what actually supports each one

Named and evidenced individually, on purpose — “the algorithm” is not one mechanism, and lumping five different, differently-evidenced claims into one verdict is how this subject usually goes wrong.

• Moral-emotional contagion

  Evidence: well established, contested effect size

  Why does outraged or moralised content spread further than neutral content?

  Messages containing moral-emotional language (words that are both about morality and charged with emotion) are retweeted at a higher rate than otherwise similar messages — in one large study, each additional moral-emotional word raised a tweet’s diffusion by 20%. The effect is stronger within a political in-group’s own network than across the divide, which is one documented pathway by which morally charged content can deepen rather than bridge political separation.

  Basis: Brady, Wills, Jost, Tucker & Van Bavel, "Emotion shapes the diffusion of moralized content in social networks," PNAS 114(28), 2017 (n=563,312 tweets on three US political topics) — explicitly correlational; the authors themselves call for controlled experiments to test causation on attitudes and behaviour, not just retweet counts.

• Out-group animosity as an engagement driver

  Evidence: well established, contested effect size

  What single content feature best predicts whether a political post gets shared?

  Across 2.7 million Facebook and Twitter posts from US news outlets and members of Congress, language about the political out-group was the strongest predictor of shares and retweets found — posts about opponents were shared about twice as often as posts about one’s own side, and each out-group word raised the odds of a share by 67%. This out-group effect was several times stronger than either negative-emotion language or the moral-emotional language measured in other studies of this kind.

  Basis: Rathje, Van Bavel & van der Linden, "Out-group animosity drives engagement on social media," PNAS 118(26), 2021 (n=2,730,215 posts). The authors state plainly that the design is correlational and that "it is unclear how algorithmic choices... might contribute to the amplification of out-group animosity, since social media companies are not transparent about how their algorithms work" — i.e., this paper documents what content people reward with engagement, not what a ranking algorithm then does with that signal.

• Engagement-optimised ranking as a business-model choice

  Evidence: weak

  Why would a platform’s algorithm favour emotionally arousing content at all?

  Feed-ranking algorithms are built to maximise predicted engagement — the metric platforms’ advertising business depends on — not to maximise accuracy, civility or representativeness. Academic reviewers describe content algorithms as an "attentional filter" that pre-selects the moral/emotional content people are already disposed to notice, "designed to increase engagement and profit for the platforms." A former Facebook product manager testified to the same underlying logic under oath: the company "programs its algorithms to maximize profits, which means it decides which speakers are heard and which are not," and Facebook’s own 2017 decision to weight "angry" reactions as five times more valuable than a "like" in its ranking formula is documented by investigative reporting on internal company documents.

  Basis: Brady, Crockett & Van Bavel, "The MAD Model of Moral Contagion," Perspectives on Psychological Science 15(4), 2020 (peer-reviewed theoretical review, not new experimental data on the algorithm itself). Frances Haugen, Written Testimony to the US House Energy & Commerce Subcommittee, 1 Dec 2021 (sworn testimony from a named former insider; not independently verified against the underlying internal documents by this research pass). Merrill & Oremus, "Five points for anger, one for a ‘like’," Washington Post, 26 Oct 2021 (investigative reporting on leaked internal Facebook documents; documents themselves not directly examined). Rated Weak because no source isolates the ranking algorithm’s own causal contribution from the entangled effects of human sharing behaviour, moderation policy and account/monetisation design — the mechanism is well-argued and consistent with insider testimony but not independently measured end-to-end.

• Filter bubbles and the "backfire" evidence

  Evidence: contested — credible evidence points both ways

  Does algorithmic personalisation trap people in one-sided information, and does removing it fix polarisation?

  The "filter bubble" thesis (Eli Pariser, 2011) argues personalised ranking narrows what people see to content that confirms their existing views, deepening division. The empirical record on this is genuinely split, and splits along method: studies measuring actual browsing/exposure data have found that individual choice about what to click narrows a person’s information diet more than Facebook’s ranking algorithm does, and that social/search channels can even increase exposure to opposing views compared with direct browsing. Separately, a large randomised experiment that switched real Facebook and Instagram users from the algorithmic feed to a plain chronological one during the 2020 US election found it substantially changed what they saw and how long they spent on the platform — but did not produce a measurable change in political polarisation, political knowledge, or belief in false claims over the three-month study. A 2024/2025 systematic review of 112 studies found the field’s disagreement tracks research method (network-based computational studies tend to confirm echo chambers; content-exposure and survey studies tend to reject the strong version of the thesis), not a converging answer.

  Basis: Pariser, "The Filter Bubble," 2011 (the originating thesis; a book, not an empirical study). Bakshy, Messing & Adamic, "Exposure to ideologically diverse news and opinion on Facebook," Science 348(6239), 2015 (n=10.1m users; Facebook-employee authors). Flaxman, Goel & Rao, "Filter Bubbles, Echo Chambers, and Online News Consumption," Public Opinion Quarterly 80(S1), 2016 (n=50,000 browsing histories). Guess, Malhotra, Pan, Barberá et al., "How do social media feed algorithms affect attitudes and behavior in an election campaign?", Science 381(6656), 2023 (n=44,764 across Facebook/Instagram; Meta-funded, several co-authors are Meta employees — disclosed here alongside the finding). Hartmann, Pohlmann, Wang & Berendt, systematic review of 112 studies, J. Computational Social Science, 2025 / arXiv:2407.06631, 2024 — finds the literature’s disagreement is methodological, not yet resolved either way.

• Social-feedback reward loop for outrage expression

  Evidence: well established, contested effect size

  Does getting likes for angry posts make people post angrier content again?

  In preregistered Twitter studies and behavioural experiments, receiving positive social feedback (likes, shares) for an outrage expression increased the likelihood a user would express outrage again — a pattern consistent with reinforcement learning. Users also matched their outrage expression to their network’s normative level of outrage, an effect that was strongest where the reward signal (feedback) was the main available cue and weaker where the network norm was already highly visible.

  Basis: Brady, McLoughlin, Doan & Crockett, "How social learning amplifies moral outrage expression in online social networks," Science Advances 7(33), 2021 (two observational Twitter studies, n=7,331 users / 12.7m tweets, plus two preregistered behavioural experiments, total N=240). The authors state the observational studies "cannot draw causal inferences" on their own (self-selection into extreme networks is a plausible alternative explanation) and note the behavioural experiments, while causally identified, use a simulated rather than real platform environment.

On the filter-bubble/backfire debate specifically. There is no responsible way to present the filter-bubble/backfire debate as resolved in either direction. The field’s own newest and most rigorous verdict on the question — a 2024/2025 systematic review of 112 studies — says the disagreement tracks methodology, not a settled answer. A single well-powered randomised experiment found real behavioural and exposure effects from changing the algorithm but no measured effect on the downstream outcomes (polarisation, misinformation belief) the filter-bubble thesis predicts should move: a genuine complication for the simple version of the thesis, not proof it is wrong.

Southport, 2024: the three days the law was silent and the algorithm wasn’t

A dance-class attack on three children, a false name, and the clearest UK case yet of the gap between what regulated broadcasters can say and what an unregulated feed will amplify — plus a live regulatory dispute, a year on, about how much of the blame that gap deserves.

Timeline

1. 29 Jul 2024A 17-year-old attacked a Taylor Swift-themed children’s dance class in Southport with a knife. First 999 call 11:47am; suspect arrested at the scene within ten minutes.
2. 29 Jul 2024 (same day)The false name "Ali Al-Shakati" began circulating on social media, originating from the site Channel3Now, alongside false claims the attacker was a Muslim asylum seeker who arrived by small boat.
3. 30 Jul 2024Two of the three girls attacked died (a third died in hospital the same week). By 3pm the false name had over 30,000 mentions on X from over 18,000 unique accounts. Merseyside Police publicly stated the name was incorrect.
4. 30 Jul 2024, eveningA vigil outside Southport Mosque was followed by violent disorder: bricks and rocks thrown at the mosque, a police vehicle set alight, a shop looted; 39 officers injured.
5. 31 Jul 2024 onwardDisorder spread to towns and cities across England and to Belfast; over 1,000 arrests followed nationally in the following weeks.
6. 1 Aug 2024Liverpool Crown Court lifted reporting restrictions and the suspect was formally named as Axel Muganwa Rudakubana — three days after the attack — the judge citing the risk that continued anonymity would let misinformation keep spreading in a vacuum.
7. 22 Oct 2024Ofcom CEO Dame Melanie Dawes wrote to the Secretary of State assessing how illegal content spread after the attack — the fullest primary regulatory document on the case.
8. 23 Jan 2025Rudakubana sentenced to life imprisonment (minimum term 52 years) after pleading guilty to all sixteen charges.
9. 3 Mar 2025The Law Commission opened a consultation on contempt-of-court law, prompted specifically by the Southport case, on whether more could be published after arrest to counter misinformation.
10. May 2025HMICFRS (the police inspectorate) published its Tranche 2 review of the policing response, finding "no conclusive or compelling evidence" the disorder was "deliberately premeditated and co-ordinated by any specific group or network" — directly qualifying the stronger framing in Ofcom’s October letter.
11. 6 Aug 2025Amnesty International published an analysis of X’s design and recommender-system choices in relation to the disorder, based on X’s own published recommender source code.

What was actually measured

• 30,000+ mentions, 18,000+ unique accounts

  The false name, on X only, within roughly 24 hours (by 3pm, 30 Jul 2024)

  Institute for Strategic Dialogue, "From rumours to riots" ↗
• 27 million+ impressions

  Posts stating or speculating the attacker was Muslim, a migrant, a refugee or a foreign national, on X, within the first day or two

  Marc Owen Jones, independent academic analysis (corroborated by Amnesty International) ↗
• 580 million+ views

  One named account’s (Tommy Robinson’s) total X activity over the two weeks following the attack, not all of it Southport-specific

  Amnesty International, "How X’s design and policies led to Southport-linked racist violence" ↗
• 260 million views, 4,379 replies promoting violence

  Six named X accounts tracked by CCDH, combined, in the week following the attack

  Center for Countering Digital Hate, "Fuelling Hate" ↗

Each figure above is its own organisation’s measurement, over its own time window, on its own platform — they measure different things and are not additive. A widely-circulated "155 million impressions / 1.7 billion reach" aggregate figure could not be traced to any primary source in this research and is deliberately excluded.

The online-vs-broadcast contrast

UK contempt-of-court and under-18 reporting-restriction law legally prevented anyone — broadcasters, the press, and in principle individual social media users alike — from naming the Southport suspect or speculating about him in ways that could prejudice a trial. Regulated broadcasters observed this restriction. An anonymous aggregator site and a large number of pseudonymous social media accounts did not, and a false name and false identity claims reached tens of thousands of mentions within a day and millions of views within the following days. A judge lifted the restriction on day three, citing the ongoing spread of misinformation as a reason. This is a case about the gap between regulated and unregulated space, not a case about broadcasters being "slow" for its own sake — nothing in this case suggests broadcasters breached their existing due-impartiality or due-accuracy duties.

The three levels, and where the dispute actually sits

What this case study cannot be used to conclude

• That "the algorithm" specifically — as opposed to a platform’s moderation policy, its verification/monetisation design, or individual users’ own sharing choices — was the cause of the false name’s spread. Ofcom names "virality and algorithmic recommendations" as a factor; Amnesty’s report analyses X’s recommender code specifically; neither isolates the algorithm’s contribution from these entangled design choices.
• That online misinformation was the cause, or even the primary cause, of the riots. HMICFRS’s statutory review and the Children’s Commissioner explicitly pushed back on that "prevailing narrative," citing deprivation, distrust of police and political disaffection as more complex underlying drivers.
• Who specifically originated the false claim, or why. The one investigation into this (Pakistani police, against a Channel3Now-linked individual) was dropped for lack of evidence.
• A precise reach total. The individual figures above measure different platforms, accounts and time windows and are not additive — and a widely-circulated aggregate figure could not be traced to any primary source and is excluded.