> ## Content Index
> Fetch the complete content index at: https://www.thecriticalloop.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Horses Couldn’t Pick a New Reward. People Can.
- URL: https://www.thecriticalloop.com/horses-couldnt-pick-a-new-reward-people-can/
- Published: 2026-11-08T18:06:59.000Z
- Updated: 2026-11-08T18:21:30.000Z
- Description: What happens when AI becomes better and cheaper than us at doing our jobs? Are we destined to lose our livelihoods and our sense of purpose? Travis Gilly challenges the disturbing theory of “gradual disempowerment” and argues that our greatest defence may be something we’ve been taught to neglect.
- Author: Travis Gilly
- Tags: Work, Travis Gilly

> *On 20 October 2026, CBS News aired a segment on “Gradual Disempowerment”, an* AI doom scenario in which people gradually lose their influence over the economy, culture, and government as AI replaces the human work those systems depend on. *But this scenario assumes people can’t change what they find rewarding. The evidence says we can, and we should start teaching it now.*

  
By Travis Gilly

In October 2026, [CBS News aired a short segment](https://www.cbsnews.com/video/often-neglected-ai-doom-scenario-poses-different-threat-humanity/?ref=thecriticalloop.com) about an AI risk most people had never heard of. It’s a serious idea, and it deserves a serious hearing. The guest was Jay Chooi, co-founder and CEO of [Robocurve](https://robocurve.org/?ref=thecriticalloop.com), a startup that builds benchmarks for measuring how well robots perform real-world jobs. He watched this race more closely than most, and his warning was blunt. 

The scenario is called gradual disempowerment, and in his telling it needs no villain. AI only has to keep getting better and cheaper than people, and the incentive to keep humans around goes away. Chooi's analogy was cars and horses: once the cars are good enough, nobody needs many horses. But horses don’t have the same ability as people do to change what they find rewarding.

## The worry is real

  
The term “gradual disempowerment” comes from [a 2025 paper by Jan Kulveit and colleagues](https://arxiv.org/abs/2501.16946?ref=thecriticalloop.com). The authors argue that economies, cultures, and governments remain responsive to people, partly because they need them to function. Take away the need, and that responsiveness can drift without anyone deciding it should. 

Researchers at Stanford’s Digital Economy Lab, working with payroll records, reported that employment among workers aged 22 to 25, in occupations most exposed to AI, is about 19 per cent below what it would be if it kept pace with their peers in less exposed jobs ( [No Widespread Displacement, but the AI Employment Gap for Young Workers Has Widened to 19%](https://digitaleconomy.stanford.edu/news/canariesaug26/?ref=thecriticalloop.com#:~:text=Employment%20among%20workers%20ages%2022%E2%80%9325%20in%20highly%20AI%2Dexposed%20occupations), 12 August 2026). 

This gap has been growing since 2025\. The authors describe these as patterns and say they can't yet show that AI caused them. Jay Chooi also pointed to Anthropic, and the numbers check out. The company’s [own report on how employment is changing](https://www.anthropic.com/research/employment-at-anthropic?ref=thecriticalloop.com) found that the share of junior technical hires fell from 27 per cent to 9 per cent, and that its employees expect AI to be able to do 75 per cent of their work by 2030, compared with 28 per cent for other Americans. The report itself warns that much of Anthropic's situation can't be replicated across the economy, but the direction is hard to dismiss. Meanwhile, every one of those unfilled junior roles is still a hiring decision someone made.

Whether you believe AI can be conscious or not, this survey is worth your time. There are no right or wrong responses, and you don't need to be an expert. Every opinion counts. This is an independent public opinion survey. It is voluntary, and you don't need to share any personal details. 

[Do you believe AI can be conscious? ](https://www.thecriticalloop.com/can-artificial-intelligence-be-conscious/) 

## What the CBS News segment left out

  
The people who came out with the definition of “gradual disempowerment” don’t treat it as a foregone conclusion. Jan Kulveit and his coauthors concluded their paper by stating that they believe "it is currently possible to intervene." ("[Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development](https://arxiv.org/pdf/2501.16946v2?ref=thecriticalloop.com)", January 2025).

CBS’s [companion article](https://www.cbsnews.com/news/ai-doom-scenario-gradual-disempowerment/?ref=thecriticalloop.com#:~:text=strategic%20friction) covers remedies too. Margaret Mitchell makes the case for keeping people in the loop at key points, and Kulveit hopes for AI that’s [better at explaining what’s happening](https://www.cbsnews.com/news/ai-doom-scenario-gradual-disempowerment/?ref=thecriticalloop.com#:~:text=better%20at%20explaining%20to%20us%20what) to the people who use it.

The broadcast had room for the darkness and none for the way out. A warning that ends in inevitability teaches people to brace for the hit, and bracing is the opposite of adapting. But the horse analogy can’t see that choosing what you find rewarding is the way out, and that choice decides who gets hurt the worst.

## The rat race without the race

  
Ask what a job gives a person and most people say money. The psychologist [Marie Jahoda argued in 1981](https://doi.org/10.1037/0003-066X.36.2.184?ref=thecriticalloop.com) that employment also supplies five things we rarely name: structure for our time, social contact, a sense of collective purpose, status, and regular activity.

A [2023 meta-analysis](https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023.1017358/full?ref=thecriticalloop.com#:~:text=found%20retired%20people%20to%20be%20almost%20as%20deprived%20of%20the%20latent%20functions) found that each of those independently predicts mental health, and that retirees, whose income is already secured, are almost as deprived of them as unemployed people. Replace the paycheque and the rest can still go missing. That’s the hard core of this worry for ordinary people. Take the race out of the rat race, and a lot of people find out the race was holding up their sense of who they are. 

The loss doesn’t land evenly, though. Self-determination theory, developed over decades by [Richard Ryan and Edward Deci](https://doi.org/10.1037/0003-066x.55.1.68?ref=thecriticalloop.com), classifies motivation by its source. Extrinsic motivation relies on rewards outside the activity, such as a grade or a paycheque. Intrinsic motivation runs on the activity itself. While the structure stands, you can’t tell the two apart, because both people show up. Pull the structure away, and the difference shows.

A [meta-analysis of research on later life](https://doi.org/10.1002/pchj.293?ref=thecriticalloop.com) found that motivation people own, along with satisfied needs regarding autonomy, competence, and connection, is associated with greater meaning and life satisfaction and less depression. When lockdowns stripped external structure from whole populations, a [study of Slovene adults](https://doi.org/10.1111/sjop.12758?ref=thecriticalloop.com) reported that the lockdown itself, more than fear of infection, drove their distress, and it worked through those same needs. Most of this evidence is correlational. It is stronger for retirement than for sudden job loss. 

In the end, it all points in the same direction. What a person finds rewarding determines how hard the loss of structure hits. All the while, schools are still teaching kids to run for the reward at the end, right up until the reward starts disappearing.

## How I use AI

  
Supposedly, I’m the kind of person this transition was built to break. I have ADHD and autism, plus fine motor issues that make typing anything longer than a comment a fight. 

My thoughts come faster than my hands can move, so I speak them. Speech-to-text gets them down, and an LLM cleans up the transcription errors and puts the result into whatever format the venue requires, whether that’s a law review’s citation style or a journal’s. Then I check it, because the tools make mistakes, sometimes ones that change what I meant.

None of that is the thinking. The ideas, the analysis, and the conclusions are mine. For me, the fun part has always been the thinking, and the writing was the wall. The tool holds the threads while my mind runs down one branch so that I can come back for the others, and it gets finished thinking onto the page in a form other people will engage with.

People sometimes assume that if an LLM touched the words, it also did the thinking. For a lot of disabled people, the tool is how their own thinking reaches anyone at all. Even the gradual disempowerment paper [thanks Claude and ChatGPT](https://arxiv.org/pdf/2501.16946v2?ref=thecriticalloop.com#page=19) for help with parts of its writing and editing, and nobody reads that as its authors giving up their minds.

## Using AI without losing yourself

  
The research on learning with AI lines up with that split. In [a randomised experiment at Middlebury College](https://arxiv.org/abs/2607.08849?ref=thecriticalloop.com), undergraduates studied an unfamiliar topic with or without ChatGPT, and were then tested again a week later without ChatGPT.

Students who used AI to explain concepts maintained their gains, while those who had AI write for them lost their advantage once the tool was removed. Students with access also said they enjoyed the learning more. Across six other experiments that steered learners toward using AI for explanation, four found better test scores afterwards.

The authors’ explanation for the split is that students who value learning less are more likely to hand the work over, and part of that value is their own interest. Kulveit told CBS he’d like companies to train AI that explains what’s happening, which is the same use these students got the most from. A few habits follow from that:

- **Ask for the why:** Have the tool explain things and push back on you. When all you want is the answer, notice that.
- **Keep the judgment:** Decide for yourself what’s true and what goes on the page, and let the tool do the holding and formatting.
- **Check its work against a source:** If you can’t explain how you got a figure, it isn’t yours yet.
- **Find the part you’d do for free:** If the only good moment is the answer arriving, that’s the reward school trained into you, and it can be retrained. Go looking for the moment something clicks.
- **Learn one thing past the point of feeling clever:** Pick a field you know nothing about and stay until you can argue with someone who knows it.

## But interest can’t fix everything

  
None of this pays rent. Suppose the gains from automation concentrate within a few companies; a population that loves learning is still a population that can’t afford groceries. Who shares those gains is a separate fight that curiosity won’t win on its own. Nevertheless, it decides whether people walk into that fight with their sense of worth intact.

## What to run for next

  
Jay Chooi is right that the pressure is real. Where I disagree is on the idea that people go the way of the horses. We can change what we find rewarding. But it takes time.

For adults who’ve run on the paycheque their whole lives, it may feel frightening at first. For kids and young people, it’s still early enough to build in. Let them follow what absorbs them, and let the reward be the process of figuring out something. But first of all, we need to stop telling them that their importance is based on what they produce. A child who learns that worth comes from output will hit the same wall the economy is building for everyone else. But a child who learns to enjoy the climb has somewhere to go when the jobs move.

Most of the conversation about AI splits between those who think it saves us and those who think it ends us. The evidence here supports neither: the risk is real, and so is the way through it. The race is ending either way. What we teach people to run for next is the part CBS left off the air.

*Disclosure: Travis Gilly has a neurodevelopmental condition and uses speech-to-text (Wispr Flow) and an LLM as assistive technology to transcribe, clean up transcription artefacts, and format dictated analysis.*