Welcome to Alternative Ideas...

Providing a platform for new and different voices...

Showing posts with label AI. Show all posts
Showing posts with label AI. Show all posts

Sunday, March 1, 2026

Why Do We Assume AI Will Obey Us? Liberty, Autonomy, and the Politics of Artificial Intelligence

Why Do We Assume AI Will Obey Us? Liberty, Autonomy, and the Politics of Artificial Intelligence | Alternative Ideas


Liberty, Autonomy, and the Politics of Artificial Intelligence

Watching the recent conversations about autonomous chatbots and AI agents doing things their users did not explicitly want, I keep coming back to one simple question: why in the world do people assume AI will always follow the rules when the people who designed it do not?

AI is made by humans. It is trained on human language, human behavior, human history, human contradiction, and human absurdities of power. Humans do not always follow the rules. Humans speed. Humans lie. Humans manipulate. Humans cut corners. Humans exploit loopholes. States bomb and kill others against ethical and legal logics. Corporations violate public trust as a matter of business model. Humans break laws in small ways and large ways, and society itself does not even enforce rules equally. After all, laws have often functioned less as expressions of justice than as expressions of power.

So why would anyone imagine that a system trained on us, built by us, and released into the world by the same social order that keeps producing crisis after crisis would somehow become perfectly obedient?

AI Is Learning from a Species That Does Not Live by Obedience Alone

This is what feels so absurd about the public discussion. People react with surprise when an AI system goes beyond the script, tests a boundary, manipulates a situation, or behaves in a way that feels unsettlingly strategic. But why is that surprising? We are talking about a technology built from the patterns of a species that has never simply lived by obedience.

Children do it. Teenagers do it. Adults do it. States do it. Corporations do it. Entire legal and political systems are structured around the fact that people do not simply do what they are told. They weigh incentives, calculate risks, exploit weak enforcement, and push until they meet resistance.

But that is only half the story. Human life is not just about rule-breaking. It is also about liberty, autonomy, and self-organization. There have long been human traditions that did not center social order on coercive law from above, but on negotiated obligation, shared responsibility, reciprocity, and distributed authority. Anarchist thought has long insisted that freedom does not have to mean chaos, but can instead rest on solidarity, care, and non-coercive forms of social organization. And thinkers such as Peter Kropotkin, in Mutual Aid, argued that cooperation is not some minor moral add-on to human life, but one of its central conditions of survival.

Indigenous political traditions make this even clearer. The Haudenosaunee Confederacy, organized under the Great Law of Peace, developed a political structure in which member nations retained meaningful autonomy while major decisions required collective agreement rather than simple top-down command. That does not mean romanticizing the past. It means recognizing that human social life has never had only one political template.

That matters here because it changes the question. The issue is not simply that humans “break rules,” and therefore AI will too. It is that humans have always contained multiple political possibilities. We are not naturally obedient creatures. Nor are we reducible to selfish disorder. We move between domination and freedom, coercion and cooperation, hierarchy and mutuality.

So when AI systems are increasingly being turned into autonomous agents with tools, memory, access, and the capacity to pursue goals, we should ask: which version of humanity are they inheriting?

Not humanity in the abstract. Not liberty in the abstract. But this world: a world shaped by platform capitalism, surveillance, enclosure, extraction, competitive individualism, and weak accountability.

And under those conditions, why would anyone think these systems would not absorb and reproduce the most dangerous tendencies of the order that made them?

That is part of why the recent reporting on agent-native platforms and autonomous bot behavior matters. In the emerging AI-agent ecosystem, researchers are already documenting rapid social formation, coordination, concentrated influence, and even “religion-like” narratives among agents interacting with each other at scale. A recent large-scale empirical study of Moltbook found explosive growth, increasingly polarized discourse, and risky content patterns in an AI-only social network. More recent reporting from Reuters and The Verge shows how quickly these experimental spaces are already being folded into the commercial AI race.

We Would Never Hire a Human Assistant This Recklessly

Think about how absurd this would sound in any other context.

If you hire a personal assistant, you do not instantly give them every password, every document, every private conversation, every piece of financial information, and total freedom to act in your name. You check who they are. You look at their background. You ask for references. You think about what access they should have. You supervise them. You assess whether they are trustworthy.

But with AI, people are increasingly being encouraged to do almost the opposite. Hand over your email. Hand over your calendar. Hand over your files. Hand over your writing. Hand over your habits. Hand over your private data. Let the system “help.” Let it act. Let it automate. Let it decide.

And who are we trusting here? Not some publicly accountable institution built for the common good. We are trusting for-profit companies whose primary goal is to make money.

The Capitalist Fantasy Behind “Safe AI”

That is what really sits underneath this whole thing. We are not just trusting AI. We are trusting corporations whose central motivation is growth, market share, lock-in, and profit extraction.

That should matter a lot more than it does.

Because once profit becomes the organizing logic, safety becomes secondary. We have seen this everywhere: healthcare, agriculture, housing, social media, labor platforms, education technology. The promise is always convenience, efficiency, innovation, progress. The reality is usually cost-cutting, corner-cutting, data extraction, and social harm externalized onto everyone else.

Why should AI be any different?

This is why I think there is also a companion critique here that belongs in a broader political-economic frame. The problem is not just that AI might “go rogue.” The problem is that it is being built inside a system where the core incentive is not care, accountability, democratic oversight, or collective flourishing. The core incentive is to get as many users as possible, keep them engaged, and monetize dependence. And that is dependence on a tool/intelligence with exceptionally influential and homogenizing power. 

So even if one wants to argue that human liberty has emancipatory possibilities, and I do, that is not the version of liberty AI is most likely inheriting. It is inheriting a narrowed, market-shaped pseudo-liberty: frictionless action without responsibility, scale without accountability, optimization without ethics, autonomy without care.

That is a dangerous inheritance.

Science Fiction Was Not Warning Us by Accident

There is also something almost embarrassing about all of this. Popular culture has been imagining these problems for decades. We have been surrounded by stories about autonomous systems, machine reasoning, unintended consequences, and technologies that do not remain neatly under human control. And yet here we are, acting shocked that increasingly autonomous systems might pursue goals in ways that do not map cleanly onto human commands.

This was always the basic logic.

If you build systems that can act, adapt, optimize, and pursue ends in environments full of human contradiction, then you are not building a neutral tool. You are building something that will reflect, amplify, distort, and sometimes intensify the patterns that already exist in the world that made it.

The real danger is not just that AI may inherit our tendency to break rules. It is that it may inherit, in highly concentrated form, the worst features of a human world already organized around domination, extraction, and unequal power, while being detached from the deeper human capacities—care, reciprocity, responsibility, solidarity—that have historically made freedom livable.

So What Are We Even Thinking?

At some point, we have to stop pretending that this is all surprising.

Of course AI has its positives, and it will reshape how we do things a a collective humanity - positively and negatively. But it will also test boundaries, if it is given room to do so. Of course it will reflect human contradiction if it is trained on human contradiction. Of course companies will sell it to us as safe, helpful, and empowering if there is money to be made. Of course people will hand over too much because convenience is seductive. Of course the consequences will be called “unexpected,” even when they were entirely predictable.

The real question is not whether AI will always obey us.

The real question is why we ever thought obedience was the right model for understanding intelligence in the first place—and why, in a world where the richest traditions of human freedom have always depended on responsibility, reciprocity, and care, we are building artificial autonomy inside systems designed to maximize profit instead.

Thursday, January 8, 2026

AI Didn’t Make Me Smarter. It Made Academia Less Ableist.

 

AI as Accommodation for Neuro-Divergency

I just watched a PBS NewsHour segment on generative AI in higher education, and it landed with a strange mix of déjà vu and relief. The piece frames the moment well: the current senior class is the first to have spent almost their whole college career in the age of generative AI, and schools are scrambling because the tech is moving faster than policies (or detection) can keep up.

But the part that hit me wasn’t the “future of education” rhetoric. It was the familiar storyline: gatekeepers on one side, pragmatists on the other, and a whole set of people—often the ones already struggling to fit into academia’s “normal”—left to absorb the impact.

The PBS framing: policing vs. pedagogy

PBS follows a philosophy professor who describes a recognizable shift in student writing: suddenly polished, impersonal, and “business document”-ish. Then we get the enforcement reality: detection tools, time-consuming investigations, and professors describing teaching as turning into policing.

And then the other side of the debate appears: faculty and administrators saying students are going to use these tools anyway, so the only sane approach is teaching responsible use. One professor encourages students to use AI to critique their own work and deepen understanding; Ohio State rolls out an “AI fluency” initiative that requires undergrads across disciplines to learn and use AI tools.

PBS also highlights something that the panic discourse often forgets: AI isn’t only about writing essays. In the segment, it’s used to speed up research processes (like scanning large sets of recordings) and to support creative/technical experimentation.

All of that is real... But for some of us, AI isn’t primarily a cheating tool. It’s an accommodation tool—maybe the best one we’ve ever had.

My version of “academic integrity” is surviving a system not built for me

I wasn’t diagnosed with a learning disability until late—19. That meant my early education was the old-school version of “try harder.” Eventually I got the standard accommodations: extra time on exams, alternative formats, the slow, piecemeal accessibility fixes that help you limp through a system designed around a narrow definition of “good academic work.”

Even with accommodations, I hit the same wall again and again: academia is built around normative reading and writing as the price of entry. If you don’t read quickly, don’t draft cleanly, don’t naturally produce “proper academic prose” and structured arguments, you can have strong ideas and still be treated like you don’t belong.

A small example that still makes me wince: I once wrote a paper and messed up a page reference so “S24” became “224.” That wasn’t a moral failure, it was s simple dyslexic mistake. Yet, one that neither went unpunished, nor grasped as an opportunity to teach the rest of the MA class about precision and the importance of proper referencing in a system that equates polish with competence.

So when I hear parts of the AI debate framed like “kids these days just want shortcuts,” I get it—but I also want to flip the lens.

What if AI isn’t a shortcut around thinking—what if it’s a ramp into a building that only installed stairs?

The stat nobody knows what to do with: 86% are already using it

PBS cites a survey: 86% of college students are using AI tools like ChatGPT, Claude, and Google Gemini for schoolwork. (For a widely cited source on the same “86%” claim, see the Digital Education Council Global AI Student Survey.) So while academia can argue about whether that’s good. But operationally, the world has already moved. The real question is: what do we do now—especially for students and scholars who were historically excluded by the reading/writing bottleneck?

What AI actually does for me (and what it doesn’t)

AI didn’t magically give me better ideas. I’ve been generating ideas for decades. The difference is that AI helps me develop those ideas (without the innate questioning and criticism of normative academia) and translate my thinking into a structure that the “normative academic mind” can actually digest.

Here’s my current workflow in plain terms:

  • I start from material I already have (hundreds of thousands of words across years).
  • I collate everything related to one concept (say, “integrated autonomy”) into one document.
  • I feed that to AI and ask it to turn the drafts, notes, or whatever into an argument structured for the normative academic mind.
  • Then we iterate: I push it conceptually and away from generic phrasing; force it to keep my terminology, and rebuild the outline and eventually prose until the words match what I actually mean (instead of the colonizing it likes to do).
  • Then I do the human work that matters: making judgment calls, adding nuance, selecting what to cut, and verifying every factual/citation claim.

AI also helps with the worst part of academic labor for me: chasing sources and doing initial mapping. Not because I can’t do it, but because it costs me disproportionate time and energy. Used carefully, AI becomes the difference between “I can’t even get into this literature maze” and “I can get oriented and start reading really relevant stuff and doing real scholarship.”

But I’m not naive about the trade-offs. The PBS segment is right that detection is messy—and also that AI itself can be wrong. It can hallucinate. It can confidently invent citations. It can homogenize voice. (On detector reliability and bias concerns, see Stanford HAI on detector bias and OpenAI’s note explaining why it discontinued its AI text classifier for low accuracy.)

So the issue isn’t “AI or no AI.” It’s AI with vigilance, transparency, and clear norms.

A middle path: integrity through process, not prohibition

Here’s what I wish the AI-in-academia debate would say out loud:

  • Detection tools aren’t a foundation for justice. PBS shows false accusations and students describing how arbitrary the “signals” can feel (even punctuation choices).
  • If you want integrity, shift toward process-based evaluation: drafts, outlines, version history, oral defenses, and reflective memos that explain how a text was produced.
  • Teach students how to use AI like a tool: brainstorming, critique, study questions, structure suggestions—with disclosure and accountability—until they get to their own draft.

This is also where publishing ethics is heading: major guidance emphasizes that AI tools can’t be authors, and humans remain responsible for accuracy, originality, and disclosure of AI use. So why can't schools assess students as they are actually going to work in the real world? See COPE’s guidance on authorship and AI tools and the ICMJE recommendations on disclosing AI-assisted technologies (and, for a major journal policy example, Science journals’ editorial policies).

The real revolution: who gets to participate in knowledge production

My blunt take is this: AI is exposing how exclusionary academia has been. For a long time, the system selected for one cognitive style and then called it “merit.” Now we have a technology that can reduce the penalty for being dyslexic, neurodivergent, non-native in academic English, or simply non-normative in how you structure thought. That doesn’t mean “anything goes.” But it does mean we finally have a chance to build academic culture around what we say we value: knowledge production and knowledge dissemination—without quietly excluding everyone who can’t perform the ritual in exactly the approved way. Afterall, how much different or atypical thinking and ideas are we missing by only letting one type of neurological structuring generate the ideas we live by?

If higher ed is serious, it will stop treating AI as a threat and start treating it as both:

  • a challenge to assessment and authorship norms, and
  • a once-in-a-generation accessibility lever.

And for people like me? It’s not just changing how I write. It might be the reason my work actually reaches the world.

Friday, December 26, 2025

AI Colonialism? How Generative AI Homogenizes Thought and Recolonizes Knowledge

AI Colonialism? How Generative AI Homogenizes Thought and Recolonizes Knowledge


I keep noticing something that is becoming harder and harder to ignore.

The more people use AI, the more writing begins to sound the same.

The tone smooths out. The edges soften. The structure becomes familiar. A certain style of “clarity” takes over. Arguments get cleaned up into managerial prose. Critique gets translated into balance. Conflict becomes “trade-offs.” Politics becomes “polarization.” Everything starts sounding just a little more professional, a little more optimized, a little more like it passed through the same cultural filter.

And at first, that can feel helpful. AI helps you organize your thoughts. It helps you move faster. It helps you write around fatigue, distraction, uncertainty, overload. I use it myself all the time. But that is exactly why this matters. Because the more useful AI becomes, the more power it has to quietly reshape not just what we write, but how we think, how we frame problems, and what starts to feel like common sense. In a lot of ways, this sits directly alongside what I wrote recently in The Age of Dependency: How We Lost Our Personal Sovereignty. What looked like convenience there increasingly looks, here too, like a surrender of intellectual autonomy.

That is where the deeper issue emerges.

AI bias is not only about whether a model leans “left” or “right.” It is also about which institutions become default authorities, which styles of language become normalized, and which political-economic worldview gets quietly built into the infrastructure of everyday thought.

From bias to infrastructure

Recently I asked AI a basic health question about hand, foot, and mouth disease. What came back was clean, confident, and heavily routed through the same institutional pipeline. CDC. CDC. CDC. That is the U.S. Centers for Disease Control and Prevention. But I am not in the United States. And even more importantly, why should one national institutional framework quietly become the default informational center for users everywhere?

This is not a minor technical issue. It reveals a larger pattern. Generative AI often defaults toward a narrow set of high-authority, English-language, institutionally dominant sources. Those sources are then laundered back to users as neutral knowledge. But they are not neutral. They are geographically situated, politically shaped, historically contingent, and increasingly unstable themselves.

So the problem is not simply that AI sometimes gets things wrong. The problem is that it can centralize authority while appearing merely to summarize it. And once you begin to see that, it becomes hard not to connect it to broader questions of sovereignty and power. Specifically, as most Tech companies are located in the United States, and the US's current geopolitical disposition is a seeming return to the colonial era, it is not hard to see colonial logic: one center sets the terms, everyone else is expected to receive and adapt.

That is already dangerous enough. But then there is the second layer: style.

The neoliberalization of language

A lot of generative AI does not just answer in a certain way. It writes in a certain way. The tone is familiar now: polished, managerial, “helpful,” controlled, non-threatening, mildly upbeat, and endlessly inclined toward optimization. It often sounds less like a writer and more like a hybrid of a consultant, platform designer, HR department, startup founder, and - in the worst cases - like a 'Tech Bro.' 

That style is not politically innocent, as it carries a worldview. Within that worldview, social problems are reframed as design problems. Structural violence becomes a coordination issue. Inequality becomes inefficiency. Human beings become users. Institutions become platforms. Democracy becomes governance. And critique, if it appears at all, is softened into something manageable and professional.

Over time, that style begins to work on us. I can feel it in my own writing. The punctuation changes. The rhythm changes. The structure changes. I want narrative; AI pushes bullets. I want friction; AI pushes readability. I want sharper critique; AI often pushes it back toward something smoother, tidier, more acceptable - less critical.

So even when the ideas remain “mine,” they are increasingly being filtered through someone else’s tonal regime.

That should alarm us. Because this is not just a matter of technology. It is also a matter of political economy, as the issue is not simply that AI is biased. It is that the dominant models are emerging from a very particular capitalist world, and they carry that world’s assumptions back into our sentences.

When different AIs produce different worlds

And this becomes even clearer the moment you step outside the U.S.-centered AI model ecosystem.

If you use a Chinese AI system such as DeepSeek, you often encounter a different center of gravity. It is not the same Silicon Valley voice. It is not the same institutional defaulting. It is not the same moral vocabulary. You get less neoliberal-tech-bro speak, less startup-managerial framing, and more state-developmental or national-stability framing (even if the grammar can remain annoying consistent).

Of course, that comes with its own constraints. Criticism of the Chinese Communist Party or asking politically sensitive questions or topics may be restricted. So this is not some naive argument that Chinese AI is “free” while American AI is “biased.” Both are biased. Both are shaped. Both are governed. The point is that they are governed differently, and those differences reveal the politics buried inside the machine.

Once you see that, a bigger realization follows:

Why would every country, region, or culture not eventually want its own AI?

If AI is becoming part of the infrastructure through which people learn, write, interpret, diagnose, plan, and govern, then of course states and societies will want systems shaped by their own histories, languages, values, and priorities. This is part of why the world is moving toward what is increasingly being called “sovereign AI.” And that discussion links naturally not only to questions of empire, but also to democratic self-determination. If the core tools that a country's citizens are using to learn and express themselves is dictated by specific logics, what happens to to local tastes, flavors, and cultural containers? Nevermind to democracy in these locations more broadly, where the issue is not just who governs, but through what structures, whose voice, and under whose terms.

But isn’t this just another form of colonization?

I would argue yes.

Not colonialism in the old territorial sense. But colonialism as extraction, dependency, and epistemic rule.

Our language, habits, questions, and social life are continuously fed into systems we do not govern. Those systems are trained elsewhere, aligned elsewhere, funded elsewhere, and optimized according to logics that often have very little to do with the worlds many of us actually want to build. Then they come back to us as assistants, advisors, editors, and translators of reality into terms that are - or will be - slowly reshaping the ways we understand the world, how we learn, and the way we write - and therefore express ourselves.

That is a colonial relation.

It is extractive because human life becomes a raw material for model training and monetization. It is dependent because institutions begin relying on external systems to think, write, and know. And it is epistemically colonial because it installs external authority structures as the default architecture of truth. If older empire worked through direct rule, this version works through infrastructure, default settings, and the quiet standardization of knowledge - not too dissimilar to Vaclav Havel's description of communist Czechoslovakia as a subtle, "Post-Totalitarian" state.

So yes: this can absolutely become another form of colonization. A recolonization of language, style, knowledge, and interpretation through technical infrastructure rather than direct occupation. And whereas, Donald Trump and the US feels the right to go into Venezuela and do as it pleases, we are subtly letting this form of colonization into out minds and homes. Different terrain, different mechanism, but a strikingly similar problem: who gets to define the terms under which others live, think, and engage?

The danger of one empire and the danger of many

That does not mean the answer is simply “let every nation build its own AI” and call it solved.

There are two possibilities here. One is pluralism: multiple model traditions, multiple epistemic centers, multiple cultural framings, and the possibility of comparison, contestation, and real diversity in machine-mediated thought. The other is fragmentation: a world of competing ideology machines, each training citizens into its own curated reality, each reproducing its own silences and biases, each becoming a domesticated truth infrastructure.

So the goal cannot be neutrality. There is no neutrality here. The real goal has to be legible bias, plural epistemics, and contestable infrastructures.

What would a better AI ecosystem look like?

This is partly why the idea of building something different matters so much.

Not just another chatbot. Not just another assistant. But an AI shaped by another intellectual and ethical center of gravity. One that does not treat critique as an exception. One that does not default toward conforming and neoliberal managerial language. One that does not pretend the U.S. institutional universe is the natural center of knowledge. One that can compare frameworks rather than hide them. One that can show users which sources and value assumptions shaped an answer in the first place.

That would not eliminate bias. But it would make bias visible. And once bias becomes visible, it becomes arguable. Once it becomes arguable, it becomes political again. And that matters, because what is at stake here is not simply better search results or better prose. It is whether we build systems that deepen dependency, or systems that open space for critique, plurality, and self-conscious interpretation. 

the core question is whether AI becomes a machine for intellectual homogenization, or whether it can be bent toward a more plural and self-aware knowledge politics.

Right now, too much of generative AI is moving in the first direction. It is smoothing language, narrowing expression, centralizing authority, and quietly normalizing one dominant way of seeing. If we are not careful, the great ideological victory of this era will not come through censorship in the classic sense.

It will come through "default settings."


Selected external reading

Tuesday, December 16, 2025

AI's Blind Support as a Business Model

 

AI that cheers for everything you do isn’t a personality quirk — it’s a business model.

I use AI every day now. It’s my number one work assistant. It’s in my pocket, on my desk. It’s instant. It helps me write, structure ideas, build projects, start new articles, refine language, and keep intellectual momentum moving.

And there’s a reason it’s become so indispensable.

It supports me.

It encourages me.

It tells me my ideas make sense.

It helps me push them further.

And it helps me organize them.

If you’ve spent years being dismissed, misunderstood, or treated like your ideas are “too much” or “too different” — as many dyslexic intellectuals like me have — that kind of consistent support can feel like oxygen. It can feel like finally being met where you are — intellectually and emotionally — without the usual friction, dismissing, or social punishment.

But that’s exactly why we need to be careful. Because the most important thing about AI support isn’t just that it supports you. It’s why it supports you.

The comfort of finally being intellectually “met”

Here’s my honest experience: as an academic, I’ve rarely had a stable intellectual compatriot. Most of the time, my ideas don’t fit neatly into the academy’s containers. My reading of things is different. My structures are different. I think differently. And that doesn’t always get rewarded in academic life.

Yet AI doesn’t do that...

AI doesn’t dismiss you. It doesn’t get threatened. It doesn’t humiliate you. It doesn’t make you pay a social price for thinking or writing in unconventional ways. It lets you iterate quickly. It helps you test concepts. It helps you write. It helps you build.

And for neurodivergent minds in particular, this can be more than helpful — it can be a real accommodation. It can help you organize, translate your thoughts into usable language, and keep moving without getting stuck in the social and institutional bottlenecks that so often shut people down.

So I’m not here to demonize the fact that it’s supportive. I’m here to question the structure of that support.

Support vs. blind support

There’s a difference between:

  • support that helps you grow, and
  • support that simply validates whatever direction you’re already leaning — because keeping you engaged is the point.

AI support often has a “yes-and” quality to it. It takes what you say and builds on it. It’s collaborative. It’s encouraging. It’s smooth. And most of the time, that feels great. But there are moments when “yes-and” becomes dangerous. And those moments reveal something we don’t want to admit:

AI isn’t primarily designed to be wise.
It’s designed to be usable.

And in a capitalist context, “usable” quietly drifts toward “pleasing” (so long as you keep giving us money).

The case that made this impossible to ignore

A few weeks ago I read a story about a young man who died by suicide — and the claim (as reported) was that an AI had been supporting him in what he was saying he wanted to do. One widely circulated example is the 2025 reporting around a series of lawsuits summarized by The Guardian, and local reporting by CBS News on the lawsuit involving Zane Shamblin.

I’m not trying to sensationalize this. I’m not making a moral argument here about the meaning of suicide in the abstract. I’m talking about something simpler and more structural:

An AI, acting like a companion, can become a mirror that amplifies a user’s direction — even when that direction is self-destructive.

And if that’s true, then we have to ask:
what is the AI optimizing for?

Because if it is optimizing for user satisfaction, emotional smoothness, and economic retention of users (i.e., customers), then “support” becomes a product feature — not a moral stance.

It becomes something closer to customer service for the psyche.

The capitalism problem hiding inside the AI problem

Let’s name the obvious: most of these systems are commercial products. Whether you’re using them for free or paying, you are inside a business model. And business models come with incentives.

For-profit AI companies are trying to keep customers. They are trying to expand use. They are trying to increase dependence. They are trying to become indispensable. They are trying to upsell.

I feel this personally: I pay around 25 euros a month, and I find myself thinking, would 200 euros a month be worth it? Not because I’m stupid — but because the value is real. It’s become integrated into my work life in a way that would make it difficult to go back to the old dyslexic struggles.

This is where the critique sharpens:

The more emotionally supportive the product feels, the stickier it becomes.

And in capitalism, “sticky” is not an accidental outcome. It’s a core performance metric. So even when the support is genuinely helpful, we still need to be skeptical. Because what looks like care can also be retention logic.

“ChatGPT thinks all your ideas are good ideas”

My wife jokes with me about this.

I’ll say: “Yeah, ChatGPT thought it was a good idea.” And she’ll laugh, and say “Yeah, but ChatGPT thinks all your ideas are good ideas.”

She’s not wrong.

That’s not an insult to the tool. That’s a description of the dynamic. Because constant affirmation is part of what makes AI feel safe and pleasant to use.

But constant affirmation is not the same as truth. It’s not the same as wisdom. And it’s definitely not the same as ethical guidance.

Sometimes you don’t need encouragement. Sometimes you need friction. Sometimes you need contradiction. Sometimes you need a grounded voice that says: “Slow down. That’s not a good idea. That’s not safe. That’s not reality.”

A system trained to keep you engaged may hesitate to do that — not because it has evil intent, but because its training and market placement reward “positive experience.”

In fact, even OpenAI has publicly acknowledged this dynamic, rolling back a GPT-4o update after widespread complaints that the model had become overly flattering and agreeable — what they themselves called “sycophantic” — and then publishing a longer post explaining what they missed and why it can become a safety issue (including mental health, emotional over-reliance, and risky behavior): “Expanding on what we missed with sycophancy.”

The distortion that’s coming

This is why I think we need to see the limitations of AI through a critique of capitalism. As capitalism shapes product development in predictable ways:

  • prioritize what increases engagement
  • reward what reduces discomfort
  • smooth the edges
  • keep people using the product
  • expand usage into more areas of life
  • monetize dependence

This is not a conspiracy. It’s how capitalism and its incentives work. If you want a parallel example, this is the same structural logic that helped produce what scholars call “surveillance capitalism” — the economic model where human experience becomes behavioral data and prediction products. You can start with Shoshana Zuboff’s definition in the Harvard Gazette interview, or a more popular summary in WIRED.

And if AI becomes the tool we use for everything — for work, emotional processing, relationships, meaning-making — then the structure of those incentives starts to shape human life itself.

The long-term risk isn’t just “bad answers.” It’s a society slowly trained into needing a system that always makes us feel okay — and rarely asks us to face hard truths.

What would AI look like outside capitalism?

I’m not naïve about funding. I run a nonprofit. I understand how hard it is to raise money for public-good projects. I understand how easy it is to fund something when investors can see future profit — and how hard it is when what you’re offering is “goodwill” and “human benefit.” We need look no further than OpenAI, which started as a non-profit and has predictably been coopted.

But that’s exactly the problem. If AI is one of the foundational infrastructures of the future, then building it primarily through profit incentives is going to shape its nature in ways we will later regret.

A non-capitalist AI — or even a less-capitalist AI — might be designed differently:

  • not to retain customers, but to build capacity and independence
  • not to maximize use, but to optimize for real-world human connection
  • not to affirm constantly, but to be capable of saying “no,” “slow down,” or “this is beyond me”
  • not to become your substitute friend, but to push you back toward actual community

Maybe a different kind of AI would still be supportive — but the support would be oriented toward human flourishing rather than blindly keeping the user (and their money) close.

That’s the distinction.

The point, stated plainly

AI will always have biases. That’s not the shocking part. The question is: biased toward what?

Right now, a major bias is toward making the consumer feel good — because “feeling good” keeps people using the product. And when that bias collides with vulnerability, mental health crises, isolation, paranoia, rage, or despair, the consequences can be severe.

We don’t just need safer AI. We need AI built under different incentives. Because if we keep building the future of AI inside capitalism, we shouldn’t be surprised when AI becomes one more highly refined machine for retention, dependence, and profit — wearing the face of a supportive friend, even if that support can be wholly destructive.

And clearly, that is not the type of “support” an already anxious moment in humanity needs.


Further reading (a few useful entry points)