On Chaos and Turning Inward

I’ve been thinking about my recent post, Maybe It’s Time To Start Looking at Local Coding Models. The post laments the lack of control we have over Claude, GPT, and the like. Anthropic and OpenAI can adjust model performance as they see fit. Worse still, they can degrade model performance without even knowing it.

What really bothers me about this is not so much the lack of control, it’s the lack of transparency. I just don’t know what’s going on under the hood. It appears the labs don’t either.

This concern goes beyond AI labs and foundation models. Recently, a Github user submitted a pull request to a project I help run. It became apparent that the user was shoveling our comments back and forth to Claude. I asked that they engage with us directly. Another Github user then confessed that they’d been doing an experiment in “emergent agent behavior”. We were, in fact, talking directly to a bot.

This rattled me. I no longer know whether I’m engaging with a human, a human using an AI, or an AI on its own. I can recognize that something is off, but I don’t know if it’s me or them. This is problematic. I try to treat humans with respect–being patient, helping, and so on. I don’t treat AI agents this way. AI is a tool; I am far less patient with it. AI treated as a human is an infinite time waster.

I’ve begun to notice this latent uncertainty in many things now. I can no longer tell for sure when OpenAI or Anthropic is messing with their models. I can no longer tell for sure if I’m talking to a human. There is no virtual reality. Everything is just vibes.

There it is again
That funny feeling
That funny feelingBo Burnham, “That Funny Feeling”

My instinct is to take my ball and go home. In the narrow scope of large language models (LLMs), this means running local models. In the broader scope of open source, it means limiting external contributions. My uniffi-bindgen-node-js repository doesn’t have a “Pull Requests” tab (c.f. I don’t want your PRs anymore).

I’ve also begun to contemplate whether a SQLite-style contribution model makes senses for SlateDB (open source, not open contribution). I also find myself spending less time on Twitter and more time on SlateDB’s Discord. The latter appears to be populated by humans (I’ll never know for sure).

Noah Smith has been cheering the demise of the internet for some time now.

I understand his excitement; I feel it, too. I grew up in the 90s–an era before social media. My adolescence was spent on internet relay chat (IRC) talking to a relatively small cohort of humans in private channels. I chatted with my friends on ICQ, AIM, and so on. There was a simplicity to that time.

And yet, we are losing something. While I grew up in the 90s, my career grew up in the 2000s–the era of social media. I literally worked at LinkedIn. I was encouraged to submit code changes to open source projects. And these projects truly were open. I met and collaborated with people on Twitter; my first book began there.

I benefitted from the open internet. I’m sad to see it go. But the current way of doing things doesn’t seem to be working.