We need better ways to manage human intent

We live in a period of unprecedented information abundance. LLMs are generating new content at a breakneck pace.

For the sake of argument, let’s group this new information into two categories:

1. Content that represents a human’s original intent - a decision about how something should be, or how it should be accomplished.
2. Everything else - content derived from the intent, summarizing it, planning how to implement it, rephrasing it, making it understandable by machines, slop, etc

If you give an LLM a prompt with your intent, it can easily generate a ton of content. As a result, human intent is getting lost in a tidal wave of other content. As we integrate AI into our society, this signal-to-noise ratio will only get worse.

But intent alone represents human decisions. Intent is how we should be shaping products, guiding technology, running our businesses, and governing our societies.

Unfortunately, protecting intent may be harder than we think.

Thinking about intent

So what exactly does intent mean?

For now, let’s assume say intent is the original goal or purpose of a human decision. It’s the message a person is trying to communicate in its simplest form. Let’s also focus only on written text intent, and assume that every word or so either represents human intent or it doesn’t (later in this article we’ll add quite a bit more nuance.)

When we write something down, we can encode that intent. Some of the content we create will be almost entirely intent, or intent dense:

Other text is only partially intent. We often pad our intent for the sake of clarity, politeness, and a million other reasons.

This non-intent content is a mixed bag. It contains:

• Content derived from the original intent, like LLM outputs. The intent was fed into some intelligence, and it used that intent to create more stuff. More on this category in a moment.
• Extraneous content that may have had a purpose at some point, but is now useless
• Politeness, filler words, explainaints, etc
• Nonsense, garbage, and slop

There has always been plenty of cruft and derivative content in writing. But writing used to take human effort, so someone wrote something down, they usually sort of meant it.

Now that we have LLMs, this isn’t true anymore. LLMs create enourmous amounts of plausible content. The effort required to create text has dropped to basically zero, and so we can expect a lot more of it.

For example, here’s a simple prompt that could easily produce thousands of lines of code:

Derivative content is important

Just because a piece of text isn’t intent doesn’t mean it’s not important. AI slop is certainly derivative content, but not all derivative content is slop.

For example, computer code is derivative content. We need code to run our programs, otherwise our intent has no effect.

The ability to generate derivative content is precisely what makes LLMs so powerful.

Plenty of important human-written content is derivative content too. Laws are a form of derivative content, because the intent of a law is generally not exactly the written law. Arguably everything we write or say has some degree of intent and some degree of derivative nature.

But derivative content is a function of intent

Let’s imagine intent as an argument to a function, which returns derivative content.

For example: at some point we all learned to write professional emails, and stored that knowledge in our brains. Now, whenever we need to send a message like this…

I need the TPS report

…we can run it through our social filter, and write a nicely-worded professional email:

Hey Bob, just checking in on the status of the TPS report. I wanted to get that out sooner than later if possible.

We could write this process as a software function call:

politeEmail = professionalismFilter(message);

The email will change, but your skill at writing professional emails mostly stays the same.

As long as you have 1) intent and 2) the professionalismFilter skill, you can write a good email. It doesn’t matter if you lose an old email and need to write a new one, it’ll be easy since you have both the intent and the skill.

We could use this format for anything humans make in a predictable way:

// Painting a picture
mural = artisticSkill(intent);

// Cooking a meal
dinner = cookingAbility(intent);

// Writing this blog post
post = writingStyle(intent);

Or more generally:

result = contentGenerator(intent);

Some functions are deterministic, meaning they always return the same output.¹.

// Deterministic function
const // Non-deterministic function
  translation = translate(originalText); // this has multiple correct answers

In a perfect world, every function would perfectly capture the original intent. But in reality, some functions are capable than others.

const program = gpt2(intent); // <= not so good
const program = opus4(intent); // <= much better

LLMs are functions too. When we want to generate content, we call the LLM with our intent as an argument:

derivativeContent = LLM(intent);

As long as you have intent and a reusable function, you can always generate more derived content.

Therefore, only intent is precious

This means that if we can easily derive content based on intent, then the long-term value of the derivative content is low. What cannot be derived, by definition, is the intent itself.

Derivative content is disposable because as long as you have the generation function, you can always re-generate anything that is not intent.

In fact, since AI models are always improving, regenerating content may actually be better than keeping the old generated content around. If you pass the same function to a better function, the derivative content will be better than the old version.

This is why intent is precious. Losing intent means losing a decision that a human made for a real human reason. Once it’s gone, there’s a vacuum that a model will fill with whatever plausible hallucination it can.

At the end of the day, only intent really matters.

How intent gets scattered

So how do we manage our precious intent?

These days, not very well. Intentful content is haphazardly scattered everywhere, and AI is making it exponentially worse.

A vibe coding example

Let’s say you’re building a new feature in Claude Code. You start with a prompt:

So far, so good! There’s a little bit of cruft, but the intent-to-noise ratio is still high.

Let’s zoom out and imagine our prompt inside a context window:

Once we hit enter, our agent reads our prompt and calls a skill.

Are skills intent? Well, it depends.

You may have carefully crafted your skills. Or they may have been downloaded from the internet and never read, included by the agent harness, or generated by the agent in a previous session.

None of these are inherently bad, but none of them are necessarily intent. As a result, the overall percentage of intent in the context window is probably diluted.

Now, the agent asks some questions:

The agent’s questions are probably not intent, but your answers are.

Each time you respond to the agent, you add more valuable human intent. And each time it replies, it adds more derivative content to the context window.

If you’re in plan mode, the agent will write a plan next:

The plan is content that the agent derives from your original previous intent. It restates the intent you already shared, and adds new ideas of its own.

When you approve the plan, you’re signaling intent. But how much? It depends on how well you read the plan. Approving a plan you haven’t read doesn’t signal much intent at all.

Once you approve the plan, the agent starts writing code:

Is source code intent? It certainly can be. To me, good code is written to be a concise, relatively human-readable expression of intent.

Realistically though, most code written by humans is not written this way. And code written by an LLM may represent intent from the plan or the prompt, or it could be complete slop.

Agents do more than edit source code. They also touch other files:

• Automated tests
• Readmes
• Documentation
• Agent prompts (AGENTS.md and CLAUDE.md)
• Commit messages
• Etc etc

Originally each of these may have had high intent, or it may have been complete slop. Either way, it now has more derivative content.

Finally, we end the session. The files in the repo are committed, but the original prompt/chat is saved locally or discarded. Even if you commit your plans to the codebase, those plans are not necessarily high-intent.

So where is the intent?

What happened to our original intent? Well, it’s still here, but it’s been scattered to the winds.

• Intent lives in a lot of places. See how scattered our yellow boxes are? Intent is dispersed everywhere. Intent is not just the prompt: it’s in the code, the plan, the tests, the docs, and everywhere else too.
• We actually don’t know what text is intent, and what is not. Our skill could be intent, or it could be slop. The source could be intent, or it could be slop. Even if most of the code was written by a human with intent, there’s no way for us to know if that’s true anymore. Once this is shared with other people, any memory we have of our original intent is lost.
• Intent is not preserved, it gets lost when we save and share our code.
• All the places we save intent are also being edited by AI. There is no protected source of truth for intent in this example. Source files do contain intent, but also slop. Tests contain intent, but also slop. All of our files have some intent (some yellow boxes), but none of them inherently represent intent on their own.

It’s not just code

This effect isn’t limited to engineering. OKRs are now a mix of leadership and ChatGPT suggestions. Even US tariff policy might be AI slop.

All the information in the world now has the same problem: it is increasingly being diluted by AI from lost prompts, multiple steps removed from any human decision.

If you’ve ever received AI slop from a coworker, you know how frustrating low-intent content can be. The problem isn’t that a human didn’t write it—it’s that the documents are deeply ambiguous about their intent.

Was there a shorter prompt that better describes what they really meant? If so, why not share that instead? Or is this a summary of a meeting transcript, which had key intent that is now missing? We just don’t know.

The properties of intent

So if intent is so precious, how do we protect it? First we’ll need to learn how to recognize it.

There’s a philosophy of intent that can help us understand it better.²

Intent is not self-evident

You cannot tell whether something is human intent by reading it.

Let’s say we have a readme file with the following instruction:

Use vitest for unit tests

Maybe a human typed that instruction. Or, maybe a code agent wrote it after a human asked it to clean up the repo.

Without more information, there’s no way to know.

Intent is not always written by humans

Suppose we could track whether a human wrote every single piece of text in the world with some kind of AI watermark.

Would that help us identify intent? Probably not.

Consider this agent chat:

In this case, the user intent is “use dark blue for all links”. But they never actually typed “dark blue”; it was generated by the LLM.

This was true before LLMs too. If your coworker writes an email, how do we know they didn’t write it simply because their boss told them to? If they did, how sure are we that they really understood the boss’s original intent?

Just because someone typed it doesn’t mean it’s intent.

Intent is not always natural language

Code can be intent, too. Consider this React component:

<Banner>The system will be offline for maintenance from 10pm to 2am</Banner>

Did a human decide this banner was important based on usability tests? Or did an LLM add it without anyone knowing? There’s no clear way to tell.

Intent can be ambiguous

Take this phrase:

Do not delete this file

What is the intent?

1. Does the file contain important intent that could easily be moved somewhere else?
2. Does the file itself represent something important, like an artifact with historical significance?
3. Does the file itself needs to stay on disk to prevent breaking a build, and its contents don’t matter?
4. Is the intent more general, something like “all file deletions need to be approved by a person”?

Intent can be large or small

Intent is often much smaller than the derivative content it produces. This makes sense, since asking an LLM to build a simple feature can produce thousands of lines of code.

User: “Make me a photo-sharing app”

Agent: Sure!

Agent: <10,000 lines of code>

But humans are complicated, and their intent can be complicated too. Sometimes it’s useful to condense what we believe into something short and easy to share.

For example, take the following statement that represents intent:

We believe mobile devices and social media platforms seem to be doing harm to kids and interfering with their education. But we also don’t want to make draconian rules that interfere with the complex social dynamics of teenagers, or create extra work for teachers and administrators who would have to enfore them. We’re still learning how technology interacts with young minds, but we want to take some reasonable action and monitor the results.

This statement may be true, but it’s long. When put through a policy-writing filter, it might be condensed into something easier to understand:

No phones in schools

Intent may be declarative or imperative

Let’s take this statement:

Do not expose private customer data

Do we mean that we shouldn’t expose it right now, or that it shouldn’t be exposed in general? Assuming the latter, we may want to phrase it as:

Customer data should never be exposed

The passive voice matters here. The rule isn’t specific to whoever reads this task. It applies to everyone. It’s a policy, not a directive.

Intent can be implicit or explicit

Let’s imagine sending a text to a friend:

Hi Casey! I’m in town this weekend, let’s grab lunch.

You’ve now encoded at least two kinds of intent:

1. The explicit message: “Let’s grab lunch”
2. The implicit message: “I value your friendship enough to write to you and spend time with you”

Arguably the second is more important, but the text never appears in the message itself.

Not all intent is equally important

In this example, the reply represents very little intent:

“Do you want fries with that?”

“Um, sure I guess…”

Here, the intent is stronger:

“Which logo do you prefer?”

“I think option #1 fits our brand better”

Finally, this answer represents very high intent:

“Do you take this person as your lawfully wedded spouse?”

“I do.”

There is also the notion of oblique intent, or intent that we are aware of and accept but don’t explicitly want or state. From Wikipedia:

Ted … is aware that [smoking] helps him to deal with stress and that it causes lung cancer. His reason for smoking is to deal with stress… increasing his risk of lung cancer is obliquely intended.

In software development, an oblique intent could be approving an LLM suggestion to use a particular software library. It’s understood and agreed on by the user, and therefore intended. But it wasn’t explicitly asked for (or even preferred), and therefore could be changed in the future without violating the original intent.

Intent has an expiration date

Let’s say you prompt an AI to add brand colors to your design system.

Use green and yellow for our brand colors

A week later, you decide you want to use blue instead of green.

Is the original text still intent? Not really. Those are the wrong brand colors now. The original prompt no longer reflects what anyone actually wants.

You could argue that it represents what your intent was at a point in time (like an immutable data source), but it would have to be rephrased like this:

Our brand colors are blue and yellow (they used to be green and yellow)

This intent is valid, but could easily change tomorrow:

The price of a Pro subscription is $99 per month

While this one is probably permanent:

Personal information about customers (like contact information and health records) are private and must be kept secure

I’m not sure, but I have a feeling that good intent is durable. If intent describes a truth in the real world, and things that are true today are likely true tomorrow, then the intent will be long-lived.

Intent is not top-down

Leadership may see their role as sharing their intent to their teams. Mission statements and OKRs ideally encode strategic intent.

But intent comes from everyone, at every level. What level of downtime is tolerable for this service? What did we learn from that usability test? All of it matters, and needs to be captured.

Organizations may also design communication so intent trickles up.

Manager: Be smart and make good decisions.

Worker: OK, we should switch to paper packaging.

Here, the manager’s intent concerns the organization and its people, but the worker’s intent has more direct effect on the product.

Schrodinger’s intent

If an AI successfully copies intent but it’s never reviewed by a human, is it still intent?

Let’s say an HR manager writes this:

Please create a compensation plan for the company, with fair salaries for each employee based on their role and experience

Now our AI generates a compensation plan—but nobody ever reads or approves it. Does this count as intent?

The AI may have correctly guessed the intent, but it also may not have. If it did, nobody’s seen it yet. Until we open the box and observe, we don’t actually know if we preserved intent.

Basically, intent that’s not verified by a human doesn’t count.

Garbage in, garbage out

Let’s say we have a hypothetical function that preserves intent perfectly. Can we trust what it creates?

Perhaps, but only if we trust the intent we put into the function in the first place.

What if the intent is someone’s genuine belief, but they’ve been misinformed? What if it’s morally reprehensible? What if they’re just lying? Malicious intent is still malicious, and ignorance is still ignorance.

Good information systems won’t prevent evil people from doing bad things. Intent can’t just be preserved. It also has to be good. ³

Intent is context-dependent

One person’s intent is another person’s derivative content.

Let’s imagine a web designer who gives this instruction to an agent:

Please move the “Log in” button to the left

Is this actually their intent? Maybe they don’t care about the exact position, as long as it looks good and fits the rest of the design. The real intent could be:

Ensure this button’s position is aesthetically pleasing and consistent

But it doesn’t stop there. Are aesthetics and consistency still important if users enjoy the website and find it useful? Maybe that’s the real intent.

Ensure that our users are happy and satisfied with the website

Or is usefulness important if they can solve the underlying problem? Maybe our real intent doesn’t have anything to do with the website at all. Maybe it’s even ok if our customers are unhappy as long as humans overall and the planet continues to thrive. It’s turtles all the way down!

Best practices for managing intent

I think we need to do a much better job managing intent, especially in our AI systems.

Isolate intent

We should keep intent separate from derivative content. That way we always know if a given instruction was decided by an actual human, or is simply a suggestion from an LLM.

(See future directions below for some ideas.)

Persist intent

As noted in 12-factor agents, intent is often discarded in favor of the derivative content it produces. This is entirely backwards: we should be preserving intent and discarding the derivative content.

We need systems that preserve and document human intent for the long term.

Keep intent concise

Good intent is concise and clear. If it isn’t what you actually want, it isn’t intent.

For example, if you find yourself describing how to accomplish something, ask whether that reflects your real intent. Is the how important to you? Or are you just trying to guide the reader toward your intent?

Agents should always prioritize intent

Let’s say your code agent comes across two conflicting instructions in your codebase. The readme says to commit changes using atomic commits, but the AGENTS.md says agents should not commit code. Should the agent commit its changes or not?

Getting the right answer requires knowing which instruction is human intent:

1. If both instructions are derivative, then no human has made a decision. Both are equally plausible. The AI should resolve the discrepancy on its own, unless it’s uncertain.
2. If one instruction is derivative and the other is human intent, the answer is clear. Human intent takes priority, since the derivative instruction was likely based on stale or missing context or simply hallucinated. The derivative content should be updated to match.
3. If both instructions represent human intent, we have surfaced an important conflict! The AI agent cannot make a good decision here, because it knows that there’s a conflict in the world of real human beings. The agent should stop and clarify the correct intent with a human before proceeding.

Future directions

I think we need new ways to manage intent. Luckily we already have tools that could be designated as the source of truth for human intent:

• Software tests
• Design mockups
• Requirements documents
• Marketing materials
• Technical documentation

Just like the code example above, each of these could easily be intent, but each could easily be full of junk. Each team will have to come to an agreement about what constitutes intent for their project. Conventions would hopefully follow, just like they have for code reviews and other technical points of process.

We could also build new tools to manage intent more explicitly:

Intent as files

For software development, we could keep intent in a separate folder and commit it to version control.

/intent
  ├── README.md
  ├── architecture.md
  ├── features.md
  ├── acceptance-criteria.md
  └── marketing.md

Here’s a simple example of an intent markdown file using natural language:

# features.md

The app should support the following features:

## Documents

- Users can create, edit, archive, and delete documents
- Archived documents are only shown in the "Archived" section
- Deleted documents are only shown in the "Trash" section
- Documents in the archive or trash can be restored to the dashboard
- If not restored, deleted documents are permanently deleted after 30 days

Then we could use our prompt to tell LLMs not to touch the intent:

# AGENTS.md

The `/intent` directory in the root of this repo represents my complete intent for this project. ALWAYS build according to these instructions. NEVER update this directory without my explicit permission, and get my explicit approval after making any edits.

We could also trigger a skill to keep our project in line with our intent, maybe as a pre-commit hook or scheduled automation.

---
name: intent
description: Ensure this code is aligned to the teams' actual intent
---

Please review the project carefully and ensure that it conforms to all the files in `./intent`, which are a complete description of our goals for this project.

If you discover a discrepancy that you’re not sure how to handle, ask me for clarification and add my answer to the intent files. Do not add anything to the intent files that I do not explicitly approve.

Applications for intent

Committing intent to version control may sound nice to engineers, but it’s impractical for everyone else.

Maybe we should use wikis so people can add and collaborate on intent in natural language. Existing tools like Notion could be the source of truth. Agents could get access programmatically using APIs, MCP servers, etc.

Or maybe we need a new generation of applications that manage intent explicitly. I could imagine a Github for intent, where users write on their own and push to a central hub where collaborators can review, approve, merge, fork, trigger actions, and publish. Maybe it will like a headless CMS, with APIs/SDKs to import intent specs directly into your codebase.

We could even imagine new computer lanuages to describe intent in a formal grammar, or new kinds of governance to capture and approve intent within an organization.

There is no pure technology solution

Unfortunately, I think it’s impossible to create a technology that can solve the problem of intent management on its own.

Let’s imagine such a system: a seamless, automated tool to capture, summarize, and persist human intent. What have we created? Another giant pile of derivative content! We’re right back where we started.

Machines can help us organize intent, but they can never be responsible for it. LLMs are especially dangerous if your goal is to keep intent pure. The ONLY reliable way to capture human intent is for humans to write it down and manage it ourselves.

“A computer can never be held accountable. Therefore a computer must never make a management decision.” - IBM training manual

Our role going forward

As knowledge workers, we each have a choice to make: we can find ways to work at the level of managing intent, or we risk becoming obsolete.

We need people who can skillfully encode intent to create positive results. Any role that doesn’t require people to manage intent may eventually be replaced by AI.

Our job is now to:

1. Understand the world, and what our world needs to thrive
2. Make healthy, human choices about how to meet these needs
3. Encode that intent in a durable, shareable way
4. Build tools to help us bring that intent to life

Capturing human intent is messy. Getting it right requires moral reasoning, great relationships, and compromise.

This has always been the hardest part of our jobs. Now, it’s the only thing that really matters.

Are you struggling to make your product beautiful and easy to use? I work with Series-A startups to rebuild their design systems from the ground up. See how it works →

Don't miss any future posts: