The memory layer: how AI agents remember what your agency knows

A memory layer is one shared, written record of what your business knows: your clients, your decisions, how you do things and how you sound. Every AI agent reads it before it starts a job and adds to it when it finishes. So does the team.

I gave a talk to a group of agency leaders this week about how I run my businesses with agents. I showed the design work, the build process and the QA. The thing people wanted to know more about afterwards was this. So here is how it works, and how to start one.

Why AI keeps forgetting

A model knows a great deal about the world and nothing about you. It doesn’t know your clients, what you agreed on last month’s call, or which client hates exclamation marks. Every job starts from zero.

Most people fix that by explaining the background again each time. That works for one person in one chat. It falls apart with a team, because everyone explains it differently, and it falls apart with agents, because nobody is there to explain.

The memory features inside ChatGPT and Claude don’t solve it either. They remember things about one person, inside one product. Your account manager’s assistant knows things your developer’s doesn’t, and none of it comes with you if you change tools.

A model knows a great deal about the world and nothing about you.

What a memory layer is

It’s the same idea you’d use for a new member of staff. You wouldn’t let them answer a client email on day one without telling them who the client is. A memory layer is that briefing, written down once, kept current, and handed to every agent at the start of every job.

In my businesses, every agent starts from what the business already knows. The whole team shares the same memory. Everything we learn updates it. Whoever is working, person or agent, in whichever business, has the most up-to-date information.

It’s what makes the rest of the operation work.

What goes in it

Four kinds of knowledge:

• Client history. Who they are, what we’ve built for them, what’s live, what’s coming, who the people are.
• Decisions. What was agreed, when, and why. The why matters most, because it stops an agent undoing a decision it doesn’t understand.
• How we do things. Our standards, our processes, what “finished” means here. For developers, this is where your coding standards for agents live.
• Our voice. How we write. My voice file is built from years of real client emails, so a drafted reply sounds like me and not like a chatbot.

A useful test: if a good new starter would need to be told it, it belongs in the memory.

How it works day to day

Every agent reads first

Before an agent drafts a support reply, it reads that client’s history. Before one writes code, it reads the project’s standards. Before one drafts an email as me, it reads my voice file. Nothing starts cold.

The work updates it

After a client call, an agent reads the notes and pulls out what was decided and what needs doing. Everything we learn updates the memory. Nobody has to remember to update a wiki, which is why it stays current.

The team sees the same thing

The memory isn’t hidden inside the agents. The team works from the same record the agents use, so there is one version of what we know about each client, not one per person.

It sits outside the models

This is the part people miss. The memory is ours. The models are rented. If a better model arrives next month, we point it at the same memory and carry on. If one is switched off, the memory is still there. I’d never put what my business knows inside one AI company’s product.

The rules that keep it trustworthy

A memory that’s wrong is worse than no memory, because agents act on it without asking. These are the rules I work to:

1. New facts are checked before they’re trusted. Things agents learn go into a draft area first. They become part of the trusted memory once they’ve been checked.
2. Everything says where it came from and when. A fact with no source and no date gets questioned.
3. One home for each kind of knowledge. Client knowledge in one place, project documentation in another. Two copies always drift apart.
4. No secrets. No passwords, no keys. Those live in a secure store, and agents get their own logins.
5. Nothing personal that isn’t needed. If an agent doesn’t need it to do the job, it doesn’t go in.
6. People can correct it. If the team spots something wrong, they fix it, and every agent has the fix from then on.

How it differs from a wiki

Most agencies already have a wiki, a shared drive or a pile of Notion pages. That’s a start, and you can build on it.

The difference is who writes it and who reads it. A wiki is written by people when they find the time, which is why it’s always a year out of date. A memory layer is written by the work as it happens and read by agents every single time, so gaps and mistakes show up fast.

It’s also how you keep what your agency knows when people leave. Any agency owner knows how much walks out of the door in someone’s head.

How to start one

You don’t need special software to begin.

1. Pick one client. A busy one, where the team keeps re-explaining things.
2. Write one page. Who they are, what you do for them, what’s been decided, how they like things done, what to never do.
3. Make it the first thing every agent reads. Whatever tool you use, the instruction is the same: read this before you start.
4. Update it after every call. By hand at first. Then give that job to an agent.
5. Add the next client. Then your standards. Then your voice.

After a month you’ll have something no model can give you: an agent that already knows your client before it writes a word.

Where this fits

The memory layer is one piece. The others are finding the person who’ll lead it, agreeing the rules for client data, and building the first agents one job at a time. It’s the piece I’d set up early, because everything else gets better once it exists.

If you’d like help setting one up in your agency, that’s the kind of work I do inside agencies. See how I work, or send me a message and I’ll tell you where I’d start.