Sebastian Wallkötter, PhD
Verified account This account is verified. Learn more@FirefoxMetzger
Helping teams build agent architecture around business processes. Context engineering, inference, leverage through software. ImageIO maintainer (1.5B installs)
48 Following 47 Followers 290 Posts
Latest posts 5
Before you learn something, nobody around you knows how to do it.
After you learn something, everyone around you knows how to do it.
I still don't quite understand this mystery...
After you learn something, everyone around you knows how to do it.
I still don't quite understand this mystery...
If you don't know something, ask AI to teach you.
Don't know how to build a website? Ask AI.
Don't know if your AI can do something? Ask AI.
Don't know how to phrase your question for AI? Ask AI.
There are no dumb questions, and your AI doesn't judge.
Don't know how to build a website? Ask AI.
Don't know if your AI can do something? Ask AI.
Don't know how to phrase your question for AI? Ask AI.
There are no dumb questions, and your AI doesn't judge.
If all my agents got deleted and I had to start from scratch, here's the first 5 I'd rebuild:
1. Digital Twin
Doctor Manhattan superpowers on the internet. I work through this agent and it collects my traces. They become a dataset to align my preferences. Once I've done something a few times and given feedback, I can task a fork of the twin while I do something else. It's a digital version of an executive assistant.
2. Deep Research
A specialist agent tuned for searching the internet. You can get far by just asking a vanilla agent to "do a deep research on X", but that tends to be token hungry. I also have a preference for highly dense results; I am making one decision, so I often only need the top 3 options. I need some way to teach the agent what a top choice means for me.
3. Inbox email agent
It looks through my incoming emails and tags them. That's it. I have static filters for some things, but an agentic catch-all for the pile of "other" emails is useful. Personal agents can judge on context outside the inbox which allows them to do a better job than the current "AI inbox" features.
4. Brainstorm
Niche, but surprisingly effective more often than I am willing to admit. It uses (inverse) stochastic consensus to surface original and weird ideas. If I need a new or non-obvious angle I task this agent to generate a list. Usually there is something on it that sparks an idea of my own, and occasionally it actually has a good idea itself.
5. Proposals
I pay my bills helping business owners fix a half-working Claude setup, and by moving working setups into the cloud for scale. The first step of that process is writing proposals. It's a repetitive task I do, so it's worth having a dedicated agent for this workflow.
If you're wondering why there's no coding agent: I use Cursor with minimal customization; I am happy to pay them to build that agent for me.
What's the no. 1 agent you'd rebuild?
1. Digital Twin
Doctor Manhattan superpowers on the internet. I work through this agent and it collects my traces. They become a dataset to align my preferences. Once I've done something a few times and given feedback, I can task a fork of the twin while I do something else. It's a digital version of an executive assistant.
2. Deep Research
A specialist agent tuned for searching the internet. You can get far by just asking a vanilla agent to "do a deep research on X", but that tends to be token hungry. I also have a preference for highly dense results; I am making one decision, so I often only need the top 3 options. I need some way to teach the agent what a top choice means for me.
3. Inbox email agent
It looks through my incoming emails and tags them. That's it. I have static filters for some things, but an agentic catch-all for the pile of "other" emails is useful. Personal agents can judge on context outside the inbox which allows them to do a better job than the current "AI inbox" features.
4. Brainstorm
Niche, but surprisingly effective more often than I am willing to admit. It uses (inverse) stochastic consensus to surface original and weird ideas. If I need a new or non-obvious angle I task this agent to generate a list. Usually there is something on it that sparks an idea of my own, and occasionally it actually has a good idea itself.
5. Proposals
I pay my bills helping business owners fix a half-working Claude setup, and by moving working setups into the cloud for scale. The first step of that process is writing proposals. It's a repetitive task I do, so it's worth having a dedicated agent for this workflow.
If you're wondering why there's no coding agent: I use Cursor with minimal customization; I am happy to pay them to build that agent for me.
What's the no. 1 agent you'd rebuild?
I have a challenge for you: Explain RAG search, deep research, and RLM without using any jargon!
Here's how I would do it:
All three are methods that allow agents to find the information to give a correct answer without inventing facts.
RAG (Retrieval Augmented Generation)
RAG searches through information you have curated in advance. The agent takes a document, e.g. a user manual, and adds a bunch of bookmarks to it. When you later ask a question, it finds the most relevant bookmarks, reads only those pages, and then answers based on what it just read.
Deep Research
Deep Research should really be called "wide research". The agent searches the web for your question, reads the result, asks itself follow-up questions. It then searches the web again for those follow-up questions, and the cycle repeats until the agent thinks it has a clear enough picture to answer. The result is the agent casting a wide net to answer your question.
RLM (Recursive Language Model)
RLM lets agents clone themselves Doctor Manhattan style. When it gets a question it clones itself for every book in the library. The book clones clone themselves for every chapter. The chapter clones clone themselves for every section, and so forth. Each clone only reads a small section and reports back up. The original agent then answers based on the collective information in the library.
How would you explain it?
Here's how I would do it:
All three are methods that allow agents to find the information to give a correct answer without inventing facts.
RAG (Retrieval Augmented Generation)
RAG searches through information you have curated in advance. The agent takes a document, e.g. a user manual, and adds a bunch of bookmarks to it. When you later ask a question, it finds the most relevant bookmarks, reads only those pages, and then answers based on what it just read.
Deep Research
Deep Research should really be called "wide research". The agent searches the web for your question, reads the result, asks itself follow-up questions. It then searches the web again for those follow-up questions, and the cycle repeats until the agent thinks it has a clear enough picture to answer. The result is the agent casting a wide net to answer your question.
RLM (Recursive Language Model)
RLM lets agents clone themselves Doctor Manhattan style. When it gets a question it clones itself for every book in the library. The book clones clone themselves for every chapter. The chapter clones clone themselves for every section, and so forth. Each clone only reads a small section and reports back up. The original agent then answers based on the collective information in the library.
How would you explain it?