Observations of current AI workflow

For the past few months I have been logging what my coding agents do. I built moatlog to capture the files they read and write, the shell commands they run, and the prompts I give them, across both Cursor and Claude Code. The original idea was to distill that history into context an agent could read at the start of a session, so it would stop relearning the same repo every time. This post is about what the data actually showed, and how it changed what I think that tool should do.

There are two sources. The first is moatlog's own event log from building moatlog: 4,714 events across 109 sessions and 844 prompts, split roughly evenly between Cursor and Claude Code. The second is Cursor's conversation transcripts from eight of my side projects: 57 conversations, 3,290 messages from me, and about 30,700 tool calls.

All of it is one developer working on personal projects, so it says as much about my habits as it does about agents. I would treat everything below as observations, not benchmarks.

moatlog groups sessions into recurring areas of work. On its own repo, six areas covered 124 tasks. 102 of them, about 82 percent, were on the docs site, the blog, and the landing page. The context engine and skill generation, which are the actual product, accounted for the other 22.

That surprised me at first and then made sense. UI work happens in small rounds: change something, look at it, adjust. Each round is a new prompt. Work on the core tends to be fewer, longer tasks. A count of tasks rewards whatever gets done in many small rounds, so the measure you pick decides what looks important, and what you end up optimizing for.

Across the transcripts, 54 percent of tool calls were discovery: reading files and searching. 35 percent were edits, and 6 percent were verification like type checks, builds, and tests. Within a single message, the agent made a median of 3 discovery calls before its first edit, and 12 at the 90th percentile. At the start of a conversation, before it had any context, the median was 14 and the 90th percentile was 53.

Most of that looking is necessary. The part that looks like waste is rereading. About 9 percent of file reads were the exact same file and range the agent had already read in that conversation, with no edit to the file in between. That is a small share of any one session, but it adds up across thousands of reads, and it is the clearest sign of an agent relearning something it already had.

The median conversation used 87 tool calls. The longest 20 percent of conversations accounted for 85 percent of all tool calls, and the longest single one used 3,619. The average session is not where usage goes. It goes into the handful of conversations that run for hundreds of messages, picking up context, rereads, and corrections along the way.

My own messages explain part of that. Of my follow-up messages, 28 percent were questions, 20 percent were long pasted content like logs, drafts, and docs, and 10 percent were pasted errors. Corrections, "still broken" messages, and requests to revert added up to about 4 percent. That was lower than I expected. Most of the back and forth is not the agent getting things wrong. It is me thinking out loud and feeding it material.

The pasted errors are the part I think is most fixable. My guess is that most of them are me acting as the agent's eyes, copying output from a browser or terminal it could not see.

This also exposed a gap in how moatlog models work. It ties prompts to the files they touch, and in its latest summary it could only do that for 18 of 427 prompts. The rest were questions, planning, or follow-ups that did not map to any file. A model of agent work built around files has nowhere to put a large share of what actually happens.

The hypothesis behind moatlog was that an agent given a summary of past sessions, covering which files change together and which commands verify the work, would search less and finish faster. To test it, I ran the same task against fresh copies of the repo with and without that summary: add a new field to the event schema and keep everything that depends on it consistent.

Every run passed the build in both setups. In one pair, the summary cut tool calls from 145 to 83, 43 percent fewer. In another, they went from 61 to 93, 52 percent more. Across four pairs the averages were 86 without the summary and 80 with it. The spread within the runs without it, from 57 to 145, was bigger than any difference between the two setups. A single run would have told me whatever I wanted to hear.

I also checked whether agents read injected context at all. I put a made-up codename in an AGENTS.md file or a rules file and asked the agent about it, and it answered correctly in 39 of 40 trials. Getting context in front of the agent works fine. The open question is whether the content helps.

A recent paper, Evaluating AGENTS.md, tested context files on SWE-bench Lite and on a set of repos that already had them. Context files generated by an LLM did not improve success rates and raised inference cost by about 20 percent. Files written by developers did slightly better, though not by a significant margin, and also raised cost. The agents followed the instructions well, and following them meant more exploring and more testing.

On the other side, a five-run test from AAIF found that a twelve-line AGENTS.md cut wall time by 27 percent on an ambiguous task and by about 10 percent on a multi-file one. Their first single-run comparison pointed the opposite way, at 44 percent slower. That is the same lesson I keep running into: small, specific context can help, and single runs are not evidence.

I started with "more context, less searching." What I believe now is narrower.

Most of the waste I can see is relearning, not missing knowledge: rereading files, rediscovering how to run things, and me pasting in output the agent could not see. A lot of that is better fixed with a script, a tool, or access to the running app than with a paragraph of context.

Every line of context is read in every session, whether or not it applies, and every claim in it becomes something the agent may go check. So context has to earn its place. The plan for the next version of moatlog is built around that. It would look for waste that repeats across sessions, propose one specific fix like a rule, a skill, or a script, and then try it in real sessions with the fix randomly on, off, or replaced by a placebo of the same length. It would measure only the behavior the fix targets, and keep the fix only if it passes. The placebo matters because adding text of any kind changes what an agent does.

On accuracy, I do not have evidence yet. Every run passed with and without context, so on tasks like these the summary did not make the agent more correct. The accuracy I care about more is scope. Without the summary, the agent edited a file outside the task in three of four pairs. With it, that happened once, though that run touched two extra files. Four pairs is nowhere near enough, but it is the measure I am going to watch.

The most useful thing the logging did was replace my impressions with numbers. I thought agents mostly needed better memory. The data says they mostly need fewer reasons to look around, and that anything I give them to help should be tested the same way I would test any other change.