Database Doctor
Writing on databases, performance, and engineering.
Cover
Reading time icon 14 min
Publish date
Share on Facebook Share on LinkedIn Share on X Share on Reddit

How LLMs changed my tooling, flow states and repo structure

Like most programmers - I am highly opiniated about what tools I use and how I want those tools to work. There have been very few changes to my preferences for at least 15 years. But with the coming of LLMs - I can no longer defend my old habits. This old dog had to learn new trick - this is my story.

Where I was

I was comfortably on Windows, using the JetBrains suite. My development tasks span a wide range of languages: Python, C++, JavaScript, TypeScript, Java, SQL, Web (HTML/CSS), C#.

Its not unusual for me to write code in 3-4 different languages in the same week.

My preference for JetBrains was driven by simple habits:

Like all tools, an IDE needs to feel like extensions of your mind. If you notice the IDE getting in the way and spend time dealing with it - the UX has failed.

Unfortunately, as I added LLMs to my toolbox - the IDE became more visible.

Java isn't the right tool for an IDE

To quickly navigate between symbols - the IDE needs to maintain an index of symbols and their locations in files. As a regular human typing code - this indexing is typically invisible as a background process. But once you have an LLM experimenting in parallel with you, the rate of change goes up dramatically.

LLMs can undertake refactors (if you guide them right) in seconds that would take a humans hours. The reindexing of the code can no longer use human typing speed as an amortisation function. And once the background indexing introduces latency in the IDE - the experience degrades significantly: Keystrokes are not appearing on time, moving to a new tab takes a few hundred ms extra, navigating to a function definition has noticeable latency, little annoyances like that. JetBrains (particularly CLion - the C++ IDE) was beginning to show its Java roots.

But to really understand why these little pauses in the editor tickle me this way, we have to talk about flow state.

What is Flow State and why is it important?

Have you ever been so deeply engaged in an activity that you forget that times passes? Your mind feels perfectly aligned with the work in front of you. Your actions appear accelerated - effortlessly elegant - and you are performing work much faster than usual. And suddently, someone interrupts you - and that feeling is lost and you have to start all over.

If you can identify with this state of mind - you have experienced "flow state".

Artists may describe this as "a burst of inspiration". Race drivers are "in the zone". Surfers are "in the tube". Musicians are "in the groove". E-gamers are "locked in" Programmers: we "flow".

Autistic minds like mine have a complicated relationship with flow - and exact behaviour various by individual. For me, it takes longer to enter flow than the neurotypical mind. But, when I enter the flow state - my perception of time entirely disappears, I no longer experience hunger or thirst. I barely move in my chair (so little that I now have my smartwatch tell me to move). I am able to maintain this state for 20 hours straight and I sometimes don't notice tiredness until I realise that my eyes no longer see the screen properly, and that I just typed and erased the same line of code 4 times in a row.

In flow state - we are able to hold highly complex structures in our thoughts. Flow also has beneficial effects on neuroplasticity: When in flow our brain has weak episodic memory (i.e. we lose track of time) - but it instead operates in a highly tuned dopamine loop that increases semantic understanding and long term memory formation. In other words: Flow is great for learning things and understanding complexity.

But it only works if we stay in that state for some time.

Why is this important and what does it have to do with LLMs?

Flow as a productivity booster

When people experience flow, their productivity goes up dramatically. This observation was described in 1987 by Tom DeMarco and Timothy Listers in their book: "Peopleware: Productive projects and teams".

Interrupting people in flow breaks the state and productivity drops. Great project managers understand this, and try to keep distractions of their programmers to a minimum.

But flow can break without outside disruption. It can occur when you are "forced to wait" to complete your current thought.

With my JetBrains IDE - that 1-2 second garbage collection break is enough to disrupt flow. It does not sound like a lot, but it is significant enough to notice.

But with LLMs, it gets worse. What happens in this workflow:

  1. Ask the LLM to do something, watch it "think" and reason
  2. Wait for outcome - review

Depending on the complexity of the task and the LLM you use - the time between step 1 and step 2 can be measured in seconds or tens of minutes.

I am convinced that the time of typing code and boilerplate is over - it is simply faster to prompt for it.

But does it save time? Can you be in flow state when you prompt?

Prompt Latency

What happens in your human brain while you wait for the LLM to finish its work? Do you maintain concentration and flow - or do you context switch? The LLM can easily keep those complex structures in its "mind" - but what does your mind do while you wait for the LLM to "reason" and code generate?

As far as I can tell, I have two real options:

In both cases, ease of code navigation, not coding and typing speed, becomes the priority. Flow becomes harder to maintain in this mode - I become less tolerant of latency. I can no longer wait for indexes to be rebuild because the change rate is so fast. My keyboard shortcuts get oriented towards structural exploration of the code - not changing it. For example, I bind CMD+Up to mean: "Show callers" and CMD+Down to be: "Jump to definition". Navigating a single line (or navigating to a line in the first place) just doesn't matter anymore. If I want to fix an error, I ask the LLM to do it and tell it how to validate that it did the right thing. I don't look up the exact code line that caused the error anymore. My IDE becomes primarily a viewer, secondarily an editor.

In all cases, my flow state moves from focus on solving a task to monitoring a task being performed and interrupting the solving process opportunistically. My life becomes one big cycle of concurrent PR reviews and pair programming. This feels more like playing a video game than coding.

As anyone who has maintained a large code base knows: PR reviews are a terrible way to spot bugs and learn about the code - unless you wrote some of the code being modified yourself. No amount of documentation fixes that - because it is through the process of creating the code that you understand why the code looks the way it does.

If documentation allowed you to learn codebases - we could all become math professors by reading Principia Mathetica. Or, staying in our industry, nobody becomes a computer scientist by sitting down and reading TAOCP - even though it contains all the information you need to become one. No amount of book reading will teach you to forge a sword on an anvil - you have to swing the hammer yourself. You master a code base by doing the code base, not by reading about it.

If all code is written by the LLM (as indeed, appears to where we are going) then how can I realistically claim to do quality reviews, when I have never been in flow state on the code base I am reviewing?

I can switch away from a Java based IDE and win those 1-2 seconds back - but what am I to do with the time spent waiting on the LLM?

My flow has been stolen. Not just because the LLM is spewing code faster than I can keep up, but because the very act of waiting breaks my flow.

Playing Simul Chess

Chess is generally a terrible analogy for LLM vs human driven coding. The game of chess is played in a set of closed rules on a well defined board. It is game of perfect knowledge. Software development, on the other hand, is done as an open ended exploration of a poorly defined environment, run by fallible humans and influenced by random events out of your control. But I think the chess analogy can be recruited for a different purpose: to talk about the flow the player needs.

To play chess well, you must have the ability to maintain flow for extended periods of time. You need to keep an extraordinary amount of information in your mind at the same time.

Hardcore chess masters are able to play in special tournaments called "simul chess". In this tournament form, the master plays multiple opponents at the same time, each on their own chessboard. The master walks from board to board - evaluates the position on the board, makes a move and then walks to the next opponenent. Win rates in such tournaments are still in the >90% range for grandmasters playing against experienced players - even when playing against 10 opponents at the same time. That is some serious context switching, while still in flow!

How do they do this?

Representing the Board

Beginning chess players think of the board in terms of individual pieces. Mentally, they represent the board as an 8x8, 2 dimensional array: Rook on (0,0), Knight on (1,0), Bishop on (2,0), etc.

Experienced chess players begin to think of the board in terms of patterns and relationships between pieces.

Examples:

In other words, the master maintains knowledge of the board in a highly compressed form - based on patterns already learned.

In programmer terms: if we want to become grandmasters in this era of LLMs, we must think in terms of patterns and architectural relationships - not look at individual lines of code.

How do we maintain this summary representation of the code?

Make the LLM follow patterns and describe them to you

I have asked my LLM to never generate comments in code.

Instead, I do this:

I keep a root level README.md pointing at each of the individual folders and documenting (with the assistance of the LLM) the relationship between modules.

Whenever I start a new session, I ask the LLM to prime its context from the README.md files in the repo and update them again with anything it thinks are missing.

Every now and again, I ask another LLM to validate that the README files are an accurate representation of what the code actually does.

These files represent the high level patterns in my repo. Each folder is a chess board and the README.md (and its diff) is the positional summary I need to make a move.

It is not perfect and the LLM still does crazy stuff - it still has LLM nature. But it does allow me to create a completely fresh context quickly

Flowing more by doing more

Here is the counter intuitive insight: I can actually code better by doing more things at the same time, than I can if I wait for the LLM to "make a move".

Staying in our increasingly fragile chess analogy: Imagine what happens if the simul chess master walks out of the room, goes to do something else and stops thinking about chess for a while. As she walks back to the board of the opponents - she is less likely to have the context ready in short term memory to make good moves. Her flow is broken!

The solution is to keep your mind focused on coding - even if that means flipping to a different window and checking on the other LLM working on the same project, but with a different task.

I currently keep 2 copies of the same repo checked out and have LLM (and two VS.code editors) open and active in those repos. I treat my editors as junior programmers that I am overseeing. This allows my programmer brain to always have something to do. That in turn maintains flow better than waiting and doing one thing at a time!

Interestingly, this pattern is exactly the opposite of what I would do if I was having hands on the keyboard, coding away.

I want to bring this concurrency number up, because I still get downtime between prompts where I am forced to jump out of flow state. I might not be able to play simul chess with 10 boards - but perhaps I can get to 5.

I am going to need a bigger monitor! Or maybe 5 monitors...