I have a hard time not cheating once I’ve started. I’ve noticed this about myself in a few different places, and I’ve known for a long time. I don’t think I’m alone in this, but I think it’s worth talking about what I see, why I think I am like that, and what that means for my work.
Cheat codes
As a kid, I played my fair share of video games, and once the internet was available to me, I fell into the same trap again and again: I’d get stuck, look up the cheat codes, and ruin the game for myself1.
The cheats took the fun out of it. I knew that while I was using them, and I knew it before I looked them up. But once the codes were there, not using them became mentally tough. The game no longer offered a challenge. Instead it gave me a choice every time I got stuck: struggle or use a code.
Some games completely changed. I remember that I would spend hours and hours on GTA San Andreas on my PlayStation just doing weird challenges for myself instead of ever finishing the storyline. I also don’t know, in hindsight, how much actual fun I was having.
I don’t think the problem was the cheat itself, it was knowing that the answer was always within reach, and that I had to make an active decision not to use my superpowers.
Learning
A few years back, I noticed the same thing again in chess as an adult learner in an online environment. Going through my games afterwards with Stockfish (or any engine, really), I would have a hard time staying engaged. The point of the exercise is to find my own mistakes and understand why they were mistakes. What I would actually do is have the engine reveal the best line, and then kind of follow along, sometimes ignoring moves I didn’t understand (those are probably not the kinds of subtle, crucial moves that will make you a better player, right?!).
Following along feels like work. You look at the moves, you think real hard, you understand each one as it is shown to you. But understanding a move when it’s handed to you and finding it yourself are two very different skills. Only one of those is actually relevant for my games.
Again, the engine isn’t the problem. It’s a wonderful tool. The problem is that once it is open next to the board, every hard position becomes a choice between thinking and asking, and asking is always easier, and you can fool yourself into thinking it’s an equivalent learning outcome.
Katas
And I think programming with an LLM is similar, especially when doing katas for your own learning. As anyone reading this blog will know, I love honing my craft, and I love passing my weird little learnings on.
The whole point of a kata is the doing. Nobody needs another solution to Sudoku or an implementation of a terrible Brainfuck compiler. You do it to train something in yourself, and the result is mostly a byproduct (and occasionally a very fun demo).
With an agent in the machine, it’s tempting to just let it spit out a solution, check it superficially, and, again, pretend that that’s learning. And it does look a lot like learning. I read the code, I understand the code, it passes the tests. But it’s the same following-along I did with the chess engine. I recognize the solution, I didn’t find it2.
What makes it worse than the engine, I think, is that the agent is always there, and it’s part of my legitimate workflows. When working, I use agents a lot. They help me reduce the amount of time spent on tedium and ceremony. Sometimes they help me with arcane parts of my work stack I don’t care to know about, like build tooling, CI YAML, even framework oddities that do nothing to my skill as a programmer.
But this workflow does not work for learning. Learning is goal-oriented, but the process matters more. At work, I don’t care how the ticket got closed and the bug got fixed, I care that the product got better. When doing a kata, I necessarily care about the how, because it’s the only thing that matters for the ultimate goal.
But it’s so damn tempting to just let a model push out some work instead of doing it myself.
Fin
I don’t have a good answer to this. I’m not going to stop using LLMs, and I don’t think I should. As I said, for a lot of my work they are genuinely useful, a cheat code to a manner of productivity, especially around the mundanity required to maintain most products3.
But I think it’s important to stay honest with myself. If I want the result, I can use the tool. If I want the skill, I probably need to make sure I stay disciplined, because if my track record is any indication, I won’t be able to resist it.
Footnotes
1. I’m old enough to remember that before the internet, you had to call an expensive phone number to get the cheat codes. I did that for Batman on the Game Boy Color, and the guy on the other end actually had a book he looked the code up in. This sounds like I’m of antediluvian age, but it happened in the late 90s/early 2000s.
2. And let’s be honest here: sometimes I don’t even completely understand the solution. There are subtleties I’ll gloss over, complications I just accept, little nooks and crannies in the code that don’t hold up to scrutiny. Crucially, they’re not there when I do it by hand.
3. That’s not a dig on product work. It’s just fundamentally different, and technical excitement only sometimes intersects with what a truly awesome product needs. That’s fine, sometimes the excitement at work comes from building something cool in an unexciting way, and sometimes it’s just work.