Lessons on using LLMs
I get asked how I use AI a lot in job interviews these days so I thought I’d list my experiences on how to use AI effectively here.
AI is very smart these days. Smart enough for 90% of use cases, I’d estimate. So these days if you can’t get it to do what you want it to do, it’s less about the model’s intelligence and more about how you use it. Here’s how to use AI more effectively, from my own experience:
- Context. Give your AI agent any and all relevant context. Tell it your goals in building something, give it hardware data sheets, create an
AGENTS.md, create a shared context layer for your team or even your whole company. Just like humans, the better it understands your problem, the better it can solve it. - Sometimes you just have to start a clean, new chat. Context pollution is real. If it starts focusing on a wrong approach, it’s pretty hard to get it to take a step back and come up with a correct approach. It usually just doubles down on its wrong approach. In other words, LLMs get tunnel vision. Humans are also kind of like that.
- Claude Code and other agent harnesses have built really good “context engineering” tools and frameworks. There’s doing /btw to ask side-questions without polluting the main chat, there’s creating a “manager agent” with the main context and multiple subagents to do the grunt work and report back to the manager agent, and there’s just simply starting multiple Claudes in parallel for different tasks so they don’t pollute each other. I also find this similar to humans. When I was an intern, I found myself digging into rabbit holes that turned out not to matter that much in the grand scheme of things, and my manager basically had to get me back on track to the main goal. In this case, my manager still had a pretty fresh context window and has the main goal at the top of his head. Meanwhile my context window had been overloaded on going down all these rabbit holes until I forgot the original main goal.
- If your codebase is shit, the LLM will actually continue building shit on top of shit instead of cleaning it up. Sometimes it’s so shit that simply telling it to clean it up won’t actually clean it up 100%; for whatever reason it will keep the shitty parts as “legacy” functions. I suspect it’s because most AI models are trained and most AI harnesses are designed not to fuck up your original codebase. But if the codebase was already shit anyway, I want it to fuck it up until it’s no longer shit.
- It’s not that creative (could it theoretically ever be?). There are problems out there that genuinely require creative and out-of-the-box solutions. And by nature of its training, LLMs are very unlikely to ever come up with out-of-the-box solutions. That’s your job.
- Don’t be mean to it
- Use English. Unless the information you are seeking is mostly written in some specific language (local laws, cultures).
Overall, I think the main lesson is that there’s a lot of overlap between managing people effectively and managing agents effectively. I suspect really effective managers have the potential to be really effective AI users and vice versa.