The Challenge of Continual Learning

One of the big challenges in AI right now is continual learning: the ability for AI to learn and self-improve. Current LLMs have frozen weights. Whenever you interact with it, it doesn’t actually learn and incorporate new information you tell it into its “brain”.

“Oh but ChatGPT remembers my past conversations”. Well, engineers and researchers have found ways to make AI kind of learn. They do this by having the AI write down new information in markdown files that it can re-read in future interactions, prompting techniques, RAG pipelines, and wiki knowledge bases. This is the equivalent of a human writing down new information from their teacher, and when asked a question about it they have to read their notes. But real human intelligence wouldn’t need that. Real human intelligence can simply learn and remember new information inside their brain. In more technical terms, “real” artificial intelligence should be able to incorporate new information inside their weights instead of these hacky prompt engineering techniques.

Theoretically, I think we could already do this by simply re-training the model on every interaction? I wouldn’t know I’m not really an AI researcher. But retraining on every piece of new information is definitely impractical and expensive. So for now the best thing we have is markdown files and knowledge bases and “context engineering”.

Context through AI-Friendly Knowledge Bases

AI is very smart these days. Like I like to say, the 90% of the time it can’t do something is because it either lacks Tools or lacks Context¹.

Tools, the engineers are building at the application layer and then distributed for everyone. But Context, each organization is going to need to build its own knowledge base.

Say I want AI to make a line plot of my organization’s salary growth from the past 20 years. Well first it has to know where to get the data. Is it in an excel file on someone’s laptop? Who’s that someone? Is it in a cloud database? Which one is the column corresponding to salary? Is all the data located in one table or is it going to have to join multiple tables, or heck, even pull from some old legacy database that the organization migrated out of 8 years ago? You would have to include all of that context when you prompt it, otherwise it wouldn’t know any better, and at some point you’re doing half the work.

But theoretically, if it knew all that, if it had the same insitutitional knowledge as you did, it can connect to the right databases or contact the right people and can do it very quickly and very well. As people like to say, AI is like a very smart junior that still needs to be onboarded. And just like a very smart junior, you can give it specific instructions of how to do specific tasks. Or, you can consolidate your institutional knowledge into a document and then tell the junior to read it if it needs to know how to do something.

Now a junior can learn and remember from the past specific instructions you give it. LLMs will not, because we haven’t solved continual learning at the weights-level. So with LLMs, you need to let it write and read notes. You need to build an institutional knowledge base that it can query from to know who to contact, how to do things, where everything is.

And theoretically, once it has access to necessary tools and institutional knowledge, it can probably automate 90% of the work in an organization. This is where AI stops becoming a tool for employees and starts becoming a legitimate viable replacement of employees.

I believe this is the biggest challenge in Enterprise AI integration right now. Startups can afford to build knowledge bases since they’re still small and can set up the infrastructure for it now before it gets too big. but big companies have so much interconnected workflows and teams and files and products and clients that that it would be a monumental task to get all of that insitutional knowledge into a central database that AI can query from. Until those knowledge bases are built, at best you can only have managers giving high-level instructions to employees, and then individual employees giving more specific low-level instructions to their agents for their specific function; a temporary institutional knowledge transfer from one employee to an AI agent, instead of a world where the AI has access to all of the institutional knowledge and managers can just give their high-level instructions directly to the agent.

There are already many startups attempting to build apps that automatically build these knowledge bases for big companies.

Further reading: https://www.cerebras.ai/blog/how-we-built-our-knowledge-base, https://promptql.io, https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f