Intro
In order to get some AI skills I’ve been playing around a little with setting up some local AI, I have an old computer with 32 GB RAM and an RTX 3090 (24 GB VRAM), so I can run some local models with ollama. Also gotten OpenCode to connect to it. Kinda curious how far I can get with that set up.
And then I had a thought - What if LLMs are a dead end
I read and watch a lot of Science-Fiction, and a concept that is sometimes used is that of an ancient race that “evolved” into stagnation. An example would be one of my favorite book series - The Well of Souls - by Jack L. Chalker, where there are traces of an ancient race known as the Markovians all over the universe, and they evolved technologically so much that they stagnated and to avoid dying out had to start over and become other (completely new) races in the universe (don’t worry, should you decide to look up and read this series that little piece of knowledge won’t spoil the story for you).
So why bring that up when talking about LLMs? Well Large-Language-Models are trained on a large amount of data, and that teaches them to respond with the stuff that they’ve learned. But two things come to mind: 1) When it comes to programming there are a lot of bad examples out there that LLMs can be trained on (believe me I know) and 2) An LLM is not truly creative, it essentially makes a statistical analysis of what the most probable answer it should give is, mixed with some randomness (Yeah, I know it is a bit more complex than that).
So it is excellent at giving you solutions to problems that are already known in some form or other and then mimics the solutions it has been trained on (including bad examples and all). It is able to draw on a a huge amount of accumulated data and find patterns that would take a human forever to find. And that can be used to solve many problems a lot faster than a normal human can.
But what will happen when we need them to solve stuff that no one has ever solved before and where the solution can not somehow deduced from the knowledge we have already gathered?
I know there are other areas in AI than LLMs, but LLMs are the main focus of most AI right now.
Will we stagnate
By relying to much on LLMs will we lose the ability to be creative ourselves? And who will then generate new solutions that can feed the next generation of LLMs? It will probably take several years and some generations of LLMs before it comes that far, but I find the thought worrying that we may be painting ourselves into a corner from where we cannot escape, and then we might become the race that has stagnated.
Just some random thoughts on a tuesday evening…