The line went up a bit faster in the last 5 years so the line must continue to go up exponentially ? That's about the only "logical" way to get from LLMs to AGI.
Replacing "the best of us" in 2 years... come on, even Altman isn't getting that high on his own supply
> I think you're mistaking me for somebody who's spent a lot less time on these issues.
Unless you're God himself I doubt you can predict the future of events that never happened in the past with any kind of accuracy. There have been thousands of people like you with prophecies of AGI in the next "10 years" since computers are a thing.
What? LLMs aren't going to be AGI, or at least I'd be really surprised if anything that was basically an LLM got to AGI. LLMs may be components of AGI, but even a "reasoning" LLM isn't going to be AGI. Not even a multimodal model. Probably not even with a planning scaffold. And even if you can call the core of your first AGI an LLM, I really doubt it'll recognizably be a transformer.
What gets you to AGI is dumping, I don't know, maybe 100 billion dollars of investment into trying out the kinds of not-particularly-LLM-like architectural innovations that people are already playing with on top of deep learning generally, plus the hybrid architectures that will inevitably pop up.
We are probably 2 or 3 breakthroughs away from AGI. When you have that much attention on a topic, especially when it's aided by what we already have, you can reasonably expect to get them relatively soon. Then there are the scaling issues.
Missing capabilities that probably won't be achievable purely with LLMs or similar:
+ Long-term learning (not just context hacks) from live experience, including deciding what it needs to learn and choosing what to do or ingest to achieve that. This basically eliminates the training-versus-inference distinction.
+ Critical evaluation of incoming data during ingestion, in all learning phases, or at least after very early pretraining.
+ Better critical evaluation of its own ideas (although "reasoning" is a step toward this one).
+ A decent predictive understanding of the physical world, which probably demands embodied robotics with a variety of sensory and motor modes (probably plus "pure digital data" modes like existing models have).
+ Complex, coherent, flexible and adaptable long-term planning and execution at a variety of time scales and levels of abstraction.
+ Generally better attention control in the "nontechnical" sense of attention.
+ Self-improvement, including both architectural innovation and performance optimization (probably falls out of the above).
+ Scalability and performance. Even if you build significantly superhuman AGI, it doesn't change the world much unless you can run a lot of it.
I would expect most architectural breakthroughs to help with more than one of those, and a really good architectural breakthrough to help with most of them all at the same time. So I'm guessing two or three breakthroughs. And I don't mean "breakthroughs" in the sense of, say, relativity. Just significant new architectural insights, on the order of transformers or maybe deep learning generally.
The one I'm least sure will be solved soon is the scaling one. If it turns out that the only way to scale up enough is to completely reinvent computers, perhaps out of biotech or nanotech or whatever, it's going to take a long time. On the other hand, if you do get even remotely decent scaling, and you have even the clunkiest form of generalizable self-improvement, you're probably basically done.
> We're also "2 or 3 breakthrough away" from anything
That whole line of argument is invalid from the start, since it has nothing to do with the actual subject under discussion. But I'll take the bait.
We are not, in fact, 2 or 3 breakthroughs away from any of those. Nor is there remotely the level of investment in any of those that AI and ML are getting, which means any breakthroughs we do need are going to take longer.
> fusion
Probably one breakthrough needed... but with or without that, fusion energy may not be feasible at all, at least not at a price point that makes it competitive with the energy sources we already have.
If feasible, fusion will need a crapton of incremental, interdisciplinary engineering and testing on top of the breakthrough. That kind of grind work is harder and slower than breakthroughs, because there are so many conceptually unrelated, but interacting things to get right before you have a power plant.
Then you have to get the permits.
Fusion plants will probably not be on the grid in 20 years. If they are, it'll be because of some radical changes beyond the borders of energy generation.
> space exploration
There's space exploration going on now, and has been for decades.
If you mean long-term, self-sustaining space living, there are a ton of engineering problems to grind through, but I don't know if you need even one technical breakthrough. The bigger problems are that--
+ You have to do it on an immense, prohibitively expensive scale for it to be viable at all.
+ The social and political problems are enormous.
+ In the end, space just plain sucks as a place for humans to live anyhow.
You might be able to fix that last one with bioengineering capabilities on the order of the ones you'd need for immortality. Which is a long-term project.
Personally I think it's not worth the trouble.
> immortality
You might eventually be able to get to (quasi-)immortality, but it's going to require a ridiculous number of diverse interventions in extremely complex systems... plus the tools to actually do those interventions. Which depend on understanding the systems. Which requires more tools.
You need to do massive amounts of grunt work and probably make a lot of breakthroughs. Nobody who has any real understanding of the problem has ever suggested that it's two or three breakthroughs away. And you might need superhuman AGIs first to understand the total effect of all the hackery you were doing.
Moderately better than human AGI can probably speed up any of those, but probably not as far as some people think. Unless you're a superintelligence, you have to put in lab time for all of that stuff. You might have to do that even if you are a superintelligence. And superintelligence feasibility, capabilities and timelines are harder to guess than simple AGI.
Replacing "the best of us" in 2 years... come on, even Altman isn't getting that high on his own supply
> I think you're mistaking me for somebody who's spent a lot less time on these issues.
Unless you're God himself I doubt you can predict the future of events that never happened in the past with any kind of accuracy. There have been thousands of people like you with prophecies of AGI in the next "10 years" since computers are a thing.