I use AI daily for coding, but I'm confused about one thing.

[vitobotta@mastodon.social]

I use AI daily for coding, but I'm confused about one thing. If these models just run inference on frozen weights, they aren't really learning, right? It feels like working with an expert who has constant amnesia. Is AGI even possible if the model never truly updates its understanding?

[pointlessone@status.pointless.one]

@vitobotta models don’t update on each request but a lot of the requests and responses are collected for the next training round. Not all of it goes into training but it gives signal for what responses are good and what are bad.

Also I don’t think any of the experts believe transformer architecture (what almost all current LLMs are) is the path to AGI. It mostly CEOs who talk about AGI right behind the corner.