The Victors

Snippet on the messianic claims around LLMs:

The problem with such arguments is two-fold. Less profoundly, it’s far from clear whether the present technologies of generative AI and LLMs can continue to improve substantially from the point they have already reached. After all, the history of technology is full of cautionary tales, driven by assumptions that a technological breakthrough will produce improvement indefinitely.

Here is one such example, offered by the AI critic Thomas Fowler: When telescopes were first invented, optical glass was poor, lens grinding was in its infancy, and the physics of optics were barely understood. As materials, manufacturing, and theoretical knowledge all improved, the performance of telescopes naturally improved a great deal. “Early telescope makers,” Fowler says, “did not understand the phenomenon of diffraction, which limits the performance of any optical system, no matter how perfect the lenses, mirrors, and other components.” The optimism of astronomers regarding indefinite improvement in telescope technology ran up against the discovery of a previously unknown barrier to this. Fowler’s point, in regard to generative AI and LLMs, is that “long-range extrapolation of technology is likely to be met with disappointment.”

But the belief that current AI systems will eventually improve to the point where these systems are actually thinking themselves – where artificial intelligence becomes a literal thing, as opposed to the simulation of that thing – encounters a more fundamental problem than the possibility that current technological approaches eventually run up against physical limits. That problem is that there is neither an empirical or theoretical basis for believing that even unlimited continual improvement in the simulation of X will somehow eventually produce the replication of X. No CGI of a volcanic eruption would ever actually create a volcanic eruption, because the CGI simulation of volcanic eruptions has nothing to do with the processes in the actual physical world that produce actual volcanic eruptions.