Some good and different thoughts about where LLMs are going

the limits of space - Dan Davies - “Back of Mind”. Dan Davies keeps writing these super interesting things; I need to actually read his book.

When it comes to commercial products, though, the comparison is very different. At this point in the dot com cycle, Amazon has been going for about six months, so has eBay and Doubleclick is quickly consolidating the online ad space. There were plenty of people claiming to be “reinventing their business around the internet”, as with today’s pivots to AI, but there were also lots of very big and clearly useful, viable businesses (even if they weren’t all showing an accounting profit and many never did) which had been built on the new technology.

Comparing to today, you can see that the economics are different. Building and training an LLM is much more capital-intensive and expensive than setting up an ISP; it’s huge amounts of capex even by telecom industry standards. And there aren’t the same network effects; one reason that I’m setting 1994 as the start point is that this was also when people started to understand that it made a big difference if, as was true of the Web, “consumers were also producers” or people using the product also contributed to the value of the overall system. That was even more the case with Web 2.0, of course. Users of LLMs or chatbots don’t have that kind of effect; they might arguably produce a bit of training data, but it’s not the sort of thing that is going to drive an exponential curve when the rest of the thing is hitting diminishing returns so hard.

But even so, my gut feel is that “AI” in the sense of “applications of LLMs” are behind the curve – there’s a lot of people selling picks and shovels, but nobody seems to be mining gold. There’s still no “killer app” in the sense of something that people and businesses will make a big capital investment simply in order to use. To the extent that we have success stories, they seem to be in coding copilots, but we’ve known for at least fifty years (“The Mythical Man Month”) that improvements in the productivity of individual programmers do not translate to more efficient delivery of commercially viable products in any straightforward way. (I’d actually argue that “innovations which can 10x the productivity of a coder” aren’t even all that rare or uncommon – a lot of high-level programming languages would have been in a position make this claim when they were introduced).