Systems not Models

10 Dec, 2024

Most marketing coming out of "ai" research entities is heavily "model" focused ¹. All of these posts seem to be touting "Our most capable model yet!", "Many parameters! More numbers wow!", "Benchmarks that we probably included in the test set but didn't know it wow!".

Don't mistake me, Reader, I do not intend to undermine the effort and complexity involved in creating these models (the marketing already does that). Focusing on quantifiable metrics while simplifying achievements down to something the mainstream can still consume probably increases hit surface. But we must admit model talk is an oversimplification. Model talk is... tired.

In an applied sense, the "model" is but a single component in a much larger software system users interact with. Perhaps at the core of it all is indeed a generative model -- which simply, just generates the next token. The system is what really brings value.

As these systems evolve in complexity and open-endedness, the discourse will transition away from model talk. At least, I hope it does. Whether or not this is realized through futility or is a side-effect of an overstimulated culture is another discussion.

"We've implemented a new tunnelling relay method to help mitigate side-channel exploitation."

"We swapped our RAG controller with..."

"We noticed our retrieval agents communicate with out-of-cluster nodes under heavy load."

"We're utilizing WebRTC to establish realtime multi-agent collaborative assessment on media as it is ingested."²

I admit, such hypothetical headlines sound a bit too technical but I'd be happy as peaches if more marketing sounded like this. Anyway, out with model talk, in with systems talk.

I am hopeful we'll start to see, in detail, how these systems communicate internally and how much freedom the components in the system have.

resources in footnotes ³