Over the past few years, we have seen amazing hardware-software codesign when it comes to AI infrastructure. This isn’t necessarily a trend, and we’ve seen almost every infrastructure team do this: Google with TPUs, OpenAI with Jalapeño, and Etched as a newer example. Furthermore, the biggest thought leaders (cc Dylan Patel) are also really long the hardware-software codesign premise as an unlock for future innovation and of course, revenue at scale. This is really amazing stuff, but in a way it isn’t the low-hanging fruit that people think it is. In fact, I would make the argument that this is perhaps a harder and more nebulous problem that will take time to solve. I think the easier problem to solve first is intelligent infrastructure in a physical sense.

With the progress that we are all seeing around us, I think that we have forgotten what America really runs on: the industrial backbone. Take for example the room I am writing this in. I’m sitting in a friend’s lecture at Berkeley in Lewis 100. I see and hear around me the following:

• A speaker system sending microphone noise around the room

• A hum of some sort of cooling/AC

• A projector

Take those three devices as examples. Right now, these physical parts are manufactured and shipped the same exact way they would have 20 years before. Furthermore, these parts aren’t really “monetized” in the same way that AI infrastructure today is, and my guess is that they probably should and eventually will be.

If these were GPUs, we would have some monitoring running on them to watch GPU-Utilization (nvidia-smi), power draw, uptime, and maybe even who is running jobs on it. In my own research group, the Sky Computing Lab, I know that is the case with compute. Compute is managed, reviewed, and monetized properly. If someone’s been running a dead job for 6 hours, it’s very likely questions will be asked (for the better), and someone else will begin to work on their jobs. It just makes sense, because we (as a people, not Sky) are compute-bound and we want to use it to its full capacity.

Given that we care so much about compute, my question is why we care so little about other physical infrastructure? At the end of the day, it’s still a cost that we want to optimize. A conservative estimate for a single lecture hall (of which > 10 exist at Berkeley, plus hundreds of smaller classrooms) comes out to $8,100/semester. It doesn’t take a lot to do some back of the envelope math and scale up that number for the number of rooms and amount of infrastructure on a single large college campus.

ChatGPT’s estimate of cost/semester of a single lecture hall at UC Berkeley, full chat is here

The idea here is that a single management layer on top of this physical infrastructure is an obvious but amazing product. The way I think about implementing this would be more of a retrofitted approach, where either a) write in hardware-level code that can track this, repackage it and then abstract it up to software-level or b) build sensors on top of these infrastructure pieces that stream data.

Now, we got a little off track because schools are not exactly America’s industrial base, but I hope the idea is clear. Intelligent sensing on old hardware is a clear long.

Who is doing this today?

There are a few companies that are doing a great job of this and that I think are severely undervalued. Here are two:

CVector: CVector solves this exact problem for chemical plants, dispatchable power, and metal foundries. In reading through their case study with Ammobia, I realized a few things:

• This market expands to live economic data, which can inform decisions like choosing production methods or prioritizing certain production when prices are low

• This sort of tool does what I love, which is ties together organizations inside a company: R&D, engineering, and FP&A all seem to benefit here

Lumaril: Lumaril is a team of ex-Palantir operators working on this same problem for the same teams, and I’m incredibly long this business. The idea that we will have autonomous decision-making in factories is not something of the future, it is here, and Lumaril is positioned to do this.

As a math person, I love tractable problems. Of course there is beauty in solving something of open-ended form (as we are seeing today especially), but solving something of closed-form has the satisfaction of a guarantee. Everyone has seen the below chart in an economics class, and we know the exact definition of perfection here. What makes me so long these businesses is that there is a measurable attainment, and algorithms will reach that equilibrium point eventually.

Alternative #1: Industrial tools are built with AI embedded

This is a possibility, and one that would likely hurt the above companies. However, there are two reasons why this is less probable to happen soon:

• Buying cycles for physical infra are slow. If you’re buying manufacturing tools for tractor parts or power generation, you’re not refreshing these tools yearly. In fact, when a tool breaks, the first thing you consider is fixing it, then repurposing it, then building a new tool with what you have, and then purchasing a new tool. For this purpose, it’s not likely that we see embedded AI entering plants for a long time (and the hope is that when it comes time to buy embedded AI tools for a manufacturer, they are already a Lumaril/CVector/etc. customer and attempt to purchase cheaper, dumber tools without embedded AI and continue this workflow).

• Embedded AI will likely still be disparate and force tool sprawl, which will ultimately open markets for companies to come in and centralize all of this (the same way we have B2B companies centralizing CRM, ERP, payroll, etc.). When this happens, the best companies to do this are the ones which already have data from these plants, which are (shocker) the intelligent infra companies I named!

Alternative #2: Manufacturers don’t want to adopt tools

This is a short term possibility and long-term suicide, so I doubt its occurrence. However, if this happens, I have a feeling that it will grow the gap between the profitable manufacturers and the less profitable manufacturers more and more, which will force the less profitable manufacturers to either get tough or die (Johnny Cash, 1969). That, in itself, will bode well for our intelligent infra companies.

Closing Thoughts

In part two I may dig deeper into the economics of these businesses, but I firmly believe that the future of America’s industrial base is intertwined with intelligence. Long industry!

Originally published on Long Intelligent Infra.