The consumer inference conundrum:

I see a lot of consumer-oriented inference projects working to address the high cost of frontier APIs
Methods vary, but the premise is to provide cheaper inference vs aggregators like OpenRouter
The challenge here is one of timing
Today, consumers get highly subsidized frontier AI
Plenty of options for consumers to get SOTA inference plans starting at $10 and the $200/mo plans offer an insane amount of usage
There just isn't nearly as much consumer demand for API pay-as-you-go inference given the existence of these sub plans
*enterprises are a different story
So the key assumption for these consumer-oriented projects is that the subsidies will go away at some point
Many expect the sub plans to become more expensive or labs reduce the limits, eventually forcing most everyone into API billing
Classic bait and switch
But if/when these changes happen is a complete unknown
Therefore if you're a builder, you could be right on the premise but too early on the timing
The best projects working in this space need to find pmf in use cases outside of high-volume consumer API inference while building capacity to be a low-cost provider if/when the bait and switch occurs
Not an easy needle to thread