I'm watching a conference talk right now from 2 weeks ago: "I Hated Every Coding Agent So I Built My Own - Mario Zechner (Pi)", and in the middle he directly references this.
He demonstrates in the code that OpenCode aggressively trims context, by compacting on every turn, and pruning all tool calls from the context that occurred more than 40,000 tokens ago. Seems like it could be a good strategy to squeeze more out of the context window - but by editing the oldest context, it breaks the prompt cache for the entire conversation. There is effectively no caching happening at all.
They are working with the same tools and knowledge like Anthropic does as Caching practices are documented. And they have as much incentive as Anthropic does to not waste compute. Can we stop acting like people who build harnesses be it Opencode oder Mario Zechners Pi are dumbfucks who don't understand caching?
can you elaborate on this? diversifying compute doesn't create more compute - is it that the different LLM vendors have different peak times and so spreading themselves over more compute vendors spreads peak load?
Yes but if there are three banana shops around and there are five banana addicted people living nearby the number of bananas available on average for every person is not 15.
In other words, if all ai companies need more compute that a single provider can provide, then there's just not enough of it. So the question "why everyone partners with everyone" must have a different answer.
It's not really "creating more compute" it's just a natural outcome of everyone desperately grabbing whatever becomes available. The dynamics make sense for all parties involved.
Firstly, it's very clear now that everyone is seriously crunched for capacity (like, each of the hyperscalers' backlogs -- i.e. capacity for which payment is committed, but as yet unsatisfied -- are in the double-digit billions.)
So as the compute providers bring more capacity online, everyone with demand wants to get a slice of that. Like, why would anyone NOT dive in and try to secure some capacity for themselves? Especially when the rate of capacity growth is constrained by the availability of GPUs and energy and data center buildouts, which is measured in years.
On the flip side, why would the compute providers NOT want multiple customers? It creates competition and drives prices up.
There are likely other forces at play too. For one, none of the parties - the model providers and the compute providers, with some of them like Google being both -- wants to get too dependent on any of the other parties, but they also want to secure a slice of each others' future growth, so they're all partnering with each other. Obviously, Google wants Gemini to win and Microsoft wants Copilot to win, but as a hedge, they'll be happy hosting their competitors' products and taking a cut.
This is partly the origin of the "circular investments" concerns. The scale at which this industry is growing, all these players have enormous mountains of money that they must invest to secure their future, but they are also the only players that can operate at this scale, and so the only place they can invest that money in is each other.
yes, what we need is for charities to operate on a quarterly reporting cycle, so that their administrative overhead increase, and (like public companies) they can be myopically focused on short-term performance.
It's truly ridiculous how detached whoever created this LLM-generated website is from the inner workings on non-profits. It's never the case that someone gives a large sum of money to do whatever the fuck you want to do with that money.
They're usually given for a fixed period of time to do something grandiose by the end of it and the NGO has to report how they've spent every cent of it, usually at the end of that grant, but sometimes along the way as well (every X months). Not to the IRS, but to grant-givers.
have you gotten a terminal interface on your phone to be acceptably usable? I haven't - not without a real keyboard attached in any case. too many parts of the UX are designed for a true keyboard.
Someone is going to solve this with a non-buggy app, but it really needs to have all the features of Claude code. Everyone is a power user in this segment
how is it different or better than maintaining an index page for your docs? Or a folder full of docs and giving Claude an instruction to `ls` the folder on startup?
It's hard to tell unless they give some hard data comparing the approaches systematically.. this feels like a grift or more charitably trying to build a presence/market around nothing. But who knows anymore, apparently saying "tell the agent to write it's own docs for reference and context continuity" is considered a revelation.
this is likely in reference to the fact that dicts have maintained insertion order since Python ~3.6 as property of the language. Mathematically there's no defined order to a set, and a dict is really just a set in disguise, but it's very convenient for determinism to "add" this invariant to the language.
Sets use a different implementation intentionally (i.e. they are not "a dict without values") exactly because it's expected that they have different use cases (e.g. union/intersection operations).
Debugging is a completely different and better animal when collections have a predictable ordering. Else, every dict needs ordering before printing, studying, or comparing. Needlessly onerous, even if philosophically justifiable.
Sure. But they're probably not going to have significantly different content than their normal motion for summary dismissal, so it adds, say, 25% more work for the courts compared to the normal process of motions for summary dismissal, while greatly benefiting individual defendants.
most of the users of those third party harnesses care just as much about hitting cache and getting more usage.