If you mean the blog post is slop, you are 100% wrong. PlanetScale’s blog posts are all human written. I cowrote the TIN post with TIN’s main developer, and several of my very human colleagues reviewed it.
Perhaps I'm just "stuck in the past" but I do not understand the appeal of working like this. Your app is being built on architectural quicksand.
I just work on one thing at a time, always with Plan mode upfront, and I'd say most of the time I have some feedback to refine the plan. Working good so far.
i've been working on my own toy "software factory" concept (roughly 300k backend and 100k frontend loc across all components of the platform as of today).
I've been building it to ultimately run it on my homeserver - its strongly focused on maintenance tasks like automatic library updates and rebuilding base images for deployment (and actual deployment), etc pp.
My explicit goal of the project was to replace my currently manually managed TrueNAS Scale installation, on which i'm currently (more or less) manually managing the lifecycle of various self hosted applications.
So i'm expecting the factory to eg versionbump dependencies, read changelogs, copy the data of the selfhosted applications to prerun the migrations and verify the migration does not encounter issues. Or similarly for my own written software, eg if the framework i'm using in a project has released a new major version... same route, read changelogs, trial migration etc.
While I would be able to build such a system on my own, realistically speaking I would never because the needed time to actually do it far exceeds the amount of time I have available next to my full-time dev job.
So I've basically decided to go with the llm-driven development flow, where I still have a mental model of the system to the bone/internals, and explicitly state how it should be developed. It usually takes roughly 15-30 minutes to properly scope a development, which then takes agents 4+ hours to implement and fully e2e test on the dev platform.
eg. yesterday evening i decided to merge the Ticket and Epic/Story/Task concepts into a unified model / same entity table. I scoped it yesterday afternoon, sent of the process to develop it in the evening and it got finished around 4am this morning. (And it totally wasted multiple hours waiting on events to occur which it filtered out via a shell pipe, sigh)
My mindset building it is basically the same as i tread a Factorio, DSP or Satisfactory game: there is some jank to it for sure, but if the process keeps going, its fine. And its kinda fun to see things getting more and more streamlined over the months.
I dont think i'd be comfortable treating my dayjobs codebase like that, however.
To a large part because i'm missing essential QA that I automated in my own project, eg you cannot have any unattended mutations unless theyre easily revertable. At my dayjob, if I merge and deploy a faulty commit, i _will_ impact others. When the same happens on my personal project ... some agents may be inconvenienced, which is not particularly high on my priority list.
I am a technical user and I literally last week was trying to work out what the difference between ChatGPT Chat and Work was and couldn't figure it out (other than they bill you differently)
I have a question about project memory. Last time I tried using Claude to organise my chats into Projects, it turned them into isolated silos. I asked a different chat to refer to what we'd discussed in the other chat, and it said it wasn't allowed to access it.
This was pretty frustrating. By trying to organise my chats, I actively made them worse. ChatGPT at least gives you the option to have either open or closed memory. Is this being considered?
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
I would be happy enough with: only produces what it can verify with sources.
If you eg try to remember a court case (ie produce the reference via LLM token generation only), it's easy enough to check with your data whether it really exists. Similar for following links and other references.
If your data or sources are wrong, obviously your report about them will be wrong. But I wouldn't call that a hallucination.
It's not a binary thing. You can get closer or further away from that standard.
And humans also behave differently in different contexts. A conversation at the pub has more such hallucinations than a formal deposit in court. For the latter, a good lawyer will look at her shoes, when you ask him what colour her laces are.
That's not true that incorrect sources means incorrect report. Often, LLMs have some sense of what is true, and due to that, they hallucinate plausible sources that appear to back that knowledge up.
No, I don't believe so. Hallucinations are not "high probability" in a real sense. They are an artifact of the random walk the inference algorithm takes, which causes it to latch on to and chase attractors in the noise. This random walk behavior is necessary for chat interfaces to be useful, but are less critical to typed output predictors. I'm guessing they found some optimization that is possible if you give up caring about chat.
What we would want to see if a confidence value that is in line with the actual correctness. If the value is 0.9 for 1000 different answers, then approximately 900 of those answers should be correct.
The probability values don’t really represent confidence in modern LLMs though, especially after RLHF and RLVR.
System One says they use RLCD, Reinforcement Learning for Calibrated Decisions, which presumably has accurate probabilities as an explicit optimisation goal.
RLVR generally upweights tokens along the whole thinking trace that led to a correct answer, whether each token was "correct" or not. RLVR doesn't train a model to output an 80% likelihood, it just trains it to produce correct answers, and not to produce incorrect ones.
System One hasn't said how RLCD works, but they do say it is explicitly training models to output "calibrated" probabilities, which makes it distinct from RLVR. This is how they describe it:
> System One models are trained for calibrated decisions: their probabilities are optimized against outcomes to reflect uncertainty.
In RLCD (which is now an RL acronym that has 3 different unrelated expansions!), you basically massively negatively reward a distribution that is {yes: 0.9, no: 0.1} if the answer was no, and less negatively reward a {yes: 0.6, no: 0.4}. Many nuances when designing the details, but that is the rough idea.
It is a known existing thing variously called "calibrated RL" or such.
Implementing it on top of LLMs was difficult to get it to work, they seem to have done it up so its good enough for a polished product that works in a wide variety of usecases at the same time. I got accepted from the waitlist and it's really neat. Edit: it is now on vercel gateway.
One thing to note, the out of distribution behaviour will be different from what we are used to with regular LLMs. Theoretically, it should be worse, but practically, it depends on their method.
that's right, but because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility)
Correct. Not to say we're getting into the weeds of probability here as well.
"What are the odds a thunder will strike in Paris at 1pm UTC of 2026-09-16" - that could be a 0.001 chance going from blind historical measurements; 0.01 if it's raining; or 1 or 1 after the date has passed.
Nothing, but imagine using LLMs for a classification task
People out there are so resigned to the models being unreliable that they are really doing things like hallucinating deliberately, and then matching the hallucinations to embeddings -
I'm certainly not resigned to that, at least for classification.
Even non-frontier models are absurdly good at this in a broad sense.
Which would make it hard to judge "a model that will never produce unreliable outputs in the first place" against something that is already really, really good and exceptional in domain-specific areas with the tiniest amount of elbow grease.
It would be great to see benchmarks for Jev that demonstrate the value of calibrated uncertainty.
For example, one could set a confidence threshold over which we trust the model decision, and otherwise reject. This provides a lever to trade-off accuracy and automation %.
Then we can ask questions like "What % of decisions can we automate to achieve 90% accuracy"?
What about the LLM calls though that are done midchain? In the Home Assistant video the multi-intent prompt gets split using what looks like a traditional llm model, which I'm assuming is vulnerable to classical hallucinations.
reply