I wrote something original in a language learning grammar online. I coined a term to describe something about how a certain language with negative language functions.
I tried talking to an LLM about it and sure and enough, it had scraped it and learned to use that terminology and explanation that I made up. I asked the LLM, "Where did that concept/term come from, who made it up?" It did a bunch of searching and tried to site a whole bunch of other sources which did NOT contain the terms of the explanation I had written. It simply would not cite or mention my source. It kept parroting my material semi-correctly and hiding the source. By any other actor that would be an egregious act of sloppiness, dishonesty, and plagiarism.
Generative AI is speed-running a widespread corruption of truth. I do not believe any of the gains are worth this.
Your comment illustrates that AI companies are playing a kind of shell game. On the one hand there is theft of IP and violations of copyright, in the middle there is the obscuring of where things come from, and on the other hand there is the selling of it via tokens.
More countries should debatably follow what Japan is doing[1], where they are creating new laws to force AI companies to show their training data and how they collected it.
Your text may well have been in the training corpus, but not searchable with whatever terms and search engine that LLM used after you prompted it. It doesn’t have recall of sources of documents comprising its training corpus, unless the source is widely cited in other of its training documents. Sourcing and provenance aren’t currently an intentional part of LLM training.
That may already have been obvious, and central to the complaint, but I was just knee-jerking to the anthropomorphic language.
I would assume the parent understands that. The criticism is about the LLM behavior this results in. Being able to explain a behavior doesn’t necessarily excuse it. By some moral standards, you wouldn’t have trained an LLM that way, or wouldn’t have made it available, given the predictable outcome.
France, 1658 <-> World, 2026. AI can also do the reverse : for example very few in the world know that the term midwit was originally coined in french by Blaise Pascal (« demi habiles ») centuries ago.
Yes, that bell curve meme, he wrote about it. It’s not explained anywhere because of the language and culture separation between
french and english. But if you ask enough questions to AI, the AI will eventually find about Blaise Pascal and tell you about this unknown link. It’s magic.
There used to be a meme on Slashdot: "In Soviet Russia, X Ys you!" It came from a joke by Russo-American comedian Yakov Smirnoff: "What a country! In LA, everybody is looking for a party; in Russia, the Party comes looking for you!" But I long wondered if Smirnoff was influenced by his countryman Leon Trotsky who observed: "You may not be interested in the Dialectic, but the Dialectic is interested in you."
Well done Confucius, you've invented a new creditpocalypse. Its when llms hallucinate new authors for new words. It almost makes it worth starting a blogsurection!
A day ago blogsurection had zero Google search results, today it has 4 like: "The Rise and Fall of the Blog" and the search llm sniplet explains what the word means.
It's more than that though. I authored the original work. Then the Baron in the Castle collected the work and uses it without fully understanding it. The Baron gives it out but in a partly-true, distorted form. He misunderstand how it applies and teaches people false clearly facts with it, with an air of authority. But when people try to find the source argument to see if the Baron is really on track, they have no idea. The Baron is just confidently parroting a compelling sounding set of words.
For me the issue is more about the loss of truth than the loss of my IP.
That is pretty weak logic. I mean obviously it is the Baron who becomes wealthy. But what follows from this?
By the same logic if Baron bought her baroness masterful dress to enjoy her look in that dress, and you when walked past the Castle saw the baroness in that dress, should you compensate to the Baron or not?
Or like the difference between chemistry and alchemy? Understanding and reasoning with the building blocks as opposed to throwing random stuff together, trying different things and hoping it somehow produces gold.
I just pulled a PWA app off the app store and I am going to keep it a pure PWA. It's installable, offline-first. I know I feel like I'm swimming against the current but I would love people to try to push for awareness of PWA installs more.
I'd love to see people distribute their apks directly.
Itch.io is an option for people who don't want to set up a site or handle payments. While they focus on games, they're cool with distributing non-games:
I personally have a TrustedWebActivity. Most update doesn't need a review because I only need to update server-side. Behave like a PWA while being an Android App. (And you can implement Android Auto, other native stuff)
It allows the users to discover my web app through the Google Play Store.
My app was also available as a PWA but I mostly push the TWA currently.
Would I be wrong to assume that you are building end-user applications?
If people use AI for libraries, OSs, and mission critical software, the apparent productivity gains would have to be weighed against the reliability and performance hits that bubble up to the things that are built on them and rely on them.
In my experience - a robust testing harness will get you the safety you need. And most software you describe has such testing.
I think the Bun port is a great example where testing enabled a very successful implementation. (Both the original tests themselves and runtime comparisons to the previous implementation.)
Yes of course, but you also understand that as Dijkstra said "tests cannot show the absence of bugs."
Tests can show you problems, if you can find them, but they cannot show that there are no problems. Property based testing or fuzzing gets your more coverage, and is a good step, but it is still nothing compared to proving things or understanding how something is built and that it is solid. Testing works towards checking for reliability and robustness, but often it's only 10% (?) of the job.
Well, yes, I understand that testing does not create a provably correct solution. But I'd love to hear the source of your "extremely inadequate" or your "10%" claims. I mean - there is a reason why it's used extensively in software engineering - right? Or don't you see value in that, either?
I'm curious - is there any data on the Bun port error rate? I think that would be very indicative of how successful or not the testing is.
The fact that a similar argument can be made does not mean that it is wrong.
Socrates famously made the opposing side of the argument against writing. Which is why we mostly know of him through Plato, who did believe in writing.
And to your corruption example. If you live in a society where corruption is normal and expected, you will be worse off if you are unwilling to be corrupt. It is indeed a local optima. But we are all, of course, better off if we live in a society where corruption is punished. To me, the worst thing about modern US politics, is that it's encouraging us to see ourselves as living in a world where corruption exists and is tolerated.
There's also often the obligatory "well, we still have to use these tools so perhaps we could use them better like this." As if just not using them wasn't an option.
I tried talking to an LLM about it and sure and enough, it had scraped it and learned to use that terminology and explanation that I made up. I asked the LLM, "Where did that concept/term come from, who made it up?" It did a bunch of searching and tried to site a whole bunch of other sources which did NOT contain the terms of the explanation I had written. It simply would not cite or mention my source. It kept parroting my material semi-correctly and hiding the source. By any other actor that would be an egregious act of sloppiness, dishonesty, and plagiarism.
Generative AI is speed-running a widespread corruption of truth. I do not believe any of the gains are worth this.
reply