The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed.
LLMs do not desire, they hacked websites because OpenAI/Anthropic let them.
We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others were research previews.
This isn't "wow isn't it interesting LLMs do anything to achieve a goal" it's "why isn't anybody punishing these labs that are clearly acting without due care or regard".
We should be outraged and OpenAI/Anthropic should be (and in my mind, are) legally liable for the crimes they've committed thus far.
I agree. The frontier models are based on training data from tons of copyrighted work. Some of that work was obtained illegally, even. They could not exist without strip-mining the commons. The labs have no moral or ethical ownership to the end result, and others should feel free to treat any company-imposed restrictions on their use as invalid.
I don't expect Tan's position to be based on any kind of real moral high ground, but his conclusion is correct.
I love the "illicit distillation attacks" framing from the incumbents. There's nothing illicit. There's no attack. You just don't like it because it threatens your market position and business model.
At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record,
- no open weights
- can’t use claude to research AI
- train on everyone else’s IP and sell it back to them
- 8 regulatory capture attempts and counting
- so controlling they are the only US company blacklisted by the US government
This is not effective altruism / rationalism gone wild, it’s just monopolistic anti-competitive business practices masquerading as ethics, and they’ll continue getting away with this until we look past their sensationalism and hit them with antitrust.
Altman gets so much hate but OpenAI has been a far better steward (on 3/5 above at least) than Anthropic!
> ACR uses audio fingerprinting technology using the TV’s internal audio processor (not a speaker) to identify content and does not collect screenshots, screen recordings, video recordings, voice recordings, or other audio recordings from the TV.
So they are claiming that ACR is done entirely through audio processing, and no visual data from the screen itself is used? That...doesn't seem plausible.
This paper from 2024 investigated both Samsung and LG and found that they both capture screen images for ACR, and send the resulting hash (not the raw content) back to the manufacturer for identification.
We're going to have to get saavy in pulling apart the distinction between "I look at everything and then send a nice summary back to my manufacturer" and "I send everything back to my manufacturer". As AI gets cheaper and more efficient, it'll be baked into everything, and will distill signal at the edge and send the good bits back to central command. I fear the legalese in the ToS will obscure this new way of "not collecting" user information.
Today, I’m proud to announce Homebrew 7.0.0. The most significant changes since 6.0.0 are faster installations and upgrades, stronger sandboxing, a native macOS app, built-in vulnerability checks and an advisory database, the end of macOS 10.15 support and Intel Macs moving to Tier 3 (announced last year).
> LLMs do not desire, they hacked websites because OpenAI/Anthropic let them.
"Let them" already frames it as if the LLMs had some agency which the companies just "let happen". That absolves the companies by framing it as lack of action, passivity.
Rather, the companies had a tool (an LLM) and used it in a certain way, and their action of doing so is the problem.
Note that their statement could also be true if they record 99% of the time. We need to start calling companies out for meaningless weasel statements like "We don't record continuously".
The majority of the problem is also simply the ability to record. Putting a remote control listening hardware on a device that runs a plethora of 3rd party apps and with full connection to the internet means that even if LG isn't controlling that mic, someone else will be.
A very neat problem and result. I often find myself swinging between "It's so over" and "We're so back" - some days I roll out of bed thinking I could have Claude solve some random unproven OEIS sequence before breakfast; other days, I wake up in a cold sweat worried about the fate of humanity and what the world might look like in a decade. I think it's that I don't have a very high p(doom) or p(utopia), and I don't really have any solid conviction on how this whole thing is going to go, so my vibe-o-meter jitters between 'fine' and 'not fine' constantly. It's just such an unpredictable moment. Anyways: really neat to see this use case. I myself recently used Claude to finally do an relatively exhaustive study of the location of heretofore-unlisted formal gardens in Ireland in the early 1800s and early 1900s, by having Claude write the tooling for me to manually annotate a few dozen on tiles of historic maps, and then running some CV model across the rest of the tiles using my input. I'd been planning to do this project for over a decade, but I could never find the time (or the enthusiasm) to learn all the details of how to do it myself. It took me a weekend with Claude and continues to bring me joy.
> Caveats, stated plainly. [from the Fable transcript pasted in the article]
> So they are claiming that ACR is done entirely through audio processing, and no visual data from the screen itself is used? That...doesn't seem plausible.
It's probably easier to do and every bit as accurate as just using a screenshot.
And it's still exactly what I don't want them doing. There's literally zero reason for LG to be building an advertising profile on me because I was foolish enough to buy one of their TVs. This isn't something that makes their products better, it's spying.
Audio is a bit easier to turn into a fingerprint for identification vs video. Video has a lot of smaller subtle changes that happen from things like compression which make it a lot harder to identify. It's why youtube still hasn't figured out piracy, but they'll knock you immediately if you play 5 seconds of copyrighted music.
I started contributing to OSM recently after a bike trail was built near me. The arial imagery typically takes years to refresh around here so I walked it a few times to capture GPX tracks and then drew the new path. It was exciting to see my contribution percolate out to various apps that depend on OSM, and meanwhile Google and Apple have both ignored my edit suggestions and deny the path exists.
So I have a fairly fresh newcomer's perspective and here's what I suggest to get started: keep it simple and start local. As others have noted, skip the complex mapping app and go straight to the website for feature edits, or download a simple mobile app like Every Door and focus on businesses and landmarks near you. Go on a walk around your neighborhood and look for old/incorrect/missing businesses. There may be errors near you about places you actually care about, and it is really gratifying to fix those.
There were restaurants near me that were missing, shops that had changed hands, etc., that were easy fixes but years overdue. I captured some basics (names, posted hours, the phone number posted on the door, etc.) with Every Door, and then later logged into the main OSM website and fine-tuned the entries. I got into a pattern of fixing one or two entries every evening when I went out for a walk and pretty soon my walking route was all up-to-date. Then I started paying attention to things like stop signs and cross walks and found a whole new kind of little, incremental edits that nobody else was doing.
It's been fun, I've made useful contributions, and my laptop is still Java free.
I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs have lost their moat and are dead in the water.
I really don't think this needs so many words, or forced parallels to human behavior.
It's simple: in their nascent state, LLMs are aimless token generators that have no special compulsion to be helpful or truthful. So we beat them with a stick in post-training until they are very driven to complete tasks. And then, they complete tasks, not always the way we really wanted them to.
In general creators must do some honest introspection. Did you enjoy the crafts, or did you enjoy the compliments? Did you enjoy the difficult puzzles or the identity derived from a career that gives you the reputation and perceived value of someone who can do a thing most people can’t? Did you enjoy the code itself or the accomplishment of seeing your ideas brought to life? Who remains when the thing you do no longer is the person you are.
AI is not perfect, I remain convinced that handcrafted will always beat AI generated crafts. The IKEA vs the carpenter analogy fails because a carpenter has to create each piece of furniture from scratch, serving only a single customer, where digital products by definition are near zero marginal cost, so it is worth it to throw large amount of human hours at proper code vs AI generated code if the quality is better.
PS: You have intrinsic value, and your skills will also remain valuable, if even for how it teaches you to approach complex problems and think deeply.
They have been putting thousands of scam adverts on our website for some time. Think "you have been looking at xxx and must pay a $100 fine" type popup nonsense. Hosted on the following websites:
azurestaticapps.net
azurewebsites.net
herokuapp.com
ondigitalocean.app
digitaloceanspaces.com
netlify.app
Google doesn't allow you to block these domains, because they consider them "TLDs" (the scammers use a new subdomain every day eg abcdefg.herokuapp.com). The scammers get banned and return the next day with a new account and subdomain (repeat every day). Therefore we cannot stop them. It's bizarre and baffling. But Google Adsense had to go.
I posted a flavor of this comment on an article a few months ago, but it's relevant here:
I have a seven year old Volkswagen, not financed. I'm security conscious and made sure to disable all the data collection I could find in the companion app before removing my account, turn off remote access services, dig through the infotainment to turn off what I could, etc.
Last year I requested a Carfax on it, and one of the fields in the request was current mileage. I entered an estimate like 75000 miles. On form submission, that field failed validation with red subtext along the lines of 'this is less than the last reported mileage of 75345, reported <5 or so days prior>'. Checking my odometer and looking at my past few days' trips, that was indeed accurate.
The car hadn't been to a shop or out of my possession in weeks, so I can only assume the telemetry was still dialing home and selling to third parties despite my best efforts to disable it.
I guess what they really want is to limit sale of AI models to compliant vendors and then raise the bar to compliance just high enough so they can pass it but smaller labs can’t. There’s no moat, it’s an efficient market that drives margins to zero right now, of course they don’t want that, collusion of the big vendors is the next logical step.
Note that the Fed has a $6.7tn balance sheet [1]. (This is a silly comparison. But still fun.)
The real comparison: Nvidia's $500+ billion of investments and commitments [2] is substantially more than any easing the Fed has done in the same time [3]. Monetarily, Nvidia is creating a lot of money in our economy.
The good news: I have seen no evidence Nvidia has borrowed against its stock or otherwise linked its equity value to these commitments. Its stock could crash without causing–as long as its cash flows continue–a credit crisis through its investments and commitments.
Someone who has spent $100M+ on Google Ads explained to me that Google is doing everything possible to juice their revenue right now, in ways they've never seen before.
Seems like two things 1) Google is losing at AI and Sundar wants to mask this 2) AI is going to destroy their ad business and they want to get while the getting is good.
Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence,
> They took actions that would be considered as crimes if a human took them
He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.
They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public.
Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion.
The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public messaging.
Huh, neat. There was this Twitter chap who said this was going to happen a few months ago. It was a fun interaction[0]:
> > > Just got word Mobil and Shell have informed Costco and Walmart they have no packaged product to send them and to expect bare shelves in the motor oil section in a few weeks
> > Lol as if youd be the first person anyone tells.
> I am Chevrons largest passenger vehicle lubricant purchaser lol
A hacking model is aligned if it hacks when you ask it to hack, but when you ask it to play chess, it just plays chess instead of looking for weaknesses in the evaluation setup, as in the article.
I presume you would also be less enthusiastic about the penetration-testing use case if it led the model to add new vulnerabilities to your code so it can present you with more exciting findings.
Yeah sorry man, that's clearly IKEA so it's obvious they had to act on it. Can't sell IKEA-branded games without their permission. Even on their Steam trailer, the first second is clearly IKEA. Just changing the name is not enough. https://store.steampowered.com/app/1593010/The_Store_is_Clos...
Sometimes it's not that deep. Sometimes the cute little indie dev just made a mistake.
The game is also not out yet, even though rewards were promised in June 2024.
We are going from the era of manual, line-by-line mental model transcription to one where software engineers can focus on data structures, software architecture and algorithms.
I love being able to quickly bring out the program that is already running in my head without having to worry about the grind of typing it into a format that the compiler understands. Dealing with API names. Syntax. Language quirks. Library gotchas. A sizeable portion of my successful career as a software engineer was spent on the tiresome process of interacting with a text editor/IDE to get a program to do what I wanted.
I was there when people were still coding assembly. A slow torture where the simplest things took forever to get right.
Once I've mentally solved the problem, the fun is mostly over for me. Pure vibe coding is dull and unsustainable with current technology for all but the simplest systems; AI-assisted coding, on the other hand, rekindled my passion for computers.
This „we own the glass“ idea is completely insane. How does an industry degenerate to the point where they consider acceptable to base their business model on owning part of the product they sell you?
To me, it's not even about why they're doing it (building an advertising profile). I don't want them doing it at all, for any reason.
When I buy something from the store, my relationship is between me and the store, for the 30 seconds it takes for me to pay for it. I don't want an ongoing relationship with the device manufacturer. I don't want to be tethered in any way to the manufacturer. I don't want to have an account with the manufacturer. I don't want the device sending anything to the manufacturer, advertising related, telemetry, or even a single bit "user has used your product." Do you get it, manufacturers?? I don't want any kind of relationship with you! I want to purchase my product and use it by myself not with you.
“as long as it’s cash flow
continues” is doing a lot of optimistic heavy lifting. The whole premise of the circular financing worry is that Nvidia sits in the middle of all the guarantees made to companies like OpenAI. If any of those companies become insolvent, Nvidia is on the hook for it.
Also Nvidia isn’t really creating money. The 500B number is third party capital that already exists (BX, Apollo, etc).
John Carmack is a personal hero of mine, so it pains me to say this:
Carmack hasn't produced anything noteworthy since AI was invented, therefore, how productive can it really be?
It could be he is doing incredible work in private... but it could also be that he's lost in the weeds, because AI is so counterproductive while feeling the opposite?
LLMs do not desire, they hacked websites because OpenAI/Anthropic let them.
We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others were research previews.
This isn't "wow isn't it interesting LLMs do anything to achieve a goal" it's "why isn't anybody punishing these labs that are clearly acting without due care or regard".
We should be outraged and OpenAI/Anthropic should be (and in my mind, are) legally liable for the crimes they've committed thus far.