This is a pretty difficult read, in no small part because the author’s frustration with the frontier labs has become a bilious, delusional cynicism that leads him to see dog whistling and obfuscation even in cases where Amodei plainly intends for every reader to see his meaning.
Anyway,
> Dario in as many words, asks for regulation/ban on open weight models.
The big labs do want this. I understand why the author and many others want open models protected. The economic and political power the labs will have if they succeed, ladder pull competitors, and avoid being nationalized (or even if they don’t avoid that) is a disturbing prospect.
But that’s the end of the issue? There’s nothing more to think about here? The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank. I’d feel better having a tank if all my neighbors had tanks, and I’d also feel better having a tank if a few corporations were giving out tanks to people with pockets deep enough. I’d much rather be in a situation where I wouldn’t feel I needed a tank, or where the tanks my neighbors and I own don’t have guns on them.
The open weights are going to need regulation. Hopefully there’s a way to do this effectively that isn’t banning them. Denying historical and reasonably projected capability gains because that reality makes the regulation conversation a necessary one is something I’d like to see less of
> the author’s frustration with the frontier labs has become a bilious, delusional cynicism that leads him to see dog whistling and obfuscation even in cases where Amodei plainly intends for every reader to see his meaning
Please, do show some examples.
> But that’s the end of the issue? There’s nothing more to think about here? The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank. I’d feel better having a tank if all my neighbors had tanks, and I’d also feel better having a tank if a few corporations were giving out tanks to people with pockets deep enough. I’d much rather be in a situation where I wouldn’t feel I needed a tank, or where the tanks my neighbors and I own don’t have guns on them.
> The open weights are going to need regulation. Hopefully there’s a way to do this effectively that isn’t banning them. Denying historical and reasonably projected capability gains because that reality makes the regulation conversation a necessary one is something I’d like to see less of
The only ones firing the guns atop the tanks seem to be OAI and Anthropic. Like I say in the post, why don't we first see actual prosecution for felonies committed by OAI and Anthropic, instead of fear-mongering about _potential_ harms of open weight models?
The labs spent immense amount of money and effort convincing you and I, to want those said tanks. Guns atop them? They put them there. "Cyber" versions of SOTA LLMs.
Centralization proponents seem to think the labs can actually deter sufficiently driven bad actors, which would be a mistake. They could not even stop distillation without, in a way, DoSing themselves by removing thinking traces.
> Dario in as many words, asks for regulation/ban on open weight models. Ban on distillation. Waivers on antitrust laws.
He doesn’t directly propose regulating or banning open models as open models in the essay (although I agree that it’s in the spirit of the proposals). So I think you mean “not in as many words,” since it’s an inference you’re making about his intent. But he does directly call for banning distillation, at least in “authoritarian countries,” so there he is asking “in as many words.”
Your imprecision in effect claims either that Amodei is directly calling for a thing that he does not directly call for or that Amodei is hiding his intent in a case where he is not hiding his intent.
> The only ones firing the guns atop the tanks seem to be OAI and Anthropic. Like I say in the post, why don't we first see actual prosecution for felonies committed by OAI and Anthropic, instead of fear-mongering about _potential_ harms of open weight models?
> The labs spent immense amount of money and effort convincing you and I, to want those said tanks. Guns atop them? They put them there. "Cyber" versions of SOTA LLMs.
I think pacing the frontier and prosecution for cybercrime are compatible aims in theory. Maybe they’re not in practice, but I don’t think that’s the exact argument you’re making.
I don’t follow your emphasis on the labs being responsible for the guns. Yes, they are, and in large part, maybe, because of negligence and rushing when it comes to RL training environments. But why do you take the fact that the labs bear responsibility for the guns to mean that we don’t need to worry about the guns? Why do you think that open weights do not pose the same potential dangers, with a 6-12 month lag? The cyber capabilities of open weights models are advancing at a similar pace as the frontier. And for what it’s worth, I’m astounded at the thought that one or more of the Chinese labs won’t have similar incidents in the next months, assuming they haven’t already.
> The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank
You could say the same about normal computers. Where are our regulations on Kali Linux, to prevent people from bruteforcing weak WPA passwords?
The single most-pressing concern with AI is that it can accelerate the process of hacking things. This is a preexisting problem that is inherent to software and needs proper addressing. Even if we regulate open weights tomorrow, people still have uncensored GLM-5 finetunes doing whatever they want on their own hardware. The "what if" of capable open models is here today, there are no guardrails.
> Where are our regulations on Kali Linux, to prevent people from bruteforcing weak WPA passwords?
Brute forcing WPA isn’t an existential threat to humanity. It’s not like a biological weapon created using an open-weight model is less dangerous than if it had been created with a proprietary model.
Why does GLM-5 already being in the wild make you think caution about a hypothetical GLM-6 is unwarranted? It seems like you’re saying “there’s no reason to worry about preventing a problem from getting worse if you already have a mild case of the problem”
Anyway,
> Dario in as many words, asks for regulation/ban on open weight models.
The big labs do want this. I understand why the author and many others want open models protected. The economic and political power the labs will have if they succeed, ladder pull competitors, and avoid being nationalized (or even if they don’t avoid that) is a disturbing prospect.
But that’s the end of the issue? There’s nothing more to think about here? The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank. I’d feel better having a tank if all my neighbors had tanks, and I’d also feel better having a tank if a few corporations were giving out tanks to people with pockets deep enough. I’d much rather be in a situation where I wouldn’t feel I needed a tank, or where the tanks my neighbors and I own don’t have guns on them.
The open weights are going to need regulation. Hopefully there’s a way to do this effectively that isn’t banning them. Denying historical and reasonably projected capability gains because that reality makes the regulation conversation a necessary one is something I’d like to see less of