It’s an interesting view-point. What are the “right reasons” to pursue an intellectual discipline? To selflessly further the total sum of knowledge?
> No reason to mourn them leaving
What field? Who’s leaving?
The conclusion has to be that human involvement in these pursuits is facing existential crisis. “Teach me, oh Oracle, the mysteries of the Universe”. “For a fee”.
Both the glory and joy of discovery now belongs to the machines (the owners of those machines). Perhaps that’s fine and well, but I read Anathem and feel the strong yearning to be the intellectual monk. And yes, to fight and debate and think; and obtain my place in history. The history of man.
"To selflessly further the total sum of knowledge?"
Yes. I think you are saying this sarcastically but I'm not sure... This is why you should get involved in research at the frontier.
"What field? Who’s leaving?"
Mathematics, and I'd extend it to other academic disciplines. Do you care more about the results and the knowledge or about who gets credit?
The drive for "glory" has many negative effects in academia that do not get enough recognition. Just look at the replication crisis in psychology for an example. The fact p-hacking was rampant among psychology "researchers" was well known even in the 90s. I remember my professors in grad school for physics joking about it way before it became a mainstream-ish story.
So why did this happen? Because there was no one with institutional power in psychology who both understood the problem and cared. No one who was interested in the furthering the "total sum of knowledge" rather they were interested in your "glory."
Automation is only good if you're putting blue collar people out of work! White-collar is out of bounds! People should do the work where they add unique value
Is the purpose of a system really what it does though? Obviously there are side effects and tradeoffs in any complex system. Is the purpose of a vaccine to make you sick - no, but that is a side effect of some vaccines on some people.
If we apply the "purpose of a system is what it does" to itself, this methodology is more a way to generate interesting (and cynical) hypotheses than to get at the truth.
The purpose of a system is not, by definition, what it does. It is what it is meant to do. That saying really gets under my skin because it is so obviously untrue. By the logic of the saying, the purpose of a hospital oncology floor is for their patients to die. Which is obviously ridiculous, but true under the "it's what it does" thinking.
I didn't give a definition of evil, so "close-minded definition" is just a dishonest way of expressing that you subjectively disagree with me.
Likewise with trying to make me judge an objective threshold for pollution. It really should be that any polluter needs to morally justify their actions. I find "mostly illusory productivity improvements in white-collar employment" to be an especially indefensible reason for firing up new natural gas generators. Let alone the real motivation being "billionaires want more money." The level of pollution from training and deploying generative AI is simply evil.
And your last question is just pure cynicism. The real sin you're committing here is contemptuously dismissing ethical concerns out of hand.
I assume you use LLMs. I don't. It truly seems like using LLMs makes people stupider and more unethical. Some cognitive work should not be outsourced, especially not to a mindless stochastic parrot.
Obviously I mean the definition of evil implied by your statement that they are indefensibly evil...
Again, what level of copyright violation is evil? Downloading music without paying for it? Apparently using images/text that artists have made public is evil, so I'm curious what the bar is here for evil.
Using electricity to improve white collar efficiency is also evil. Good to know
Thank you for using the phrase "stochastic parrot". I only request that you use it at the beginning of your long comments, rather than the end. It saves time.
"LLMs are stochastic parrots" and "stochastic parroting is a powerful medium for computation" are not contradictory statements. [Lisp is a mindless symbolic list processor.] The inability of certain people to accept this speaks to a decades-old contempt for scientific thinking in AI.
> There's definitely going to be cheap or open source models
What makes you think your "cheap or open source model" running on your piddling desktop cluster will be able to complete against a SOTA one running in a billion-dollar datacenter?
It's a cyberpunk fantasy. It won't work out that way.
Local models that run on a laptop (not even needing a "cluster") are already better than ChatGPT from a couple of years ago. Yes, Claude and ChatGPT today are certainly better than these local models, but they can't keep getting better indefinitely -- there's only so much info to scrape. When they hit a plateau, it is only a matter of time that consumer hardware will catch up to it.
While that's most likely true, it rests on the assumption that consumer hardware stays affordable enough, and isn't locked down to disallow running "untrusted" models. I would have never believed that these assumptions could ever turn out false, but the recent developments have shown that even if unlikely, it's not impossible.
Maybe? We dont really know this right? People have been saying this for 5 years now and the models are still getting better. The companies running the frontier models have already scraped everything on the web, but the models are still getting better, even if it's only marginally better, with each release. Maybe eventually some company will actually achieve AGI/ASI, who knows..
I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.
> I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.
And then they take that model, put it in a billion-dollar datacenter, and kick your desktop cluster's ass with it.
For those of us who care about the answers to these questions, rather than who gets credit for doing it, we will welcome any faster means of solving these problems.
The trick is, at that time most of the possible mass range was excluded experimentally, so it is a bit less impressive. I'm not sure how much tuning went into it (possibly none)
reply