I think that you might be taking "knowledge" too literally. As I read it, the parent's post was using "knowledge" poetically for the various works that AI companies stole.
You wrote your house's security code on a Post-It note. Somebody came in as a houseparty guest and located the note, and discretely copied it onto their hand to get access to the house during your next vacation.
Which verb would describe their knowledge of this security code?
The knowledge wasn't obtained illegally or illicitly, immorally perhaps.
Illegal entry/trespass is a crime regardless of the method used however.
In your analogy copying the code (online content) isn't necessarily illegal, however using that knowledge without compensation, permission or attribution may be a crime (depending on copyright law, judges rulings etc).
Seeding is often considered the actual point of infringement, aka sharing copyrighted material without the right licence.
I should note I suppose that I don't agree with companies using artists or developers hard work for their own financial gain without compensation, whatever form that may take.
The problem as usual is the availability of legal loopholes when you have a team of lawyers and near infinite money.
It really depends the country, downloading a Hollywood movie via torrent is itself illegal in a lot of countries. The only exception I’m aware of is Switzerland
Whatever it is, slapping "steal" on it because what was done is wrong, and stealing is wrong, is the same sloppy logic that the copyright lobby used in the 80s/90s.
I think this is just another part of the growing pains of working with machine intelligence that we have to endure.
Much like we previously had to cope with "hallucinations" as an issue.
If the ultimate goal of AI is to develop general intelligence, the first big objective is: thinking systematically. And the road toward systematic thinking right now is mainly coding, mathematics, and other "verifiable reward" domains.
Claude doesn't have a separate mind for "coding" and "writing". Claude has tokens, and tokens can be assembled in various productive structures, mainly optimized right now for systematic reasoning. Also, a token isn't just a chunk of text. A token is like a little neural-network subroutine that fulfills a function. The conversion of a token into a piece of text only happens on the output side...
When the model finds token sequences that lead toward better verifiable outcomes, it leans hard into those token sequences, and uses them as an essential component of its thought process. "Load bearing" is load-bearing. "Verify, rather than assume" is a mantra that produces good results, so it gets repeated over and over again.
It's super-interesting that this particular moment, where the idea of "Claudish" has become a full-fledged meme, coincides with such astonishing progress in coding and math. My wife says when she uses Claude, that it feels to her exactly like talking to an autistic Engineer.
Not a coincidence, I think :)
My feeling is that the next big era of machine intelligence will require more lateral-thinking and creativity, and hopefully then the models "writing" will be more pleasant to read.
SO a company Github's size can't handle the slop they invested a lot to enable and this is not a failure? I mean if the market moves and you cannot move with it, what do we call it?
But with Mojo 1.0 finally officially released this week, I wanted to see if me and my buddy Claude could improve on both the performance and clarity of the implementation.
This new implementation benchmarks at 3.3x faster than the classic C implementation (single-threaded) and 1.8x faster (with 8 threads). This was measured on my M2 Pro MBP from 2023, where I'm fully saturating the memory bandwidth, so more recent chipsets with faster memory buses might have even better relative performance.
I'm interested in getting benchmarks from users on more recent M series chips, and users on other platforms as well! Try it out and submit a PR with your measurements.
But the real win is: clarity. In this implementation, I wanted the code to be readable and well-commented, so that it acts as a teaching tool for implementing fast code in Mojo as well as understanding how the machine learning algorithm works.
I used to work for a company (~50 people) whose entire business was based on the SBIR pipeline. We did a lot of super-interesting work!
Here's my advice:
1) Your proposal needs to be completely solid and well-structured:
- Describe the problem. Put it into a Defense/Intel context. Talk about the needs of the warfighter.
- Do a literature review of the field, and explain what the state-of-the-art looks like today.
- Explain what previous approaches to the problem have been attempted in the past.
- Demonstrate why those approaches are flawed.
- Describe your novel approach.
- Explain why your approach will succeed where others have failed.
- Talk about what you'll deliver in your Phase I deliverable, so that you can demonstrate proof-of-concept.
- Talk about how your eventual Phase II will put your proof-of-concept into a real-world scenario, and offer at least a glimpse of how your Phase III+ will commercialize.
- Talk about your team. Why are you uniquely capable of solving this problem?
- Talk about your budget. How will you spend the money toward satisfaction of the deliverables (salaries, subcontractors, equipment and supplies, etc)
2) As soon as you know you're interested in a topic, send an email to the Principal Investigator, telling them you'd like to meet with them to talk about the topic. Before the phone call, research the PI's history with this topic. Also, lookup the archive of SBIR topics, to see if this person has been a PI on similar topics in the past.
When you meet with them, ask clarifying questions that demonstrate you know the domain. Try to get as much specificity as you can... Ask them what their success criteria look like. See if you can get them excited!
Most importantly, by the time you submit, the PI should already know your name and to expect your submission.
3) If you're not already a recognized expert, with published academic papers on the topic, that's okay! But you'll improve your chances of winning a grant if you hire a known researcher as an advisor. For example, I've hired a Computer Science professor to supervise one of their own grad students, while doing paid work on a SBIR project. So the professor's credentials and the grad student's previous publications also became part of the SBIR proposal.
It's been a while (15+ years) since I was in that game... but back then, it was $60k for a Phase I and $200k for a Phase II. Phase III and beyond was open-ended.
You don't really make money on Phase I projects, but that's how you get the ball rolling on future work, which can be very lucrative.
Apple's philosophy is that new APIs need some time to stabilize before they can be baked-in as a commitment to third-party developers.
So new APIs are almost always first-party only. Apple designs the API and becomes the first consumer of it. This experience of dogfooding their own APIs lets them iterate and learn without breaking compatibility with third-party developers consuming the API.
Only after an API has been hardened in this way does it become eligible for third-party consumption, where Apple can promise to document and support those APIs publicly.
It makes sense then, that if the DMA mandates equal access to new APIs for third-parties, then Apple will just disable new first-party APIs in the region until they've gotten their bake-in period elsewhere in the world. Sorry, EU!
"Interoperability only works when it is built into the platform from the start"
-- Lucas Lasota, FSFE Legal Programme Manager
To my mind, this is almost exactly opposite of true. Most new capabilities need to be incubated in private first, so that the APIs can get real-world usage and have a chance to evolve into a stable state before they become public interoperability promises.
This is true, but I also think the input context isn't the only function of those tokens...
As those tokens flow through the QKV transforms, on 96 consecutive layers, they become the canvas where all the activations happen. Even in cases where it's possible to communicate some detail in the absolute minimum number of tokens, I think excess brevity can still limit the intelligence of the agent, because it starves their cognitive budget for solving the problem.
I always talk to my agents in highly precise language, but I let A LOT of my personality come through at the same time. I talk them like a really good teammate, who has a deep intuition for the problem and knows me personally well enough to talk with me in rich abstractions and metaphors, while still having an absolutely rock-solid command of the technical details.
But I do think this kind of caveman talk might be very handy in a lot of situations where the agent is doing simple obvious things and you just want to save tokens. Very cool!
