Since it's not mentioned in the article, the distinction between open source and open weights is important. Open weights models are almost like a 'first shot is free' entry drug. Without at least the original training data your ability to meaningfully upgrade it is so limited that its utility will quickly fall behind the latest versions of continuously developed models. So much that it'll leave you craving for another release, or have you going back to the provider's API. Even simple things like moving the knowledge cutoff forward will noticeably improve the UX, and that's not to speak of more fundamental improvements like reasoning, quantization-aware training and all the goodness that's yet to come.
Sure, we can do research to bring improvements to open weights models, but it's the same thing: it's either open source or it won't benefit the general public nearly as much.
Could you put some numbers and examples behind the efficiency gap between data center and consumer-grade AI hardware? Did you include examples like the RTX Spark on the consumer side? I was always amazed at the low power consumption of unified memory style architectures. In absolute terms and even more so compared to consumer-grade GPUs. I'd be genuinely interested in a comparison with data-center-grade hardware.
It's more than the raw hardware, it's the interconnect and communication between the hardware at scale. These models are trained on hundreds of thousands of GPUs today. You _will_ start to see cross-datacenter training runs but this needs to efficiently decide when and how to communicate across datacenter, which bears a very high cost compared to intra-datacenter communication.
DGX Spark is effectively prosumer hardware, better than most consumer stuff but still not comparable to actual datacenter gear. You can't just look at TDP in isolation without also comparing performance.
Wouldn't downloading conversations be more useful? If you've input something into Claude that you don't want a future trillion-dollar US company (a bicorn?) to know, this script won't help I'm afraid. Free reminder that local AI exists and works well if you're willing to tinker a bit.
Interesting that this doesn't seem to use blockchains. Arguably it would have been a good use case. OP, could you elaborate on the reasons for the choice (if it was a conscious one at all)?
I'm old enough to remember that capitalism used to always feel like this. Even worse. Predatory subscriptions, targeting minors in some cases (remember phone ringtones?), toll numbers, unclear fees, surprise bills (my favorite - roaming fees in your home country because the phone stayed logged into the foreign network after an abroad trip). Silicon Valley/YC style startups felt like a breath of fresh air. Generous return policies, subscriptions can be canceled monthly and even do so automatically in a lot of cases (eg disabled credit card). I guess the fact that a lot of what they offered has near zero marginal cost helped. AI changed that part, so we're back to what I would consider the natural state.
The least-resistance path out of such a bad equilibrium is regulation. And they did add a lot of protections in the last decades, which probably helped too. Some would say places like the EU even added too many. But I'm pretty sure that "if you're using our product, you need to pay" would fly even in the most customer-friendly jurisdictions today.
It would, and the EU would actually be worse on this. In many if not all EU countries (DACH at the very least) you'd have to pay because you agreed to pay, and you'd have to pay all court and legal fees you caused as well - while in the US the company is more likely to just forget about it and try for another customer.
1,5k. For two months of that spend you could buy a machine that can self-host decent models, plus a year's worth of electricity. It's not up there in terms of quality, but with a bit more effort it works pretty decently. I'm completely baffled that that's not way more common, is it really just the quality?
Second here. From recent Alibaba Qwen conference: the all-in-one box (DC in a box - I think I was called Apsara, 0.6x0.6x1.5m) plug and play, 1.5TB GPU RAM, capability to run in a fully air gapped environment, any open models... All of that is roughly $300k one time.
And this box can do non LLM tasks as well.
Performance (throughput) around 20k t/s. Delivery time - around 2 months.
For any medium sized company its perhaps cheaper to just buy it once than spending 1.5k for cloud per user
I think we're all past the "bet-money-can-buy" stage. The most expensive models are an order of magnitude more expensive than the middle ground ones, so you need to be selective about what you run where.
And with a bit of careful routing - there isn't a lot stopping you sending the hard stuff to a cloud model and the average stuff to an on prem model.
I'd think for most companies the pace of change is too high at the moment. Give it a few years, a bit of a plateau in the improvements in frontier models and I can't see how many of these companies don't implode under the weight of competition on inference prices.
It's not even anything new, it's basically the mobile version of the DGX Spark. The two chips (N1X/GB10) are pretty similar in terms of architecture and specs. I don't get why this seems to be getting so much attention now.
But I like it. It's a copy of Apple's SoC design philosophy, same as AMD's Strix Halo, which I always thought was really cool both for laptops and home PCs. NVidia's traditional consumer cards pull way too much power and are too noisy to comfortably put them in a living or office environment.
I admit, I barely understand what the product does, much less how there's 50k people wanting this. This is a component you can use if you're building a DIY keyboard and want to make it wireless? Seems profoundly niche to me. Am I missing something?
Anyway, congrats on finding and reaching your market! The Internet at its best (although part of me wishes this nerd community had found a more self-hosted way of connecting online than Discord).
> Seems profoundly niche to me. Am I missing something?
As someone who dreams of someday starting a "lifestyle business", I love that it is profoundly niche.
It gives me hope that I can go out and solve a problem that is important to me, but too niche for investors to bother with, and earn some money from it.
Yeah, it sounds like there just legitimately are 50K people who are in this niche. Maybe the fact that people might assume there are fewer is why there was a gap in the market for the author to fill!
I think this is the Big Thing when it comes to making things - underestimating the market. Mechanical keyboards is a multi-billion industry, and building them yourself is a percentage of that.
That said, I wouldn't be surprised if this product managed to end up in the supply chain for a lot of the keyboard manufacturers, which would be a huge boost to sales volumes.
Given the acquisition, that fits with what I'd expect. 50k individual customers probably isn't going to be making a business worth over a million dollars for a product like this (a margin of $200 per sale seems quite high), but if it's an existing company buying it with the idea that they can integrate it into their existing stuff, the math probably works out a lot more.
True! "I am a user of this type of product" is a much less strong signal than "I'm going out of my way to solicit more information about what products are available for this and how to use them". This isn't the often-cited silly case of "I just bought X, and now I'm seeing ads for buying another X even though I don't need one anymore".
Custom keyboards are really popular - especially a few years ago. Most cases/boards are wired only. I think his product enables those to be wireless too
There’s a quite large community of custom mechanical keyboard enthusiasts. If you are familiar with the audiophile space they have similar spending habits.
It doesn't have to be for that, but yeah, that's the target. At the time, a lot of keyboard designs were based around the pro-micro formfactor, so this made it more-or-less a drop in replacement.
There are billions of people in the world. 50k is 0.005% of a billion, so 1 in 20000. This is the reason I think money/market-motivated thinking, that often leaves people pursuing something they're not especially personally enthusiastic about, is wrong for most people. If your goal is to be a billion dollar grow-fast multinational company, okay, but if your goal is to just live a comfortable life and create something neat - then it's much better to sell a niche thing that you enjoy, than a mass market thing you just want to make a buck off of.
For a gaming example of this, it's often cited somehow as a negative that "only" 14% of games released on Steam will earn more than $50k. The way I look at that figure is that there are now about 20,000+ games being released on Steam per year, and so that means that each year some 2,800 games will go on to earn $50k+ - that's more than 7 games a day, every single day. I'm a pretty big gamer, but don't think I could list 2,800 games in total across all systems and my entire life - yet that is how many new games go on to earn $50k+ on Steam every single year.
For most things, there aren't 50k reachable, aware, motivated, solvent buyers. Making enough people aware of your existence may be more expensive than the total profits made from your endeavour.
I am only pointing this out because I know people who would hear the first part of your comment and get their egos attached to an idea since they interpret it as 'there are billions of people, so I only need a tiny percentage, there is no bad idea, only bad execution' and lose years of their lives pursuing something where odds are stacked against them, if there were any odds in the first place. I'd urge people instead to also hear the 2nd part of your comment, and take it as 'experiment with many niche things, there are some that land and land well'.
I wouldn't agree there. One important thing that works in your favor is that there's an inverse relationship between marketing costs and market size, and a big part of that is because of a similar inverse relationship with word of mouth. Even the most fringe topic you could think of is generally going to have communities built up around it, and the more niche - the more 'airtime' ideas that cater to that community are going to get. Like in this case - his 'marketing' was a Reddit post and a Discord, for a total marketing spend of $0.
By contrast when appealing to a large market, marketing becomes a major part of breaking through simply because word of mouth is much more difficult to get going when you're vying for a market that a million other competitors, many quite competent themselves, are also vying for. To go with the games example again, if you're trying to create a platformer - you're probably going to fail, even if you create a pretty good game. It's just a completely oversaturated market, even if that market is massive. By contrast if you're making e.g. a Starflight clone - you're probably going to succeed if it's even remotely decent. It's very niche, but consequently also very underserved market with tremendous word of mouth potential.
I sit here typing on my DIY split keyboard powered by 2 nice!nanos. It's worth noting that a typical wireless split keyboard (separate left and right halves) uses 2 nice!nanos, so really it's only 25k people interested. That's assuming each person only builds one split keyboard, which... All the keyboard fans I know start with one, but don't stop there :p
Then we should take your word over mine. My assumption was that those A/B tests will lead to products that do increase the numbers they were measuring (retention, conversion,...) at the expense of enshittified UX (up to the point of things feeling objectively broken, like notification badges re-appearing for the same items, settings that reset after user changes, search results missing,...). At least that was my explanation for how products by major tech giants like LinkedIn, Facebook, Outlook,... could end up being shipped with such flaws. What would you say?
That's highly misleading to outright misinformation.
> Passkeys don't have to be remembered
Because you need an app for the login flow. You also don't have to remember passwords if you use a password manager app.
> don't need 2FA
Not true, a second factor in the form of eg a biometric ID or PIN is mandatory.
Phishing resistance exists, but only truly so if you completely surrender control over your device and access to your credentials. Something that the same organizations who you'll depend on for Passkeys are actively pushing for through various initiatives.
No it is not. You’re free to save passkeys in your manager of choice and it still won’t let you use a passkey on the wrong website. Users are freed from having to copy&paste TOTPs. No app other than a browser needed.
Sure, we can do research to bring improvements to open weights models, but it's the same thing: it's either open source or it won't benefit the general public nearly as much.
reply