> So instead, Warden’s company is releasing a new sensor that can scan a QR code. The idea behind this $6 sensor is that appliance makers can put it inside their products as a method of getting devices onto Wi-Fi easier. A user could simply show their Wi-Fi QR code (I find mine in my router app) to the sensor and get their, say, fridge or washer online. I think it could be neat as a way to transfer a recipe to an oven, or specific washing instructions to a washing machine for particular items of clothing. Unfortunately, unlike scanning a new shirt and getting the machine to change its parameters to provide the best wash, many of the use cases for TinyML are going to be kind of boring.
Sorry. What’s the ML piece here? QR codes are from the 90s and don’t use any ML I’m aware of…
> However, at the conference Warden told me that, while he’d quickly discovered that the model worked, educating people about new gestures was tough. “No one knows that these gestures are available,” he said. This makes sense. If you remember back to the launch of the first iPhone and its touchscreen, the first ads and demonstrations focused on things like taps and pinch-to-zoom. Those weren’t intuitive; they were taught.
And if that were true, that’s solvable just like with the iPhone by having tutorials when you boot your TV. I think what’s actually the case that the CEO doesn’t want to admit is that he’s having trouble convincing TV makers this is a useful model when they’re all going into voice-operated UIs. That and the BOM cost makes it unappealing.
> Elsewhere at the event, HP showed off two TinyML implementations with ST Micro that are embedded in new laptops. The first TinyML model uses a gyroscope to detect if a laptop has been placed in a bag or taken out of a bag. The idea behind the implementation is that the laptop will start booting up when it’s taken out of a bag in preparation for its owner to use it. If the model detects the laptop has been placed in a bag, it will change heating and cooling parameters to make sure the laptop doesn’t overheat.
And when the model gets it wrong (which it invariably does) you’ve got a laptop that failed to be in sleep in a bag (maybe you fallback to more primitive models that are foolproof like increased temp + lid closed = in bag). But seriously. If you go to sleep on lid close, putting it in a bag doesn’t really change your thermal envelope (should have happened on lid closed). And booting before your lid opens seems silly when Apple shows that it can be done near instantaneously. In other words, this seems like a PM developed feature for promo instead of good engineering being done.
> The second use case also helps with thermal management. In that use case, the laptop detects when it is on a hard or soft surface. If it’s on a soft surface, like a bed or a person’s lap, it will try to run cooler so as to avoid overheating.
Ok. Maybe this is interesting. But do you actually need ML or is it enough to define a thermal budget and recognize a solid surface can probably dissipate heat more quickly and anything beyond that doesn’t buy you all that much.
How dare you? AI pioneered that kind of empty promises decades before blockchain was even a thing!
But, seriously, this is just a "look at me!" scream to get customers or investors. People do that with every single thing (doesn't even need to be an actual thing), and it's no fault of the thing at all. It's not a matter of being there yet or anything, it's just dishonest people.
Yes, with these kind of issues you need to keep it very simple.
I have an old Chromebook that runs Linux. I did have the problem that it would sometimes wake up in my backpack and run very hot. Eventually I found out that the plastic lid was soft enough that it would bend and let the screen touch the touchpad which would wake up the Chromebook. It was possible to disable this behavior but it was not simple enought.
It's not so much about "booting" or "waking" befor opening the lid. It's more about improving what Apple calls "power nap": to get on wifi and download updated mail/chat/info before opening the lid.
Yeah, these are not good use cases for ML. They're tractable with traditional sensors and approaches, and they're horribly annoying when they get it wrong, but even more horribly annoying when they get it wrong inconsistently.
> I think it could be neat as a way to transfer a recipe to an oven, or specific washing instructions to a washing machine for particular items of clothing.
...is there also a neat way to remotely transfer the recipe's ingredients to an oven or load clothes into a washing machine?
One of my buddies once joked about the smart homes that they allow you to unlock your front door while being anywhere in the world — but sadly, there is almost never a useful reason to unlock your house's front door from 2000 km away.
What about when you’re at work and want to let the repair person in? Or you run an Airbnb and want to let in your guests without having to worry about keys and lockboxes. Or you want to let in a family member. I think there are valid reasons for that.
Yeah, this reminds me of the other article on the frontpage, about Google not understanding why anyone needs 5m files in Drive. Well, maybe you don't.
Maybe it's not something that affects you, or 80% of people, 80% of the time... that doesn't make it useless. Try to step outside your own life even a few steps...
Sure. In such a scenario I could have cameras in place if I’m super paranoid. In my experience repair people have been extremely professional and have a reputation to uphold, so if I can’t be at home and a repair needs to happen, I don’t see a problem if that convenience is important to me.
Different areas of the world have different trust levels. In northern Europe nobody would think twice about letting in the repairman, we trust each other. It is a great place to live.
If you ever have serious work done on your house, it requires letting contracts in for weeks. If you're not home, you need to leave them a key outside where dozens of people likely have access to it. I always change my locks afterwards, but a digital solution would be pretty nice.
Perhaps I misunderstood the problem you were talking about. I thought it was leaving a key unprotected. A lockbox resolves that, by restricting access to the key to people who know the lockbox combination.
You mentioned rekeying the lock after the work, so I assumed that key copying wasn't the issue as you found a solution for that.
Rekeying the lock is the main issue. It costs money and then you need to redistribute keys to anyone who has one (other family members, nanny, cleaner, etc.)
Given that this stuff was built into the original and revised kinect software on the xbox (360 and one) and no-one really knew or cared. I believe that the initial setup stuff on the xbox one covered the voice commands / gesture commands. I have a feeling they were soon forgotten. Unfortunate given how powerful the kinect, especially version 2, is. When it comes down to it, gesture control is often fiddly and obscure (there is no button on the screen to push to remind you the function exists, you just have to know 'if i wave at it, it will probably turn on'). Both of those downsides turn it into a non starter for most users.
Probably you're washing more than one shirt at a time, so the optimal washing parameters for the whole load is going to be some not-quite-average of the the data on all of the input QR codes, combined with whatever the machine sensors learn. Seems pretty ML to me, though the writing could use work.
At which point the user can pick the optimal parameters anyway (and likely do a reasonable enough job) without fancy ML. Fancy ML would be things like “take out items xyz” to wash just whites.
That being said, it’s possible the ML piece is about reading multiple QR codes at once in different angles. I could see that requiring some ML. But still. It feels like a solution in search of a problem since they seem to be taking the “throw spaghetti at the wall and see what sticks” approach to building products.
Yeah, about 15 years ago when there were companies planning to replace barcodes with RFID tags on all products, a big selling point was that washing machines could detect red socks in a white wash.
I can't be bothered to go through the tags and think about settings. If the laundry destroys it I'll just stop buying things like that going forward. Probably there are people as lazy as me, but with a better eye for fashion. Maybe they would care?
> Sorry. What’s the ML piece here? QR codes are from the 90s and don’t use any ML I’m aware of…
Presumably they are using object detection to recognize where the QR codes are located in the image? Which they then feed to the standard QR decoding algorithms.
> That being said, it’s possible the ML piece is about reading multiple QR codes at once in different angles. I could see that requiring some ML. But still. It feels like a solution in search of a problem since they seem to be taking the “throw spaghetti at the wall and see what sticks” approach to building products.
What I’m saying is that washing clothes doesn’t seem to benefit from any QR codes. And the cooking example is even more confusing because presumably you’d have one for the recipe, not per ingredient
I can see how it could be useful to have such an input method in an embedded system for some configuration options like WiFi credentials.
> What I’m saying is that washing clothes doesn’t seem to benefit from any QR codes. And the cooking example is even more confusing because presumably you’d have one for the recipe, not per ingredient
Well, we are in the age of IoT. Every appliance now wants to be connected to the Internet.
I'm very into TinyML - not because of IoT but because small and fast models work well in (clientside) webpages, in realtime systems and pretty much everywhere. All while consuming less resources, which should be celebrated. Also, it's more democratic - anyone can compete in this field, supercomputer not needed.
IMO, what is needed is a larger focus on resource consumption in competitions, benchmarks and rankings. For example, I like that you can rank some lists on paperswithcode by number of parameters [1].
That’s weird. Being able to use AI without being beholden to Big Tech seems like a huge win to me. Especially when I want to ensure that the AI is making best effort for my local community’s best interest, and not the best interest of a large corp.
That’s a huge burden on the natural resources (electricity).
Beyond local control, I gave other examples of embedded AI in other comments — or more specifically, what you can do with an embedded LLM. Namely, being able to have a better human interface for complex settings, and being able to reprogram protocols (or anything that is “software-defined”) for future-proofing.
Integrating an accelerator into a dishwasher is also expensive. How many inferences are you going to get out of it before the hardware is outdated? Once a day for 5 years may not be enough to justify the cost. Compare that to having a single GPU in a server in your house that services all your IoT devices, which can be readily upgraded and requires no special software. The is no benefit to doing inference on device unless you are in the business of selling said devices.
A big part of TinyML is finding alternate paths for implementing AI in resource-constrained space.
We already have SoC that is functionally not so much different from modern microcontrollers. Depending on economics of scale, it isn’t that big of a leap of imagination to see a microcontroller which includes 4-bit vector ops like a GPU.
If by "private cloud", we mean carving out something from a public cloud, I'm still not sure I'd want something as important and private as AI to be in a cloud facility.
Even if you put together a private cloud in a data center, it's still going to use up a lot more electrical resources compared to say, an iphone-sized usage, much less in a low-powered, embedded application.
Also, there's a tendency for our civilization, when we make efficiency gains with breakthrough technologies, to then expand our usage. We don't do a great job of actually reducing overall energy expenditure.
I'm sure there are lots of fun things to do with an Arduino that could recognize gestures; I'm very surprised nobody's "excited" by this?
For example, I made a photobooth simple device based on a Pi and a regular DSLR; the person takes a photo by pressing a button, then the Pi checks the camera and sends the latest image in a web gallery somewhere. The problem is the button: it needs a remote. If wired, it risks destroying the whole apparatus if someone pulls on the cord; if wireless, it risks being lost.
A gesture-based trigger would be super cool; but having it run on the Pi through the DSLR risks damaging the camera, so it should run on a different device, cheap and not too power hungry, such as an Arduino.
What about some kind of MIDI-controller based on an Arduino that could recognize gestures?
I also made a webapp to learn sight-reading (babeloop.com) but it requires users clicking or tapping on the screen, which is not natural.
It would be cool to be able to listen to users reading notes out loud and detect if they're right or wrong, on the fly, locally in a browser or on a phone. A light general speech recognition model such as VOSK is 40Mb and is able to recognize most phonemes; but for music sight reading, there are only 7 syllabes, so one should be able to make a much smaller model? I don't know how hard it would be though to train my own model...?
Embedded platforms generally have very little memory and very few hardware threads.
Tiny SoCs will likely need accelerators to run more interesting models. For example, ARM is working on a ML coprocessor to pair with their Cortex-M chips.
> I'm sure there are lots of fun things to do with an Arduino that could recognize gestures
Arduinos can recognize gestures, faces, etc. right now with well-established tech, no servers needed. I've had a couple of robots running around my place that do this sort of thing for a number of years now.
I've been learning how transformers work, and also reading about RNNs.
It seems that transformers have succeeded because they can be parallelized, and solve many of the problems that RNNs have. This has lead to the advancements we've seen recently with GTP. Big companies are able to throw their data centers at the problem and train impressive and massive models.
Part of me hopes that a non-parallelizable AI architecture might be discovered which performs even better. Perhaps the problems with RNNs could be solved in a way that doesn't parallelize? We would be so fortunate if a desktop computer could run an AI that's half as good as Microsoft's or Amazon's best AI. I would love to see the advantage of the data center removed.
Philosophically, this does make some sense. The wisdom behind sayings such as "adding more people makes the project even later" exist because the greatest intellects we're aware of (ourselves) do not parallelize well.
ML is still in a compute hungry growth phase. Every exciting new achievement in accuracy requires more compute than before. It makes sense that tinyML, which has less compute, would always have lagging modeling support and be a bit less interesting.
TinyML does result in much cheaper products, though.
After the initial obsession with Jupiter-scale models, we quickly discovered that very small models can perform even better if you reorganize your problem.
Last week, we were trying to fine tune a 175b parameter model to take a natural language prompt with hopes of directly-outputting correct, domain-specific SQL. Now that reality has passed, we are looking at different paths.
As of this week, we are trying to hit everything with the binary classification hammer. Turns out you can train a model to output 1 of 2 possible tokens with exponentially fewer parameters, training items, machine hours, etc. The statistics available in binary classification are also incredibly powerful and the results are trivial to reason with.
Even if you need thousands of binary classifiers, their scale and granularity makes this a non-event or potentially an advantage.
The real integration magic with AI/ML is starting to look like a weird form of set theory. At this level of complexity, detecting (and potentially confirming) the user's intention is way more important than trying to draw a direct map from input to destination.
The ability to have models that can run on resource-constrained devices does feel like a strong direction for ML to go in and could lead to greater user privacy. However, I’m unconvinced by the IoT-aspect of this tech. In many ways, it feels like IoT has “failed” to be as popular with consumers as expected and feels overhyped. Will adding ML to IoT devices really make a difference?
Adding an embedded LLM as a human interface for every appliance is a huge win— for consumers at least.
For example, I have a dishwasher with a bunch of settings, can sense load, etc. It’s got a touch interface that works with wet hands. Or I can tell it to start with the usual settings, or that a particular load is a bit different. Same with the laundry, the pressure cooker.
It is less mind bandwidth when you got kids.
What I don’t want, is for my appliances to do is to phone home to the makers.
LLMs (if you don’t somehow trigger its insanity) can be far more capable than Siri. How do you get that into something more energy efficient than a high end gaming rig?
Something more hidden is using LLMs to reprogram machine-to-machine protocols. That might extend the lifetime of machines that have to talk with other machines, but it breaks planned obsolescence.
There are plenty of exciting product ideas. Whether they are exciting revenue generators are another thing entirely.
No, there’s an underlying logic and underlying settings that are still accessible.
But transparently wrapped around that there’s a “good Clippy” who can teach, interpret, and orchestrate those settings with a CUI (conversational UI, pronounced “koo-ee”).
Oh, a settings assistant is much easier to get right.
It is just completely against the modernly accepted "best practices" for devices and interface development. So I don't see how we can get it. But yeah, it could be good.
Skeptical you should be, but I’m optimistic. We have papers showing that knowledge in these models can be edited and deleted. Sam Altman makes the point that too much compute is being spent on using the LLM as a database.
Thinking about how few things any of these CUIs need to know about, I’m optimistic that we can distill them down to a workable size while maintaining the LLM magic.
“Fridge, what is the meaning of life?”
‘Sorry, I don’t know about that. Ask me something about what’s in your fridge.’
“Okay how many eggs do I have.”
“I see 3 eggs.”
When I can have that conversation by proxy through my phone’s onboard CUI while at the store, I’m going to get a lot of value out of that.
If it works like ChatGPT does, then I would find it a greater mental burden. You'd have to carefully craft what you're telling it, or engage in a conversation of some sort, instead of just hitting a couple of buttons or turning a dial.
ML has been on IoT devices for years. Heck, there are embedded arm SOCs with built in CNN coprocessors that will run your tensorflow models as-is. Again, they've been shipping in volume IoT products for years.
If ML is a win for an IoT device, the hardware's been there for a while, though I'm sure yet-cheaper hardware might unlock a few more applications, it doesn't feel like much of a game changer.
IoT sensors powered by ML may not provide good use cases for consumers, mostly because all things they can do can be done by a large model in the cloud, plus a phone. It will get interesting when we ask what use cases can't be solved by phone+cloud combo.
Such things as air quality management are good use cases. You can't use your phone to do that.
It seems like many of the TinyML use cases are hidden or somewhat dull, making it difficult to get people excited about it. Do you think there are ways to make TinyML more exciting and appealing to a wider audience? How can we better showcase the potential of TinyML and its ability to bring more privacy to IoT and give everyday products superpowers?
How about just being able to run a home assistant with AI support to reprogram the home assistant, without having to spend a lot more electricity? They don’t have to have superpowers to be useful.
The clear use-case for it is mechanical control and feedback. But it seems that every robot has enough tiny problems that you can justify a larger CPU anyway. I too am having a hard time being excited about it.
Why do you say that? There are microcontrollers announced with neural network accelerators integrated, and low power FPGAs with accelerators on the market, along with many chip startups aiming to provide milliwatt scale dedicated neural chips.
From my work in AI & robotics, its generally finnicky and a bit crap even when a ton of work goes into it with a lot more compute (jetson nano or higher). I feel like trying to optimise those kinds of models is a bit niche/premature
As a hobbyist that has been my perception of the situation as well, though I wasn’t sure if I was just missing something.
The potential is immense and exciting, but try as I might, I can’t seem to make even fairly simple things work reliably as I want them to.
One thing I love about embedded projects is that you can achieve pretty incredible reliability because everything can be so dialed in and isolated from points of failure in, say, an operating system. Trying to use ML for simple tasks felt like it eliminated that and introduced seemingly arbitrary failure into projects I really needed to work perfectly.
Again, just a hobbyist, so I can’t make any broad statements. It seems to align with what you’re saying though. Once we get past this hump and have more powerful/effective models at a reasonable price point, I feel like it could be transformative. At the moment it’s still extremely interesting and fun to experiment with.
Sorry. What’s the ML piece here? QR codes are from the 90s and don’t use any ML I’m aware of…
> However, at the conference Warden told me that, while he’d quickly discovered that the model worked, educating people about new gestures was tough. “No one knows that these gestures are available,” he said. This makes sense. If you remember back to the launch of the first iPhone and its touchscreen, the first ads and demonstrations focused on things like taps and pinch-to-zoom. Those weren’t intuitive; they were taught.
And if that were true, that’s solvable just like with the iPhone by having tutorials when you boot your TV. I think what’s actually the case that the CEO doesn’t want to admit is that he’s having trouble convincing TV makers this is a useful model when they’re all going into voice-operated UIs. That and the BOM cost makes it unappealing.