As a Norwegian this sounds like a mistake. Who will use this LLM? Where? For what? The underlying data could be made more easily searchable and digestible for agents in general if the goal is better knowledge of Norwegian culture.
That said, they are quite limited in what they are allowed to share of in-copyright works, and nb.no is a fantastic resource as it is (though you'll need a Norwegian IP address for too much of it - it's one of th main reasons I maintain a VPN) - if they are allowed to make it accessible there, it'd be great.
But they also have vast amounts of out-of-copyright data that I hope they'd make more easily accessible...
Why would the gap grow? There is no more training data to acquire, frontier model are training on the entire internet. Everything from now on is just fine-tuning.
Your statement assumes training data is the only thing that matters for the big players, while not considering it limiting for the small Norwegian model. That’s a fallacy.
Exactly, if there's one thing transformers are good at it's translation. One I've found particularly nice: any question ChatGPT can answer in English it can answer in French. I'm assuming Norwegian too. So there's no point.
The point is that norway willl have its own LLM. And will not have dependencies to another state or private company. The goal is not to be the best model. But to have a model that include more Norwegian data then other LLM and that it's not screwed against other sources.
But what does that give you? If the model is far less capable? What will it do for you with that Norwegian data, that a better model could not do with better search or context?
A model has many dimensions. You can't have them on one scale from good to bad. The model will most likely be poor at coding. But will give better answers about Norwegian cultur. I assume the tone of voice will be (by default) much closer to how Norwegians talk and write then what we current see from model from the US. They seem to be a bit to much.. Norwegian people are a bit more down to earth
Yes transformers are great at translation as that is their purpose.
LLMs are not great at preserving cultural uniqueness and diversity. Take how “delve” has reentered the lexicon because the human assessors for pre training dialect of English uses “delve” a lot.
There is a lot of benefits to training specifically for a unique culture with unique norms to preserve the culture as we increasingly rely on LLMs.
There is a lot more to it than literal translations. Even if an american can talk Norwegian it doesn't mean that they get the cultural context right.
"Oh yeah after you drive 2 hours home from work your wife and kid will greet you with some delicious pie" doesnt work so well even if its in Norwegian.
Both Claude and ChatGPT can translate into minor dialects of Norwegian they will have seen very few works in because very few printed works exist in them.
E.g. I've tested both my local spoken dialect, which is rarely written, and a sociolect used by a 1970's Maoist group consiting of a few hundred people, where most of the printed material consists of novels from a couple of ex-members that became authors.
In the latter case, it claimed to not know, but was able to get a good match from just a description.
I also just had it ape Norwegian orthography from the 1910's by having it look up the rules and translate a text it had first translated from English to modern Norwegian, and it did just fine.
They will have seem some work in these dialects, but mostly it transfer really well to know related languages (English, Dutch, German, Swedish, Danish, roughly form a continuum from least in common to most in common with modern Norwegian; they all share vocabulary and significant parts of grammar with Norwegian), and then a relatively limited exposure to Norwegian itself is sufficient to do fairly well.
They're also really good at "style transfer" of text in the form of tweaking orthography, word order, and minor grammar changes from descriptions and examples.
(incidentally, the latter is one way of getting an LLM to sound a lot less like an LLM)
This is all true, but I assumed the original posters were talking about cultural knowledge, not linguistic correspondences.
To do translation well you still need cultural knowledge. (E.g. the particular modes of specific kinds of legalese, or slang and the nuances of social class, etc)
I think it's not that this knowledge isn't present in the model somewhere, but probably more that it gets killed by instruction tuning for US corporate values.
Some people have mentioned that this is a U.S incorporated company (Delaware). Recommend reading Moneyland by Oliver Bullough if you want to know more about the U.S role as the new shell company haven.
What are the alternatives for aspiring tinkerers now?
My wife (cybernetics engineer) and I are buying a 3D printer and planned getting an Arduino as an entry point. What should we do instead? What are the best communities and resources?
* RP2040 / RP2350 - If you don't need connectivity, this is a great chip for flexible IO. Good software support, easy to use, well documented.
* ESP32 - Good community support, bluetooth and wifi connectivity, some powerful variants as well for driving screens and other things.
* STM32 - Widely used, and an absolute boatload of chip variants for different tasks, from small little GPIO twiddling cores, to beefy chips running DSPs and outputting high-res images to displays.
* nRF52840 (and other variants) - Good for bluetooth devices, should be lower power than ESP32.
My recommendation would be to buy something like a Xiao RP2040:
They're cheap, have USB-C, and are super easy to use. Oh, and they have a reset button which for some reason, the official pico board does not. On top of that, the official pico board uses micro USB, so overall I would recommend NOT buying them, they're annoying to work with. The Xiao boards don't have a ton of IO pins, but they're at least good for learning and if you determine you need more IO you can move to a different dev board, or design your own PCB.
I first got into Raspberry Pi Picos, but I've also been experimenting with Esp32's and some of the nRF chips. I mostly do CircuitPython on them but Arduino is a supported platform on those I believe.
I got a couple of RP2040 boards recently and I'm amazed at how easy it is to just get stuff done. Between the native usb support and the circuit python support it's been a breeze. I just got a couple of boards up and running uart in a daisy chain. It was intimidating, but the circuitpython docs made it relatively simple.
ESP32 - quite a range of dev boards and places like Seeed and Adafruit have a nice selection of accessories. Adafruit develops CircuitPython which is IMO the lowest barrier to entry for programming MCUs. Adafruit even has CircuitPython sketches on their site for how to interface with the components they sell.
Rust on ESP32 is still a bit early - the HAL crate is still pretty unstable, but the toolchain is quite nice and I'm able to be productive enough that I never reach for C or C++.
You are on the right track i.e. stay with an Arduino in the beginning. Note that "Arduino" is a family of boards with different MCUs but all providing a common API (mostly). So you choose the Board/MCU combination best suited for your system and can always move to something else later after you have gotten some experience. The reason is the Arduino Ecosystem. There are thousands of free tutorials, designs, libraries etc. all available for you to try out for your app and more often than not you can have your PoC/MVP by just plugging in some libraries and writing some glue code. You only have to learn the Arduino API and not any specific MCU's datasheets unless and until you are doing something more lower level. It is all way easier.
The Arduino Cloud offering (runs on AWS) makes integrating your Arduino-based system into an end-to-end SaaS app simple (just watch and follow some tutorials on Youtube). There is also the Arduino PRO series of hardware for you if and when you want industrial-grade hardware for demanding systems/environments.
If the Qualcomm c-suites have half a brain amongst themselves they will not kill the goose that lays the golden eggs.
The feather series of boards from Adafruit + Curcuit/Micropython works really well if you just want to make stuff happen instead of tuning a toolchain and, like, setting up clocks with asm.
Everyone I know who is into tinkering with microcontrollers moved onto ESP32 a long time ago now. I actually thought this headline was going to link to an article about ESP32's popularity. VSCode with the PlatformIO extension has been great for me when working with them:
I'd like to use apps out there for model railroading - locomotive control and accessory automation, especially 3-rail. There is a LOT written for Arduino; I wonder if any other platforms come close. Someone mentioned some sort of Arduino emulation layer on top of ESP32.
ESP32 or RP2 based boards with for example MicroPython/CircuitPython, or platform.io + VSCode. Though the good old Arduino IDE seems to be unaffected by this change though.
That might be precisely why OpenAI is pushing an over investment in infrastructure. When VCs are no longer willing to substitute compute, having more compute available than natural demand will drive the prices down.
Agree! We saw this a lot. Launching with the Quest 3, we were often the first company to do X, Y, Z despite being months after new features had been released in the SDKs because they were poorly documented (and often even conflicting).
Diverging even slightly from the demo use case would quickly feel like Sisyphus; so close, but never succeeding in getting over the hill.
Good for marketing in certain cases (to be the first), but bad for the community of builders
Cloudflare sits in the middle of a vast amount of web traffic now, offering easy global payments and skimming off the top of that is going to be very profitable potentially.
I don't trust Cloudflare, the larger they get the bigger the abuse potential becomes.
CF: No criminal convictions I know about. No reason to distrust yet. Most controversies seem to be about providing services to political organisations.
GCP: Earth Engine is quite good, but Google have multiple criminal convictions. As a repeat offender they should be avoided at all costs. They are just so exceptionally good at manipulating people, markets and academia it's genuinely terrifying.
Azure: Microsoft still don't take security seriously. They're just a bit bumbly, not really smart enough to be as terrifying as Google.
AWS: Pretty useful, annoying to use, distrust because I can't bear Amazon's use of dark patterns in consumer products.
Their customers pay them to be the front door to their websites. Customers want a way to reduce the massive traffic from AI crawling, to block malicious traffic, and to be compensated for access.
That leaves Cloudflare well positioned to implement a pay-for-access check along with all of the existing bot services they offer. AI crawling goes from a threat to a win if I can serve OpenAI a demand to pay for each request. Bot abuse goes down if traffic isn’t free. Businesses like journalism become stable again if readers pay for content rather than relying on advertisers to subsidize it.
But then why do a blockchain thing? Why not just make CloudflareBucks they centrally control? Surely that is much easier to monetize and cheaper to implement.
... which will then be immediately destroyed by law because it gives the actual tax man a single target, along with a money flow that comes from within the control of the tax man.
PLUS just imagine how many corrupt politicians will be tempted to force these payments to go through their company.
Their customers are hit the hardest by the shift away from google search to AI. They probably are the right company to try to help them monetize their content.
What does the traffic from Google Search look like though?
I can't imagine all the low effort content farms that were providing things like dictionary definitions or ridiculously elongated ad-stuffed versions of kitchen recipes are doing too hot under the pressure from AI Overviews. And they can't be the only ones impacted.
I agree this makes little sense for Cloudflare to jump on the crypto bandwagon now. Maybe they want to retain some talent by turning this into an official project.
Is the premise that it makes more sense for an AI agent to pay in prepurchased stablecoin tokens instead of direct access to a credit card?
They are the moat between AI content crawlers and websites. They will probably start charging a fee and a stablecoin is a good way to do that globally.
