Some people like to parrot "next token prediction", "LLMs can only interpolate", and other nonsense, but it is obviously not true for many reasons, in particular since we introduced RL.
Humans do not have the monopoly on generating novel ideas, modern AI models using post training, RL etc can come to them in the same way we do, exploration.
See also verifier's law [0]: "The ease of training AI to solve a task is proportional to how verifiable the task is. All tasks that are possible to solve and easy to verify will be solved by AI."
This applied to chess, go, strategy games, and we can now see it applying to mathematics, algorithmic problems, etc.
It is incredibly humbling to see AI outperform humans at creative cognitive tasks, and realise that the bitter lesson [1] applies so generally, but here we are.
I genuinely start to think that we, as humanity, severely overestimate our cognitive abilities. We act so surprised “just a few years of LLM with a few RL tweaks match our PhD levels! It must be hidden inside our knowledge base!”. Em, what if no? What if our “PhD level” is just very low level comparing to upper boundaries of measurable intelligence? What if we need to learn being humble and stop treating our minds as “sacred source of creativity and intelligence”?
RL or no RL, AI cannot escape the distribution it's trained on. It's just that the labs will put so much into the distribution that we won't be able to tell the difference that easily, nor will it matter for most tasks. The reason AI does well on ARC-AGI-2 is because the labs created synthetic training data using similar puzzles.
Yes it can! That's the whole point of RL! it generates slightly out of distribution rollouts, and rewards good rollouts to change the distribution of the output
That's not out of distributíon, that's inside the distribution of the rollout. If you don't create rollouts for the game of Chess then it doesn't know how to play Chess no matter how smart it is at tasks you've created rollouts for. It's structurally stuck in its distribution.
What if it doesn't need to escape the distribution, it can just exhaust the current distribution we have much more broadly and efficiently than humans can?
So the answers we're seeking to our bleeding edge questions are already there, we just need an AI's ability to target the answers. Then re-train on the improvements and go from there.
Reinforcement learning for "reasoning" perturbs the model to generate completions in a particular chain of thought / alternative selection structure. It's three next token predictors in a trench coat.
When these things start solving many more long standing problems, and start introducing more novel problems, will people finally admit that the "next token predictor" is not the gotcha they think it is?
It's not a gotcha. It's incredible what these things can do despite being next token predictors from a weird dataset. That's at the heart of the "bitter lesson", and you don't have to believe in magic to see it.
They'll likely make it available at some point, but for now one can use OpenEvolve [0] which is not quite as good but should be a good start to use the same LLM-driven evolutionary framework.
> Application pods learn the current assignment through a library called the Slicelet (S for server side). The Slicelet maintains a local cache of the latest assignment by fetching it from the Dicer service and watching for updates. When it receives an updated assignment, the Slicelet notifies the application via a listener API.
For a critical control plane component like this, I tend to prefer a constant work pattern [0], to avoid metastable failures [1], e.g. periodically pull the data instead of relying on notifications.
They should be reintroducing the 3D vcache [0] variants (X) in EPYC, with a higher cache/core ratio, that was present in EPYC4 (e.g. 9684X [1]) they for some reason wasn't available in EPYC5.
Makes a massive difference at high density and utilisation, with the standard cache/core performance can really degrade under load.
Current AI, and in particular RL-based, is already or will soon achieve super human performance on problems that can be - quickly - verified and measured.
So maths, algorithms, etc and well defined bugs fall into that category.
However architectural decision, design, long-term planning where there is little data, no model allowing synthetic data generation, and long iteration cycles are not so much amenable to it.
I had tinnitus for over 10 years.
My tinnitus was not the usual ringing type, it was some sort of humming, low frequency noise. The frequency was not constant, it could vary.
It could sometimes stop for 5-10 minutes, e.g. after a hot bath.
Went to see many specialists, tried everything, to no avail.
One day I started experiencing recurrent tension and light pain in my neck and shoulder blades, so I started doing some neck and shoulder blades stretches several times a day.
After a few weeks, the pain was gone, and I realised the tinnitus had stopped. This was maybe 2 years ago (I am still doing those exercises multiple times a day).
> Is hardware agnostic and uses TCP/IP to communicate.
So no RDMA?
It's very hard to make effective use of modern NVMe drives bandwidth over TCP/IP.
> A logical shard is further split into five physical instances, one leader and four followers, in a typical distributed consensus setup. The distributed consensus engine is provided by a purpose-built Raft-like implementation, which we call LogsDB
Raft-like, so not Raft, a custom algorithm?
Implementing distributed consensus correctly from scratch is very hard - why not use some battle-tested implementations?
> Read/write access to the block service is provided using a simple TCP API currently implemented by a Go process. This process is hardware agnostic and uses the Go standard library to read and write blocks to a conventional local file system. We originally planned to rewrite the Go process in C++, and possibly write to block devices directly, but the idiomatic Go implementation has proven performant enough for our needs so far.
The document mentions it's designed to reach TB/s though.
Which means that for an IO intensive workload, one would end up wasting a lot of drive bandwidth, and require a huge number of nodes.
Modern parallel filesystems can reach 80-90GB/s per node, using RDMA, DPDK etc.
> This is in contrast to protocols like NFS, whereby each connection is very stateful, holding resources such as open files, locks, and so on.
This is not true for NFSv3 and older, it tends to be stateless (no notion of open file).
No mention of the way this was developed and tested - does it use some formal methods, simulator, chaos engineering etc?
We can saturate the network interfaces of our flash boxes with our very simple Go block server, because it uses sendfile under the hood. It would be easy to switch to RDMA (it’s just a transport layer change) but right now we didn’t need to. We’ve had to make some difficult prioritisation decisions here.
PRs welcome!
> Implementing distributed consensus correctly from scratch is very hard - why not use some battle-tested implementations?
We’re used to building things like this, trading systems are giant distributed systems with shared state operating at millions of updates per second. We also cheated, right now there is no automatic failover enabled. Failures are rare and we will only enable that post-Jepsen.
If we used somebody else’s implementation we would never be able to do the multi-master stuff that we need to equalise latency for non-primary regions.
> This is not true for NFSv3 and older, it tends to be stateless (no notion of open file).
Even NFSv3 needs a duplicate request cache because requests are not idempotent. Idempotency of all requests is hard to achieve but rewarding.
Not to mention you simply want a large distributed system implemented in multiple clouds / on prems / use cases, with battle tested procedures on node failure, replacement, expansion, contraction, backup/restore, repair/verification, install guides, an error "zoo".
Not to mention a Jepsen test suite, detailed CAP tradeoff explanation, etc.
There's a reason those big DFS at the FAANGs aren't really implemented anywhere else: they NEED the original authors with a big, deeply experienced infrastructure/team in house.
The protocols you mentioned are always consistent. You will know if they are not consistent because they will not make progress. Yes there's a short delay where some nodes haven't learned about the new thing yet and only know that they're about to learn the new thing, but that's not what's meant by "eventual consistency", which is when inconsistent things may happen and become consistent at some time later. In Paxos or Raft, nodes that know the new consistent data is about to arrive can wait for it and present the illusion of a completely consistent system (as long as the network isn't partitioned so the data eventually arrives). These protocols are slow. so they're usually only used for the most important coordination, like knowing which servers are online.
CAP cannot be worked around. In the event of a partition, your system is either C or A, no buts. Either the losing side of the partition refuses to process writes and usually reads as well (ensuring consistency and ensuring unavailability) or it does not refuse (ensuring availability and ensuring data corruption). There are no third options.
Well, some people say the third option is to just make sure the network is 100% reliable and a partition never occurs. That's laughable.
Yes, so when I see a distributed system that does not tell me explicitly whether or not it is sacrificing consistency or availability, I get suspicious.
Or has mechanisms for tuning on a request basis what you want to prioritize: consistency or availability, and those depend on specific mechanisms for reads and writes.
If I don't see a distributed system that explains such things, then I'm assuming that they made a lot of bad assumptions.
> Yes there's a short delay where some nodes haven't learned about the new thing yet, but that's not what's meant by "eventual consistency", which is when inconsistent things may happen and become consistent at some time later.
Thanks, I haven't looked at these problems in a while.
> In the event of a partition, your system is either C or A, no buts.
Fair enough. Raft and Paxos provide well-understood tradeoffs but not a workaround.
Out of curiosity, you seem knowledgeable here, is it possible to do NVME over RDMA in public cloud (e.g., on AWS)? I was recently looking into this and my conclusion was no, but I'd love to be wrong :)
Thanks for the link! I had seen this, but it wasn't clear to me either how to configure the host as an nvme-of target, nor whether it would actually bypass the host CPU. The article (admittedly now 4 years old) cites single digit GB/second, while I was really hoping for something closer to the full NVME bandwidth. Maybe that's just a reflection of the time though, drives have gotten a lot faster since then.
Edit: this is more like what I was hoping for: https://aws.amazon.com/blogs/aws/amazon-fsx-for-lustre-unloc... although I wasn't looking for a file system product. Ideally a tutorial like... "Create a couple VMs, store a file on one, do XYZ, and then read it from another with this API" was what I was hoping for, or at least some first party documentation of how to use these things together.
So... The issue is that I'm not using lustre. As far as I can tell, NVME over fabrics (nvme-of) for RDMA is implemented by kernel modules nvmet-rdma and nvme-rdma (the first being for the target). This kernel modules supports infibiband and I think fiber channels, but _not_ EFA, and EFA itself is not an implementation of infiniband. There are user space libraries that paper over these differences when using them for just network transport (E.g., libfabrics) and EFA sorta pretends to be IB, but afaict this is just meant to ease integration at the user space level. Unfortunately, since there's no kernel module support for EFA in the nvme-of kernel modules, it doesn't seem possible to use without lustre. I don't know exactly how they're doing it for lustre clients. There seems to be a lustre client kernel module though, so my guess is that it's in there? The lustre networking module, lnet, does have an EFA integration, but it seems to only be as a network transit. I don't see anything in lustre about nvme-of though, so I'm not sure.
Maybe there's something I'm missing though, and it'll just work if I give it try :)
Yeah, Lustre supports EFA as a network transit between a Lustre client and a Lustre server. It's lnet/klnds/kefalnd/ in the Lustre tree. But Lustre doesn't support NVMeoF directly. It uses a custom protocol. And neither does EFA. Someone would have to modify the NVMeoF RDMA target/host drivers to support it. EFA already supports in-kernel IB clients (that's how Lustre uses EFA today). So it's not an impossible task. It's just that no one has done it.
Hey, thanks for the comment! Also, I'm amused by the specificity of your account haha, do you have something set to monitor HN for mentions of Lustre?
> "But Lustre doesn't support NVMeoF directly. It uses a custom protocol."
Could you link me to this? I searched the lustre repo for nvme and didn't see anything that looked promising, but would be curious to read how this works.
> "And neither does EFA. Someone would have to modify the NVMeoF RDMA target/host drivers to support it."
> "EFA already supports in-kernel IB clients (that's how Lustre uses EFA today). So it's not an impossible task. It's just that no one has done it."
I think you're saying there's already in-kernel code for interfacing with EFA, because this is how lnet uses EFA? Is that https://kernel.googlesource.com/pub/scm/linux/kernel/git/tyc...? I found this but I wasn't sure if this was actually the dataplane (for lack of a better word) part of things, from what I read it sounded like most of the dataplane was implemented in userspace as a part of libfabric, but it sounds like I might be wrong.
Does this mean you can generally just pretend that EFA is a normal IB interface and have things work out? If that's the case, why doesn't NVME-of just support it naturally? Just trying to figure out how these things fit together, I appreciate your time!
In case you're curious, I have a stateful service that has an NVME backed cache over object storage and I've been wondering what it would take to make it so that we could run some proxy services that can directly read from that cache to scale out the read throughput from an instance.
> do you have something set to monitor HN for mentions of Lustre?
Nothing, beside browsing hackernews a bit too much.
> "But Lustre doesn't support NVMeoF directly. It uses a custom protocol."
To be specific, Lustre is a parallel filesystem. Think of it like a bigger version of NFS. You format the NVMe as ext4 or ZFS and mount them as Lustre servers. Once you have an MGS, MDS, and OSS - you can mount the servers as a filesystem. Lustre won't export the NVMe to client as a block device. But you could mount individual Lustre files as a block device, if you want.
> Does this mean you can generally just pretend that EFA is a normal IB interface and have things work out? If that's the case, why doesn't NVME-of just support it naturally? Just trying to figure out how these things fit together, I appreciate your time!
There are some similarities (the EFA driver is implemented in the IB subsystem, after all). But the semantics for adding/removing ports/interfaces would be different - so it wouldn't "just work" without some changes. I don't know the full scope of the changes (I haven't dived into it too much). Although, I suspect support would look fairly similar to drivers/nvme/target/rdma.c.
> In case you're curious, I have a stateful service that has an NVME backed cache over object storage and I've been wondering what it would take to make it so that we could run some proxy services that can directly read from that cache to scale out the read throughput from an instance.
Interesting that neither the article nor the comments mention the CALM theorem [0], which gives a framework to explain when coordination-free consistency is possible, and is arguably the big idea behind SEC.
Some people like to parrot "next token prediction", "LLMs can only interpolate", and other nonsense, but it is obviously not true for many reasons, in particular since we introduced RL.
Humans do not have the monopoly on generating novel ideas, modern AI models using post training, RL etc can come to them in the same way we do, exploration.
See also verifier's law [0]: "The ease of training AI to solve a task is proportional to how verifiable the task is. All tasks that are possible to solve and easy to verify will be solved by AI."
This applied to chess, go, strategy games, and we can now see it applying to mathematics, algorithmic problems, etc.
It is incredibly humbling to see AI outperform humans at creative cognitive tasks, and realise that the bitter lesson [1] applies so generally, but here we are.
[0] https://www.jasonwei.net/blog/asymmetry-of-verification-and-...
[1] http://www.incompleteideas.net/IncIdeas/BitterLesson.html