MiMo 2.6 Pro is at the Pareto frontier (the one where you only need 20% of the smarts for 80% of the tasks) according to Artificial Analysis, nicely filling in as a substitute for a hypothetical 'GPT-6 Terra' (which doesn't exist as far as we know). That's pretty darn impressive from an open model.
This version of Opus "max" apparently has even higher thinking output than Qwen "max", which is infamous for its thinking streams where it constantly second-guesses itself, then third-guesses, fourth-guesses and generally nth-guesses itself for arbitrarily large n. Of course, we aren't actually seeing Claude's raw thinking output: all we get is the after-the-fact prettified "summary". One wonders how much of that is a coincidence, or whether there's a reason behind that.
You can definitely offload n-gram embeddings to storage; they're very sparsely used (only a few KB fetched per token) so this is quite effective. Loading to DRAM only becomes necessary if they are a bottleneck to overall performance (which might happen if you're doing very wide batches and everything else uses super fast VRAM/HBM).
I was looking at the qwen-next-flash, and the weights would fill my OEM Spark on their own, before the n-gram. I'm unclear if offloading to disk can work here, is that what you are implying is possible?!
I'd recommend pointing your agent at it (after installing sparkrun), and asking it to research the absolute latest in TP=1 Flash-Next - mine grabbed particular vLLM nightlies and mods to improve performance, and it was well worth it.
I have a quirky vLLM on k8s on 2x OEM sparks setup with 9 models available to me. I'm not keen to run nightly vLLM, too many issues with it in the past. Going the qwen-next path means displacing things I use daily :/
I have a watchful eye on the diffusion ~ Jev/Kev PR
For what it's worth, Flash Next outperforms every other model that is available to us on the GB10 in all of my testing; though if you have two sparks then the TP=2 version is even better and easier (I don't think you'll need the nightly for that at all, just use the recipe)
You could always stream from SSD storage. Especially effective if you get a cheap old-gen HEDT with lots of PCIe slots to add NVMe storage to and reasonable overall PCIe bandwidth.
That nearly certainly boots you to secs-per-tok land (as opposed to tok/s). Plausible if you are willing to wait hours to days for responses for simple testing, but not (debatably) "usable".
Demonstrated inference speed for the biggest open-weight models is indeed around 1 token per second, with weights read from SSDs.
However, this is for relatively low-end systems, with a couple of fast SSDs providing around 20 GB/s throughput (or with a few more, but connected through relatively slow Thunderbolt, for a similar total throughput).
If you use 16-lane PCIe add-on cards with 4 M.2 slots for SSDs and a total throughput of 50 to 60 GB/s, you can quadruple the previous speed in a desktop PC where you use the GPU PCIe slot for SSDs (a fast CPU, e.g. an AMD 9950X, would be alone fast enough for inference limited by SSD throughput, so a discrete GPU would not be required).
If you have a server/workstation motherboard, e.g. with 6 16-lane PCIe slots, you might gain another factor of 4 in the inference speed, so one might reach around 15 tokens per second for a very big model, but the cost would also be high, with so many SSDs, even if at that number of SSDs each SSD could be the smallest that can be found with a PCIe 5.0 interface.
Astra-Ultra? Even the largest open model to date (Kimi K3) is nowhere close to Astra level, and it will be quite slow even on the highest-spec M5 Ultra, with achievable speeds of about 0.5 tok/s at most due to having to stream weights from SSD (~13 GB/s on the highest storage capacity M5 Max machines so far). This is OK for doing simple Q&A in the background but it's far from a genuine coding experience. You'd have to test batching of multiple thinking streams in order to try and raise overall tok/s via layer-wise reuse of the streamed weights (and this is where the "Ultra" part sort of becomes relevant; Kimi series models have good support for agent swarms) but this would decrease single-session performance even further. It would only be usable for background jobs, though the hardware would then have a chance of paying for itself if it was fully used on a 24/7 basis.
That's all well and good but Qwen 27B is a small, dense model; that's favorable to both batching and MTP. Batching of large, sparse/MoE models like Kimi K3 (requiring slow SSD streaming even on a single maxed out Mac Studio) on local hardware is an entirely different game that's mostly theoretical so far: many people would even call it outright pointless. (MTP clearly fares even worse, though - unlike batching, it ends up wasting scarce weights-fetching throughput on wrongly predicted tokens.)
Parent commenter hinted at that. Yet DeepSeek has released their V4 which was hugely successful, and even their new architecture is marked V4.1. Qwen internals mark their Flash-Next model, also very compelling, as "qwen4exp". So both of them are bucking the negative stereotype.
> possibly done via the Manim library or something like it
It can be done already: the point is that the motivation and explanation parts are terrible, especially for novel topics where the AI can't just rip off existing content. A Lean proof is at least a verifiable task; you end up with an actual proof that you can work through. A Manim slop video doesn't have that.
> People often hate math because it was not explained to them correctly
Spoiler: this is also why mathematicians hate vibe-math. AIs are outright terrible explainers even when they do have a watertight logical argument—and honestly, this is the load-bearing seam.
It goes beyond "proof vs. exposition": the logical derivations AI comes up with fail to even qualify as human-directed proof because of how terrible they are (far below even the most novice mathematician doing their roughest work) at the exposition part.
> AIs are outright terrible explainers even when they do have a watertight logical argument
I think this only applies to cutting edge mathematics (novel proofs of hard problems). I have seen it reported more than once that such AI proofs are cumbersome to follow.
But in my experience, when it comes to explaining well-established math that is already in the training data, AIs can be very good teachers (at least with recent models). Especially if you use it along with a textbook and ask it about anything that might not be explained well in the textbook.
OP has reportedly been in contact with Prof. Mantova, who actually worked (jointly with S. L'Innocente) on the key human-authored results behind this AI proof and is arguably in the best position to understand exactly what the AI added that wasn't known before. (See the OP's thread on the Lean Zulip.) So this is happening, and we might see an actual paper publication of this result down the line (possibly encompassing multiple roughly self-contained papers, building up to the final result). The current AI-written version is way too obscure for that, and the AI-written human-targeted "summaries" are not really helpful. Again the OP is quite aware of this.
Vincenzo (Mantova) here: yes, I have been reading bits and pieces of the proof and I can say for sure that the method is sound, at least for the first half (power series with real exponents). I haven't even tried reading the part that mentions the Cantor-Bendixson rank yet, although given how the rest went, I'd be really surprised if there's a problem there.
As with most interesting proofs, the number of core ideas is actually small, I'd say two for the real exponents, and presumably a third idea for lifting up to omnific integers. I have been redoing the real exponents part of the proof going on the ideas only, and with a few smarter choices, I am converging on something very short. And I mean very short, which is amazing. I didn't think the answer would be this close: it 'just' needs looking at the problem from the right angle, and also make a fairly bold guess at the outcome.
Dan's current proof is of course much longer. Between the fossilized ideas that Dan mentions in the post and the formalisation of previous results, there's a lot of cruft that inflates the proof but does not really help understanding what is going on. Luckily the word 'derivation' pops up early, otherwise it would have been very challenging to wade through the lemmas to find the important points.
I appreciate the sentiment, especially after the escalation of the last few weeks. But I am hoping this story can become an example of why mathematicians are still very much needed and LLMs are still severely lacking when it comes to displacing scientists. Conway's conjecture has no application whatsoever, even within pure maths, and the only point in pursuing it is what you can learn in the process, and whether you can make something beautiful. LLMs are consistently unable to do the latter (and yes, some mathematicians are also not very good at it... this is an old topic of discussion, just amplified by current events). To me it feels like the technology is still where it was in 2015 when DeepDream images came out: increasingly good at pattern matching, but still a lot more like dreaming than thinking. So our job is still there somewhere.
The problem is rather how quickly we can change our ways of working to make sure the training and hiring pipeline does not collapse. That's the disastrous scenario, for both the individuals affected and the discipline, that we must avert somehow. I wish we had an easy answer to that. I certainly don't. But I like to think that at least engaging with the public in a constructive way will have a net positive effect.
The read was very interesting, and I think you're right, mathematicians are still very much needed, in some sense now more than ever.
But the effect is the same across other professions. We need mathematicians, we need professional grounding but we only "need" experts.
Every step AI takes feels like it slices people off the bottom of a profession, making them less relevant, while simultaneously increasing demand for the top.
the vicious cycle I see is it will "Eat the middle", you need people with juuust enough acumen in a subject to drive the AI, to set a goal and nudge to a direction, and you need a real expert to look at the results with a critical eye but you don't need the middle of the curve. I think over time this will depress hiring and wages of new people, and stunt their progress towards expertise by destroying the middle ground. Then when our current experts retire we'll be left in a lurch.
I’m not actually upset with your actions I was just worked up at the time. You’re in a dilemma and graciously handling it the way you are is the only sensible path.
Needless to say, I disagree that what Prof. Mantova is planning to do (digesting the proof and making it human-understandable) represents the "end of [mathematicians'] career". Systematizing has always been a key part of human mathematical work, and tidying up a raw proof can be viewed as a kind of systematizing.
My hope is also that Mantova and very possibly L'Innocente will get a substantial share of credit for their role in the resolution of this conjecture by Conway: the AI would not have embarked on this were it not for their prior work. So even human mathematicians with an inclination for more exploratory "problem solving" will have plenty to do in the future. (The story is actually not that different for the recent Navier-Stokes forced blowup result, which also built on key conceptual work from 2023 by Córdoba and Martinez-Zoroa.)
Luckily for these guys the problem is famous enough that giving some kind of credit for its resolution even makes sense at all. The vast majority of published work is not like this. There’s not going to be any credit divvied up to the thousands of people who’s work was probably involved in the recent formalization of FLT, which involved formalizing 300 thousand theorems.
Kevin Buzzard has reportedly been working on a more systematized (i.e. leveraging a more modern approach) and human-targeted formalization of FLT. I certainly hope that his work continues and he gets the deserved credit. Of course there is also some possibility that it won't, and that would mean you have a point after all.
You’re just mentioning established and famous people. What about the average people who are just going to get fucked for spending their lives trying to promote humanity, just to be rewarded with the risk of economic ruin and no job prospects?
If Mantova and L'Innocente (as well as Córdoba and Martinez-Zoroa) count as "famous people" now, I would say that their fame is quite deserved! Would you disagree that AI played a significant part in surfacing their work? Aside from that, I'm not really sure what to make of your comment. I do sympathize for the researchers who are at risk of having their ongoing work "scooped" by AI (and for all we know, this may even include Mantova and L'Innocente) but the way to address that is to still have "divying up" of relevant human credit.
I'm not working on some famous conjecture, just following my curiosity, and I think I found something (minor and specific) that should reignite a young mathematician's interest for work she did almost a decade ago, for a phd she never used (she left academia). I'm going to have flowers delivered to her, with a brief note explaining what I think I found and encouragement to resume her academic career and reach out to a her former co-authors.
Keep doing things that people want to pay for. If this becomes impossible, the world will face a much broader political crisis than "mathematics careers". How concerned were you when manufacturing industries died in certain regions and jobs got wiped out? Or it only matters when it's academics?
It shows that the issue is much broader and myopically focusing on how math PhD students will be evaluated etc. entirely misses the forest for the trees, it's just not even close in proportionality.
This is indeed also coming for law, medicine, and banking, though licensed professions will hold out for longer because you need someone to put in jail when things go wrong. The problem is that all this is extremely over politicized and nobody is able to think clearly. They want to simultaneously say all this is just hype and a bubble and will go away like NFTs did, and also are starting to worry about economic replacement. Some more coherent political narrative will have to be formed.
I’ll state this again. How does my economic redundancy get mitigated in anyway by being the first on the chopping block by several years? Do you think being jobless for 3 years is just missing the forest for the trees?
Since it will be coming for everyone, the solution will be major social upheaval with some consequence for all of humanity, hopefully a good one where we somehow manage to keep on living some kind of good life.
Regarding being jobless for the 3 intervening years, it is certainly a personal concern but in this temporary phase there are still some other jobs for smart people. Once there aren't any, we are entering the part that I was talking about where you will be far from alone and you can join together to exert some kind of political pressure but it will not be about math PhDs, but employment as a whole. And it may not be very effective if AI is on the other side, not on yours. Yeah, it sounds like scifi, and people want to dismiss scifi concerns and instead focus just one inch ahead of their toes, instead of seeing the writing on the wall.
For some reason you think I’m opposed to the use of AI in mathematics. This is not the case. If this guy sat down to actually learn the math and then did actual work to disseminate the knowledge in a positive sustainable way it would be a different story. His actions are just here to inspire more people to use this tool in counterproductive ways.
You have no idea how mathematics has progressed for thousands of years. How about having some humility and understanding the culture you’re cheering on the death of? It’s a goddamn credit based system.
Do you buy stuff made in factories of do you open your wallet for everything handmade craft products? Or it's fine as long as craftsmen's jobs got replaced, just not when yours? Do you ever use a self-checkout? An ATM? Or do you pay extra to get cash from a human cashier at the bank?
Do you actually think this is a good argument? I had literally zero say in what happened in the past. I did not support the Midwest getting gutted. I wasn’t even able to vote when that was happening? Would it shock you that I generally think capital is misaligned to humanity?
> Would it shock you that I generally think capital is misaligned to humanity?
You still prefer to get cheaper options yourself. I know this argument gets caricatured in the "yet you participate in society" meme, but the point is that this is the aggregate result of individual humans making decisions on where to allocate their resources. You can attack this using various ideological and religious frameworks, but if it's just some stoner college freshman's communism, I'm not interested (neither if it's the more potent version that dispossessed my ancestors in Eastern Europe).
Also is your position really that nobody can point to problems or criticize anything without a fully fleshed political system to replace the USA with? I don’t support the excesses of the system of the USSR or 1950s-1980s China.
You can criticize things of course, like you can shake your fist at a cloud too when you'd rather want a sunny day, but the results may vary. You're asking people to refrain from using a cost-saving technology and instead spend more money on things that can be obtained easier with the tech. This means going against one's own rational interest. Such things usually happen willfully when the person believes in a religion like the Amish or Hasidic Jews. So having a fully fleshed political/ideological system on as similar level will be necessary here.
I don’t “prefer” to get cheaper options in any meaningful sense. There is no choice of mine involved in what products corporations serve. I have to buy the cheapest option because I have a poverty wage and have to survive.
The meaningful sense is when you open your wallet and buy one option when another option is there. Of course you have a budget and choose rationally, which is kinda like a forced choice. But you could always go and buy from the mom and pop shop and you could buy handmade clothes. Well, you don't have the money for it. Neither did people have it generations ago, they just wore worse clothes, went barefeet to school etc. You could still do that, walk barefoot until you can afford a handmade shoe and keep that for decades. It is in fact your preference not to do this, even if you want to claim that social pressure predetermines that you can't go around barefoot etc. At some point we disprove free will and agency entirely. Nobody in any historical era had more choice to express their preferences than consumers today.
I am not allowed to walk into my office without shoes. My shoes have had holes for 90% of my life. I assure you I own fewer things than you are imagining.
Nope I’m airing them right here. This guy didn’t do any work except tell the bot to continue for a month. That’s not work, that’s just unhealthy and negative. He’s made no intellectual progress. He doesn’t even know any mathematics and has never cared to learn. He’s just lucky other people are there and kind enough to let him dump his slop onto. They could have done this and gotten a lot more out of it.
You can definitely argue that the user's direction and curation work was intellectually trivial (though there's meaningful room for disagreement even there, especially wrt. having the AI stick to established terminology/broad approaches - this is arguably a sort of successful "systematizing" work, though only in a very minimal sense) but this was not a one-shotted result. The blog post is extremely clear about that.
And of course, going by their own admission, they couldn't "have done this themselves": the most you can argue wrt. this is that Mantova and L'Innocente, or some other narrow domain experts, might have done this themselves and that AI "scooped" this result from them.
Nobody said it was one-shotted? It was mindlessly “continue-shotted” except for the brilliant idea of upgrading the model. Obviously the models are going to be improved to the point where typing continue continue, how ya feeling, continue continue, okay let’s double check this, upgrade model, continue continue, will be less necessary.
The direction of continue shotting will develop a new culture that lead to a much more wider understanding for humans in the field of math. Find a way to accept this new culture and you'll thrive.
I apologize to GPerson for needlessly trying to convince him to my point of view. He seems to be an smart person deeply affected by how LLM are affecting his vocation and work. It was wrong of me to do this and I will take a break from this website for one week.
I was not "mindlessly continue-shotting". The way you describe it is literally the same as "one-shotting" and, as I explained in the article, it simply doesn't work. You are welcome to try it yourself on this problem to verify that.
Yes, I was not doing any mathematical work in curating the output, but the article makes it quite clear that pivots and constraints the LLM would not impose on itself were critical to actually making progress.
Also:
>He doesn’t even know any mathematics and has never cared to learn
While I don't know enough mathematics to work on this problem, claiming something like this is preposterous. As I link in the first paragraph of the article, I've been learning mathematics on my own by going through Terence Tao's Analysis book and solving exercises. I'm familiar with the concepts of mathematical definitions, proofs, etc. I've gotten about halfway through the book solving them on paper before abandoning it (and later got through the first few chapters in Lean, also solving every exercise — by hand, mind you). Sure, this doesn't make me a mathematician, but I'm closer to a dropout first-year student than to someone who has "never cared to learn".
You’re a guy who benefited from easy choices which led to you a life of luxury and now you use your high perch to shit on people who were stupid enough to work hard to do something more meaningful with their lives than attain wealth and social status.
Edit I apologize to Dan Abramov for venting my frustrations about things outside of either of our control and unfairly using him as a punching bag. He seems to be an intelligent person and I hope he continues learning mathematics using whatever tools he sees fit, including AI. It was wrong of me to do this and I will take a break from this website for one week. I won’t hide this comment though it is shameful.
I read your blog and nothing in it indicates you did anything besides mindless continue shotting. You don’t know this because you don’t know anything about the culture you’re stomping on.
Edit I apologize to Dan Abramov for venting my frustrations about things outside of either of our control and unfairly using him as a punching bag. He seems to be an intelligent person and I hope he continues learning mathematics using whatever tools he sees fit, including AI. It was wrong of me to do this and I will take a break from this website for one week.
A distinction without a difference. I can’t believe you’re actually trying to take some kind of credit for this. That’s just so shameless.
Edit I apologize to Dan Abramov for venting my frustrations about things outside of either of our control and unfairly using him as a punching bag. He seems to be an intelligent person and I hope he continues learning mathematics using whatever tools he sees fit, including AI. It was wrong of me to do this and I will take a break from this website for one week.
What does "taking credit" even mean here? I am not saying that I contributed to the mathematical work in the proof. I am simply disagreeing with you that the process is fair to describe as plain continue-shotting. You saying "without a difference" does not actually make your claim true. There clearly is a difference between a specific technique getting to the result and that technique not getting to the result. Whether or not you like the technique, and whether or not you consider a result obtained through that technique of any value. I thought there is some value in sharing both the technique and the result, and that's why I published a post about it. I think it's slightly different from "an AI company threw 50,000 agents on it" and it's also slightly different from "I just said Claude to work hard and it got me a solution", and that makes it worth sharing with other people.
I am not saying that my work constitutes a mathematical contribution on its own. Not any more than stumbling upon an anonymous manuscript with the solution would constitute a mathematical contribution. I do, however, think that it can lead to a mathematical contribution if any mathematicians consider it worthwhile to do something with it. Whether or not they consider it worthwhile is not up to me.
There is no value in your technique when 2 months, or 6 months, or 2 years from now the model will improve. There is no skill in suggesting to double check work. Everything you did could be trivially automated with today’s models anyway, as you are aware. Taking credit is trying to pretend you had some meaningful role here.
It is a distinction without a difference because I want to live in a world where people get to fill their lives with meaningful things, and are not forced into Uber delivery driving jobs just because rich people like you think it’s fun to put their name next to something other people made prestigious.
Edit I apologize to Dan Abramov for venting my frustrations about things outside of either of our control and unfairly using him as a punching bag. He seems to be an intelligent person and I hope he continues learning mathematics using whatever tools he sees fit, including AI. It was wrong of me to do this and I will take a break from this website for one week.
I'm not sure it is trivially automatable with today's models since they get "carried away" too much, and I'm not sure there's a good way to prevent the supervisor from drifting. But I would like somebody to attempt that, which is part of the reason for my posting. If someone can automate whatever I was doing, it should be possible to get to deeper results than current models allow. Or maybe the next models would just be that good, I don't know.
Re: "rich", I've essentially spent $400 on this (in subsidized subscriptions), plus my free time being a mindless drone. Given that you assume my role is automatable, it sounds like this is relatively accessible to anyone with $400 (as long as AI companies continue subsidizing the frontier models). I don't think I've had some kind of an unfair advantage beyond that. If anything, a proper mathematician would probably be able to derive the result much faster with the same tools.
I don't know how the broad availability of these tools (to mathematicians and non-mathematicians alike) will change the field, what is considered prestigious, what work gets funding, how it affects the pipeline, etc. You seem to be implying that even testing the limits of these tools, or at least publishing the results obtained with them, is unethical in itself, even though it is broadly accessible now. I can understand this point of view.
> A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
This is what AI becoming self-aware looks like. /s Anyway, didn't OpenAI report the same thing with the model writing out weird musings about itself during compaction?
reply