I've been working on a voxel game engine called Bonsai for ~10 years.
Probably the most interesting thing about it at the moment is the editor. The world, and most things in it, are represented as collections of SDFs. More accurately, they're density fields, but, potato-tomato.
Bonsai has undergone a large rewrite over the last couple years that's nearing completion. A world edit is defined as a bunch of SDF parameters which get projected/rasterized into the voxel grid by a shader on the GPU. One neat thing about SDFs is they've been thoroughly researched and documented by a guy named Inigo Quilez, and they have a lot of nice mathematical properties. For example, you can do a smooth union of arbitrary SDFs to get nice rounded contours where shapes join.
I've written every system from scratch, all the way from the memory allocators and font rasterizer to the collision detector and simulation loop. I even wrote a metaprogramming language as a replacement for C++ templates, which is a whole other story. IIRC the only external dependency is the C runtime library for starting the process and my very occasional use of variadic functions arguments.
For a long time, an explicit non-goal of the project was to ship a game. It sounded insane to me to write an entire 3D engine and then ship a game. As it turns out, I've gotten it to the point where I can actually make a game. I've got a start on the game systems in a closed-source repo, and hope to have a steam page for it by the end of the year.
I'll do some shameless self-promotion and leave some links here for anyone interested in looking at the engine, language code, or some pretty pictures.
Definitely interested in where you take things with the actual game - the engine looks really cool. It seems like we're in the midst of a bit of a cambrian explosion around "survival games descended from Minecraft" (Hytale is top of mind, but there's also Vintage Story and iirc there's a few others of note either in early access or development). Not that a voxel engine can only do a Minecraft-like - but there's definitely a market for someone who wants to take that formula and run with it.
Yeah, it seems enough people have done enough work on voxel engines now that we're actually starting to see games proliferate. I'm stoked about it!
I'm a lot less interested in minecraft-style survival games than I am in exploring .. other .. types of games that push the medium further. Most voxel games that make it to release basically boil down to 'hack-n-slash RPG with a lego world'. As you mentioned, it's obviously a successful recipe, but I feel like it's barely scratching the surface in terms of how far the underlying tech has come.
I've got a game in-flight where the primary mechanic is modifying the environment to achieve a goal. Teardown is really the only other game that's made it to market and done this. The mechanics are going to be intentionally simplistic .. it's more of an existence proof that a deeper game could exist and people are interested in it.
minecraft offers a good interface. i am experimenting with blocklandscapes as control plane for other applications (currently photo editing). Its fun :D (not as useful as i hoped lol)
What would you say its defining features are? What makes it different than Minecraft? Seems like its probably more performant.
Whats the license situation. I see your own license which sounds maximally permissive but written by you. Is that just how it works? I can do whatever I want with that license? It doesnt need to be a “real” license?
Maybe your bonsai logo should come from your voxel engine instead of pixel art?
Idk why but my first thought was somehow that i misunderstood “voxel engine” when i saw the pixel art.
I'd say the defining feature is the editor and terrain generation system. At the moment it's not well documented, but once the big rewrite I've been working on has settled I plan to take some time to make tutorial materials.
As far as the license goes, I chose WTFPL originally because I figured it would dissuade 'real' companies (with legal teams) from using it. I've since thought about changing the license to something permissive that might play nice with legal teams, like MIT. You can do whatever the fuck you want; it says so right in the license ;) I'd actually love it if people used it. My intent now is to keep the engine open source, and build games with it in closed-source repos.
You're right, I should change the logo. It's on the todo list .. it's just a really long list ;)
SDFs are the future at least when it comes to level design - especially because they have that "subtract" railgun cone of death functionality baked in. A boss fight feels different, if where the hammer comes down there is a crater afterwards.
If you are trying to rapidly iterate a level layout, nothing comes close to the power of a hierarchy of CSG brushes. You can change one parameter and have the work of an army of artists executed in milliseconds.
These are SDFs? - The problem is though to do level of detail with these functions. Like you have to add interpolation rates to the various subfunctions - and distancevalues to them.
omg you're the poof guy!! i really like trying to shove metaprogramming systems into C and poof is one of the best i've ever encountered!! it's so damn good!
good luck with the game btw, is there anywhere i can follow development?
Wow, high praise! Thanks :D I'm slightly biased, but I tend to agree with you; it's actually the best metaprogramming language I've ever used. JAI is probably better, but I haven't sat down and done a real project with it yet.
As far as following development on the game .. I'll probably start doing marketing when I've got enough for a trailer. I'll announce updates on Discord and Youtube, if you'd like to follow along there
I actually convert the SDFs to voxels and render triangles, but.. some people have written direct SDF renderers. There's a guy that posted to YouTube not too long ago that wrote a really cool one with physics and everything
Interests : systems programming, compilers, 2D & 3D graphics, performance
--------
Hello, my name is Jesse.
I describe myself as a competent generalist, and a lifelong learner. I'm frequently working on something I've never done before. I've worked at every level of the stack, from cycle-shaving hot loops to the frontend of large web applications. My most recent professional experience has been about a year at a semiconductor startup; I wrote prototypes for an ISA, compiler, a cycle simulator and visual debugger.
In terms of hard technical skills, I can quickly become productive in nearly any language, and at any level of the stack. I'm comfortable working in heavily multithreaded/async soft-realtime environments where performance is a key acceptance criteria. I have a good understanding of modern hardware architectures, including GPUs, from main memory to registers and instruction pipelines. I have working knowledge of interpreters, dynamic language runtimes and garbage collectors. I'm convinced I can learn nearly anything (albeit some things more quickly than others), given an appropriate problem domain to apply it to.
In terms of business value, I can take hand-wavy visions of new products and turn them into working prototype(s), quickly. I'm comfortable refining those raw materials and delivering real value to customers, internal or external. I have a track record of successfully identifying 80/20 solutions and love the feeling of making tools that make peoples lives better.
In my personal life, I enjoy travelling, surfing, climbing, backcountry skiing, snowmobiling, pirates, and attending raves. I like the phrase "have strong opinions, weakly held".
If I was born to do one thing in this world, it's program computers.
"It takes five years to become an overnight success" I heard a variation of this from a VC who invested in my startup in the 8Os. I had to use it on him when we went back to the well a few times for more funding.
I don't understand how you can claim that using a GC does not make a language slower and less predictable.
Running a GC takes time, pollutes the cache, and is often run at an unpredictable time. Sure, the GC is not necessarily the SLOWEST thing about the language (python), but it's not helping, either.
Yes, but for a moving collector that's less time than it takes to run malloc and free. The interaction of a moving collector with most object is bump allocation when they're allocated (similar to stack allocation) and... that's it. The GC never sees them again, scans them again, or is even aware of their existence (moving collectors don't have a free operation). Overall, moving collectors (but not other kinds of GC) reduce the work of memory management compared to malloc/free.
In low-level languages we try to avoid doing a lot of malloc/free not because heap memory management is slow in general, but because that approach to memory management is slow. Moving collectors are an optimisation designed to make heap memory management fast, but it requires that (nearly) all pointers be movable, something that low-level languages can't do because they have constraints that are more important to them than speed (you can't interact with the OS or hardware directly, i.e. without an FFI API, if your pointers are movable, and such direct interaction is the point of low-level languages).
That moving collectors (NOT the GC Python has; NOT the GC Go has) can, in principle, make heap memory management cheaper than stack allocation has been well known since the eighties. But until recently they had excellent throughput (somewhat similar to arenas) but potentially long pauses. It was only recently that they were made "pauseless".
> and is often run at an unpredictable time
How much work malloc and free need to do is also unpredictable, and a modern pauseless moving collector like ZGC spreads the work needed for memory management more evenly than malloc and free.
> Sure, the GC is not necessarily the SLOWEST thing about the language (python), but it's not helping, either.
There is very little resemblance between CPython's GC and Java. Python's memory management is closer to C's than to Java's. GCs cover such a wide spectrum of algorithms that it doesn't make sense to talk about them as a single category as far as performance tradeoffs are concerned.
I've written two separate moving collectors for dynamic language runtimes, as well as done significant work in realtime 3D graphics, and what you're saying mostly smells like bullshit.
> The interaction of a moving collector with most object is bump allocation when they're allocated (similar to stack allocation) and... that's it. The GC never sees them again, scans them again, or is even aware of their existence
I mean .. sure, but, a copying GC eats somewhere on the order of 10% of your total memory bandwidth just copying shit around. I can guarantee that if you use sane allocation strategies (arenas & freelists, pools, whatever) you spend <1% of your total system resources fucking around with memory allocation.
> Moving collectors [...] requires that (nearly) all pointers be movable, something that low-level languages can't do
Completely false. You have to do some manual bookkeeping in C++, Rust, Zig, whatever, but you can do it, and in fact many commercial GCs do (V8 is a good example).
...
The rest of what you said is just empty-sounding claims that I'm not going to address. I looked at the single paper that you linked in another comment, from the 80s, which is hardly relevant on modern hardware.
Please, if you're going to make the claims you're making, back them up with hard evidence. I've looked, and the overwhelming majority of papers out there claim that GCs are slow, memory hungry and, generally, a waste of time.
> I've written two separate moving collectors for dynamic language runtimes, as well as done significant work in realtime 3D graphics
That's nice. I have ~25 years of experience with large C++ software, including hard and soft realtime systems, and I now work on the JVM.
> I mean .. sure, but, a copying GC eats somewhere on the order of 10% of your total memory bandwidth just copying shit around
Good moving collectors are designed to copy very little. That's the entire purpose of generations.
> I can guarantee that if you use sane allocation strategies (arenas & freelists, pools, whatever) you spend <1% of your total system resources fucking around with memory allocation.
In theory. When you're in charge of a >2 MLOC C++ system, maintained by a large team for well over a decade, you find that these optimisations are very costly. Plus, you commonly find large C++ software that needs sophisticated malloc/free allocators for acceptable performance (BTW, some of those allocators are almost the same size, in LOC, as that of ZGC, probably the world's most sophisticated moving collector).
That's why, where I used to work and oversee large projects, we migrated pretty much all systems (mostly soft-realtime defence software) to Java from C++ - for better performance. One of the goals of the JVM was to tackle the familiar performance issues that plague large C++ programs.
And BTW, using arenas isn't so easy when programs get large and sprawling, or even in general. Zig definitely makes that much easier, though.
> Completely false. You have to do some manual bookkeeping in C++, Rust, Zig, whatever, but you can do it, and in fact many commercial GCs do (V8 is a good example).
What I said is completely true, but you may have misunderstood it. You can, of course, combine moving collectors with things that expect stable ones in the same process. In fact, you absolutely must, because at some point in the stack you need to talk to the OS and/or hardware, and they expect stable pointers. But you have to have some distinct FFI layer between the two, and the entire point of low-level languages is to be at the lower level.
> Please, if you're going to make the claims you're making, back them up with hard evidence. I've looked, and the overwhelming majority of papers out there claim that GCs are slow, memory hungry and, generally, a waste of time.
I don't know what you've read, but that is very clearly not the consensus among memory management experts. My "claims" are pretty common industry knowledge, and why the majority of performance-critical large software has migrated away from low-level languages over the past couple of decades, and the trend continues. I'm not trying to change anything, I'm just explaining why the industry is doing what it's doing to those who may not be familiar with large, long-lived software.
It's funny, but 25 years ago, the people who doubted the amazing performance-per-effort of moving collectors and JITs were those who (like me) had not used those technologies and were mostly familiar with low-level languages. These days, it's the people who have little experience developing and evolving large and complex software in low-level languages (TBF, there's much less such software written in low-level languages these days) that believe the low-level languages are inherently fast.
Having said all that, when programs are relatively small and/or not very concurrent, the effort required to match or beat Java's performance in a low-level language through careful manual optimisation is sometimes worth it. In large software, it gets harder and harder.
> I can guarantee that if you use sane allocation strategies (arenas & freelists, pools, whatever) you spend <1% of your total system resources fucking around with memory allocation.
I just spoke with someone on the performance team at one of the world's largest tech companies who told me that some of their larger Rust programs spend 30% of their CPU on malloc/free. Of course, it's possible in principle to reduce this given enough effort, but this is identical to the experience we've had with C++ for decades: When programs are small, it's easy to get good performance, but as they get larger, the areas where low-level languages have intrinsic inefficiencies (such as dynamic memory management) tend to become more pronounced in practice and the programs are not so easy to optimise.
Interests : systems programming, compilers, 2D & 3D graphics, performance
--------
Hello, my name is Jesse.
I describe myself as a competent generalist, and a lifelong learner. I'm frequently working on something I've never done before. I've worked at every level of the stack, from cycle-shaving hot loops to the frontend of large web applications. My most recent professional experience has been about a year at a semiconductor startup; I wrote prototypes for an ISA, compiler, a cycle simulator and visual debugger.
In terms of hard technical skills, I can quickly become productive in nearly any language, and at any level of the stack. I'm comfortable working in heavily multithreaded/async soft-realtime environments where performance is a key acceptance criteria. I have a good understanding of modern hardware architectures, including GPUs, from main memory to registers and instruction pipelines. I have working knowledge of interpreters, dynamic language runtimes and garbage collectors. I'm convinced I can learn nearly anything (albeit some things more quickly than others), given an appropriate problem domain to apply it to.
In terms of business value, I can take hand-wavy visions of new products and turn them into working prototype(s), quickly. I'm comfortable refining those raw materials and delivering real value to customers, internal or external. I have a track record of successfully identifying 80/20 solutions and love the feeling of making tools that make peoples lives better.
In my personal life, I enjoy travelling, surfing, climbing, backcountry skiing, snowmobiling, pirates, and attending raves. I like the phrase "have strong opinions, weakly held".
If I was born to do one thing in this world, it's program computers.
Cool project! I viewed the landing page on mobile (chrome, android) and touching to drag the globe worked once or twice, but stopped working afterwards. The globe would appear to freeze (and, actually, the whole page locked up for a second or two), but then resume rotating on its own. Seems fairly reproducible on reloading the page.
I like this comment. The idea that animations should be able to be picked apart frame by frame and always be coherent doesn't make much sense, because the user will never actually do that.
I do like the point the article makes about using ui fidelity as a proxy for software quality, and agree that they pointed out some bad animations. But, I think you hit the nail on the head .. frame by frame coherence isn't the best yardstick for measuring animation "goodness".
Exactly, you should optimize for quality and have a high bar forbit but I'm not sure sacrificing framerate for pixel perfect rendering or perfect subpixel antialiasing and compositing a perfect glass chroma aberration when wheezing a superfast animation is more important than it being solid 144Hz or audio synced.
Animations are highly effective tools for conveying state information.
Consider a toolbar with a mix of enabled and disabled buttons. Hover effects (which I would consider animations) convey that something is clickable, and on-click effects confirm an action. These effects convey meaningful information to both beginner users and power users of any software, and are in no way inconvenient to either group.
I generally agree animations tend to get in the way when you want to get shit done, but the idea that animations are only applicable as artistic effects rings untrue to me.
Hover effects are a terrible way of indicating if something is clickable, because you have to mouse over them instead of just looking at them. This problem was already solved a long time ago by rendering inactive elements in gray. I'm not sure which GUI did this first, but the Apple Lisa (1983, first mass-market personal computer with a GUI) definitely did it.
Interests : systems programming, compilers, 2D & 3D graphics, performance
--------
Hello, my name is Jesse.
I describe myself as a competent generalist, and a lifelong learner. I'm frequently working on something I've never done before. I've worked at every level of the stack, from cycle-shaving hot loops to the frontend of large web applications. My most recent professional experience has been about a year at a semiconductor startup; I wrote prototypes for an ISA, compiler, a cycle simulator and visual debugger.
In terms of hard technical skills, I can quickly become productive in nearly any language, and at any level of the stack. I'm comfortable working in heavily multithreaded/async soft-realtime environments where performance is a key acceptance criteria. I have a good understanding of modern hardware architectures, including GPUs, from main memory to registers and instruction pipelines. I have working knowledge of interpreters, dynamic language runtimes and garbage collectors. I'm convinced I can learn nearly anything (albeit some things more quickly than others), given an appropriate problem domain to apply it to.
In terms of business value, I can take hand-wavy visions of new products and turn them into working prototype(s), quickly. I'm comfortable refining those raw materials and delivering real value to customers, internal or external. I have a track record of successfully identifying 80/20 solutions and love the feeling of making tools that make peoples lives better.
In my personal life, I enjoy travelling, surfing, climbing, backcountry skiing, snowmobiling, pirates, and attending raves. I like the phrase "have strong opinions, weakly held".
If I was born to do one thing in this world, it's program computers.
Probably the most interesting thing about it at the moment is the editor. The world, and most things in it, are represented as collections of SDFs. More accurately, they're density fields, but, potato-tomato.
Bonsai has undergone a large rewrite over the last couple years that's nearing completion. A world edit is defined as a bunch of SDF parameters which get projected/rasterized into the voxel grid by a shader on the GPU. One neat thing about SDFs is they've been thoroughly researched and documented by a guy named Inigo Quilez, and they have a lot of nice mathematical properties. For example, you can do a smooth union of arbitrary SDFs to get nice rounded contours where shapes join.
I've written every system from scratch, all the way from the memory allocators and font rasterizer to the collision detector and simulation loop. I even wrote a metaprogramming language as a replacement for C++ templates, which is a whole other story. IIRC the only external dependency is the C runtime library for starting the process and my very occasional use of variadic functions arguments.
For a long time, an explicit non-goal of the project was to ship a game. It sounded insane to me to write an entire 3D engine and then ship a game. As it turns out, I've gotten it to the point where I can actually make a game. I've got a start on the game systems in a closed-source repo, and hope to have a steam page for it by the end of the year.
I'll do some shameless self-promotion and leave some links here for anyone interested in looking at the engine, language code, or some pretty pictures.
https://github.com/scallyw4g/bonsai
https://github.com/scallyw4g/poof
reply