I would be interested to know if there is a method similar to this one in Rust [0] that allows a single binary to support multiple optimization levels depending on the executing CPU? It feels wasteful to not enable these optimizations but I don't really want to force a user to choose between a complex feature matrix.
I've long been surprised there isn't more multiversion stuff built right into every language compiler; I would have thought Intel would be very motivated to get more binaries lighting up the features they add to their expensive top line CPUs.
But yeah no, on the whole cost of the checks and duplicated binary size aren't seen as worth it, so instead it's piecemeal implementations mostly in numeric packages like eigen and lapack.
> so instead it's piecemeal implementations mostly in numeric packages like eigen and lapack.
Because that’s where the user-noticeable gains can be made. Using popcount in code you run once is going to shave off, maybe, 100 cycles. That isn’t worth the extra cycles of that approach.
Also, FTA: “and arguably the whole scheme should be replaced by finer-grained feature detection”. Such feature detection would lead to a combinatorial explosion of different binaries.
Finally, where it really matters, it’s not only a matter of recompiling the same code. For optimal performance, you also want to change loop unrolling strategy, stride count, etc.
Doesn't that get into the domain of a distro like Gentoo though? (Or sort of Nix as well) - rebuild everything to precisely target your actual architecture.
Yes, and the maximally performant configurations are at odds with a usefully hardened security posture. This only gets more true as time goes on and additional mitigations pile up.
POPCNT is an interesting example. Runtime dispatch (with a conditional branch) would actually make sense for it because it's comparatively difficult to implement from scratch. PDEP and PEXT might be similar (but I don't think compilers pattern-match for it, unlike POPCNT). AArch64 uses localized run-time dispatch extensively because LL/SC atomics are so very bad on current cores, but there isn't anything comparable in the x86-64 space (POPCNT isn't that frequent).
For many other things, like using a YMM register to copy a 32-byte struct or a variable shift, run-time dispatch just not make sense. You will only see a benefit if you generate this code unconditionally. For FMA, you wouldn't even get bit-identical output, leading to testing concerns.
> Also, FTA: “and arguably the whole scheme should be replaced by finer-grained feature detection”. Such feature detection would lead to a combinatorial explosion of different binaries.
I see I wasn’t clear enough. The tool I discussed generates multiple binaries and then packs all of them into a single binary. I was referring to the former.
“After building the different versions, it computes a hash of each version and it filters out the duplicates (i.e., the compilations that gave the same binaries despite having different CPU features). Finally, it builds a runner that embeds one version compressed (the source) and the others as compressed binary patches to the source. For instance, when building for the target x86_64-pc-windows-msvc, by default 4 different versions will be built, filtered, compressed, and merged into a single portable binary.
When executed, the runner uncompresses and executes the version that matches the CPU features of the host.”
Hopefully (and likely) the patches will not be too large, but for 6 binary compiler flags, you’d still have 2⁶ binaries.
Yeah, but that's because of pragmatic choices to limit the scope of the tool. In the wider context of "I've long been surprised there isn't more multiversion stuff built right into every language compile" it's easy to imagine a compiler that can heuristically detect which functions would benefit from certain CPU features, and walk over the call graph to find locations for runtime feature detection that balance detection overhead with code duplication for the fallback functions. For example merging the feature detection of adjacent function calls, making sure feature detection is moved out of hot loops, etc.
Obviously this is much easier to imagine than to implement. And in some languages it might be made impossible by certain language features (function pointers might become tricky). But this is more or less what some people do by hand in Rust with the more manual is_x86_feature_detected macro, so there's no obvious reasons why compilers couldn't automate it in at least some languages.
Ok, then it will be an explosion of binary size, if you have several code blocks optimized for each architecture level - I'm not very familiar with the subject, but I imagine it would have to be relatively large chunks of code, otherwise the constant branching would eat up the speed advantage.
It’s a real problem. I threw it at an old MUD game just to see how hard it is [0] then used differential testing and LLMs to rewrite it [1]. Just seems to be time and money.
Wow, as a former MajorMUD addict (~30 years ago) that's extremely interesting to see. Especially since MajorMUD is rarely discussed on HN, even in MUD or BBS-related threads.
Did you find it worked reasonably well on any portion of the codebase you could throw at it? For example, if I recall correctly, all of MajorMUD's data file interactions used the embedded Btrieve library which was popular at the time. For that type of specialized low-level library, I'm curious how much effort it would take to get readable code.
I am getting closer and closer to a full verified rewrite in Rust. I have also moved to a much easier sqlite relational structure for the backend.
I actually sidestepped the annoying btrieve problem by exporting the data using a
go binary [0] and I write it to a sqlite instance with raw byte arrays (blobs). btreive is weird because it has a dll but also a a service to interact with the files.
P.s. I have spent a lot of hours on this mainly to learn actual LLM capabilities that have improved a huge amount in the last year.
There was a recent post on here where the creator of Ladybird (Andreas Kling) translated a chunk of his novel browser from c++ to Rust in two weeks -- a feat he estimated would take him months: https://ladybird.org/posts/adopting-rust/
I, in my own way, have discovered that recent versions of Claude are extremely (as in, super-humanly) good at rewriting or porting. Apparently if recently released coding agents have a predefined target and a good test suite then you can basically tell them that you want X (well-defined target w/ good suite of tests) written in Y (the language/framework you want X written in but it isn't) -- and a week or two later you have a working version.
I have spent the last month wrapping my head around the idea that there is a class of tasks in software engineering that is now solved for not very much money at all. More or less every single aspirational idea I have ever had over the last 20 years or so I have begun emabarking on within the last two months.
I am curious, have you attempted to do this to any binary packed with commercial obfuscation/"virtualization" schemes (e.g. Orean's Themida/Code Virtualizer and VMProtect)?
No, I would need to find a binary to test on. I suspect it would produce horrible code at the decompiler layer but ultimately I would expect that function signatures are still relatively clean?
Its scary - once you get the differential testing harness set up it seems to be just a matter of time/tokens for it to stubbornly work through it.
Decompilation does not preserve semantics. You generally do not know whether the code from the decompiler will be compiled to semantically equivalent binary that you initially decompiled.
My test harness loads up the original DLL then executes that in parallel against the converted code (differential testing). That closes the feedback loop the LLM needs to be able to find and fix discrepancies.
I'm also doing this on an old Win32 DLL so the task is probably much easier than a lot of code bases.
I am applying differential/property based testing to all the side effects of functions (mutations) and return values. The rust code coverage is also used to steer the LLM as it finds discrepancies in side effects.
It is written up in my link - please bear in mind it is really hard to find the right level to communicate this level of detail at - so I'm happy to answer questions.
Many of the decompiled console games of the '90s were originally written in C89 using an ad-hoc compiler from Metrowerks or some off-branch release of gcc-2.95 plus console specific assemblers.
I willing to bet that the decompiled output is gonna be more readable than the original source code.
Not related to what I was saying. Compilation is a many-to-one transformation & although you can try to guess an inverse there is no way to guarantee you will recover the original source b/c at the assembly level you don't have any types & structs.
A process for using LLMs to do brute-force decompilation of binaries and conversion to another programming language - including testing to prove equality.
This process is targeting an old computer game but there is nothing preventing this being run on any binary.
1. I work mostly in Rust so I'll answer there in terms of async. This library [0] uses queues to manage workload. I run a modified version [1] which creates 1 writer and n reader connections to a WAL backed SQLite and dispatch async transactions against them. The n readers will pull work from a shared common queue.
2. Yes there is not much you can do about file IO but SQLite is still a full database engine with caching. You could use this benchmarking tool to help understand where your limits would be (you can do a run against a ramdisk then against your real storage).
3. As per #1, I keep connections open and distribute transactions across them myself. Checkpointing will only be a problem under considerable sustained write load but you should be able to simulate your load and observe the behavior. The WAL2 branch of SQLite is intended to prevent sustained load problems.
I went through the same mental process as you and also use num_cpus [0] but this is based only on intuition that is likely wrong. More benchmarking is needed as my benchmarks show that more parallelism only works to a point.
You can see how the transactions work in this example[1]. I have a connection `.write()` or `.read()` which decides which queue to use. I am in the process [2] of trying to do a PR against rusqlite to set the default transaction behavior as a result of this benchmarking so hopefully `write()` will default to IMMEDIATE and `read()` remains DEFERRED.
Whilst I love the idea of SQLX compile-time checked queries it is not always practical to need a database connection to compile the code in my experience. If it works for you then thats great but we had a few tricky edge cases when dealing with migrations etc.
Also, and more fundamentally, your application state is the most valuable thing you have. Do whatever you feel makes you most comfortable to make sure that state (and state transitions) is as well understood as possible. rusqlite is that for me.
Weren't the compile-time connections to DB optional btw? They could be turned off I think (last I checked, which was last year admittedly).
My question was more about the fact that sqlx is integrated with tokio out of the box and does not need an extra crate like rusqlite does. But I am guessing you don't mind that.
SQLX has an offline mode where it saves the metadata of the SQL database structure but then you run into risk of that being out of sync with the database?
Yeah I just drop this one file [0] into my Tokio projects and I have a SQLite with single writer/multi reader pool done in a few seconds.
In other abuses of SQLite, I wrote a tool [0] that exposes blobs in SQLite via an Amazon S3 API. It doesn't do expiry (but that would be easy enough to add if S3 does it).
We were using it to manage a millions of images for machine learning as many tools support S3 and the ability to add custom metadata to objects is useful (harder with files). It is one SQLite database per bucket but at the bucket level it is transactional.
It builds a single Python WASM module with all dependencies included (they use VFS) and a Dockerfile to make the process easy (and actually worked first go). It does produce large files though: wasi-python3.11.wasm 110MB
[0] https://github.com/ronnychevalier/cargo-multivers