Mold is absolutely excellent work for modern systems.
I've recently been trying to speed up our project builds, and found that linking is absolutely a huge bottleneck. I've got 24 cores * 2 threads to build on, and maybe 30% of that power goes unused because of the linker.
I've made a previous attempt to build with mold but it didn't quite work at the time. I'll be giving it another try.
Mold is amazing. I was playing around with O3DE some months ago, and tried switching to Mold to see if it could improve my build-run cycle times, and it absolutely did. I don't remember the exact numbers, but it was something crazy like multiple seconds with gold and lld, down to under a second with mold.
I'm not OP but I've worked on projects with similar link times. Large games that are one monolithic executable and have enormous debug information filed (sym or pdb depending on platform). Heres a good blog post [0].
Not the OP either, but I also had terrible link times in SQLPage before I switched to mold. SQLPage is a web server that bundles a lot of html component templates directly in the server binary. This is very practical for users, but was painful for developers, before mold.
We've also done some PoCs at work, and the total build time for our UI layer (complex C++/Qt stuff) dropped from 44-45 mins to 29 by going from lld to mold on a smaller test build machine.
Very happy to read about the license change! I hope they are able to earn money from the project, but the likelihood of being able to integrate it into any work projects is much higher with MIT licensing. If we do use it, I'll try to get our company to sponsor the project.
The license on a linker shouldn't matter. It isn't injecting copyrighted code and there's already precedent for excepting trivial boilerplate in the GPL ecosystem so nothing in the generated binary should be affected by a copyleft license on the tooling. AGPL would only restrict deploying a privately modified linker via a network service which isn't a realistic scenario for a basic dev tool.
While that is true, it requires a very specific understanding of both intellectual property law and the inner workings of development tooling that is pretty rare in practice. As such, companies are generally conservative in such decisions. Most organizations have a single list of licenses allowed for use rather than getting into specific use cases for specific licenses. These types of environments will generally limit uptake of copyleft tools.
All that said, I'm not convinced the licensing is the issue here (although I wish them the best). We are in a world that has grown accustom to free development tools and building a commercially viable business in tooling is incredibly difficult. I'm always amazed by how many developers I know who make a living by creating software yet are unwilling to pay for software themselves.
> Most organizations have a single list of licenses allowed for use rather than getting into specific use cases for specific licenses.
Bullshit. Most organizations are absolutely fine with different ad-hoc licenses for various closed source software. Applying different standards to copyleft licenses is just due to FUD.
Ignoring the problem of convincing legal team that linker license doesn't affect it's output, there is the problem of including the linker in prepackaged toolchains - like the ones part of Visual Studio, Xcode, Android studio or embedded toolchains provided by hardware vendors. In some of those cases you might still get away with AGPL, but it will require putting in some effort to comply with license since in those cases the linker itself is redistributed. It's no secret that Apple is avoiding GPL as much as they can, with many of commandline tools they provide either being BSD versions or very old GPL2 versions.
> AGPL would only restrict deploying a privately modified linker via a network service which isn't a realistic scenario for a basic dev tool.
Interacting with linker over network service may sound weird, but it's not that uncommon. For example Unity offers Cloud build service for their engine which means indirectly interacting with Android and iOS toolchains. All major cloud providers are making solutions where the development tools and libraries are tightly integrated with their cloud service in attempts to make it harder migrating your project away from them. Regular CI/CD service providers are including the most popular development tools in their default environment, both to simplify the development process and also so that they can better cache them thus saving network costs and speeding up builds compared to each customer downloading the toolchain manually. There was also a period where multiple companies where pushing a remote dev environment as a solution to minimize the hassle of having each developer setup things locally thus improving onboarding speed, ensuring everyone is working in the same environment and also simplifying work for company wide IT management.
In many of those cases there might be 2 or even 3 companies repackaging and redistributing between the original software (linker) author and final user (programmer).
Even in the land of open source things aren't that simple. Not sure if it's still a thing but there was a period when FreeBSD was trying to remove GPL from it's base packages.
Third party CI builds aren't distributed to anyone. Even if the compiled binary was subject to GPL rules, the binary will almost never exit the CI server where it was built so copyleft protection won't be invoked.
You can use modified GPL code to your heart's content in a corporate setting. As all your coworkers are part of one legal entity, private use within the organization is not distribution per the terms of GPL. You have to distribute a binary to someone who can make a claim under the terms of the license before copyleft is activated. Furthermore, you only ever have to disclose source to someone with possession of a derived binary.
I was talking about the case where linker is the software and user is the developer performing linking. And MOLD was previously AGPL licensed, meaning that having access to binary isn't required, it's sufficient to interact with the software over network for copyleft to activate.
Let's imagine Google wanted to include Mold in Android sdk, and CI service company (like Travis or Github with their Actions) want to include Android SDK in their VM images. I would consider using of such CI service for building android app interacting with the linker software over network. Meaning everyone in the middle (Google and CI service) has to deal with license requirements.
The main purpose of using AGPL as the original license is to force companies that have a blanket-ban on AGPL to buy a commercial license or just buy the entire project, as told by the author.
Good luck convincing corporate lawyers that the license terms don't matter since we're just using it for linking. I don't know many (any?) that would risk it.
I'm curious about the license change? This is an executable is it not? Invoking it as a separate process does not require you make the software calling it GPL so switching to MIT should have no affect in the common case.
If the authors really wanted a more permissive license, then instead of relicensing from AGPL to MIT they should have relicensed from AGPL to AGPL with linking exception. An example of a project that is GPL with linking exception is libgit2 [1]. This licensing is more permissive but still permits the author to sell commercial licenses to those making closed-source code changes.
> This licensing is more permissive but still permits the author to sell commercial licenses to those making closed-source code changes.
I think the point is that the authors don't want want to continue selling licenses, as it wasn't worth the hassle. I guess `sold`, the macOS version is an exception.
If selling licenses is a hassle, then that indicates a problem with the open source ecosystem as GitHub and other code hosting websites should offer monetization tools for selling closed-source licenses directly from their web interface. I'm talking legal forms, templates, payment processors, and product tracking. Selling licenses should be easy, not a hassle.
The MPL or the BSL would be a better license. The best would be the MPL with something like Facebook's "700 mil mau" and above requiring a support contract. The Mangas corps benefit the most from mold given the inherent size of their C++ codebases.
This linker noticeably improves rust development happiness on an exploratory, chunky repo of mine that is trying to be a big ole web monolith (uses SeaORM and axum/tokio). You don't want to know the size of my `target` directory, but incremental builds are snappier!
just in case I confused: i'm not sure mold helps with the size of that directory :D. I was commenting on how big my rust repo is really, just saying that mold seems to help big builds like mine
Is there a way to override this linker setting only for your local install? Ie i don't want to change production code or binaries, but it would be nice to have faster builds
Yes, it should just work for all projects if you put in in ~/.cargo/config.toml, or as a per-project setting.
Git life pro tip (that you didn't know you needed until now): You can use .git/info/exclude (present in every Git repository) as a local, private version of .gitignore. It has the same syntax as .gitignore, but isn't tracked by Git. So you could add .cargo/config.toml to .git/info/exclude, changing the linker locally without Git considering it as an untracked file. I generally use this feature to ignore files that are specific to my development setup which I don't want to list in .gitignore.
With the timings coming from the second build. The project links in LLVM, and the tests were done under WSL1 on Windows 10 with the files on the Windows file system, which is how the project is developed in general.
Very wise decision. I know in my own products/code I sometimes have to swallow the tough pill and admit that some things just aren't a good fit for monetization. The market speaks, unfortunately, and it is important to try not to fight it, but instead find something else the market rewards.
At the very top of mold's README (my guess is no, it won't become OSS):
> This repository contains a free version of the mold linker. If you are looking for a commercial version that supports macOS please visit the repository of the sold linker.
But how (and why not) such a fine cratsman should be rewarded? Companies (maybe of all sizes) should allocate some funds to their top dependencies/tools or something and this should be listed on their main websites. More of a cultural norm I guess?
> Mold 2.0.0 is a new major release of our high-speed linker. With this release, we've transitioned our license from AGPL to MIT, aiming to expand the user base of our linker. This was not an easy decision, as those who have been following our progress know that we've been attempting to monetize our product through an AGPL/commercial license dual-licensing scheme. Unfortunately, this approach didn't meet our expectations. The license change represents our acceptance of this reality. We don't want to persist with a strategy that didn't work well.
Anyone remember having to pay for cc on Solaris? [0] It was horrible and a terrible way to treat developers who are writing software for your OS!
We have been conditioned for a very long time to not need to pay for low level developer tools and to pay for support instead. I'm surprised they even tried to license it like that.
There’s a distinction between the maker of the OS/hardware and an independent vendor trying to monetize the tools.
The OS/hardware maker’s primary interest is in selling more OS or hardware, so it makes sense for them to give away the tools (or their work enhancing the already-free tools) that enable more and better applications.
An independent tool vendor is in a different position, and there have always been vendors with better tools, maybe for a specialized market, or maybe just going above and beyond what comes for free, which is what Mold tries to do.
In other words, it’s clear to everyone now that the essential tools should be free, but surely not that all tools should be free!
They were charging $100/yr [1]. Literal peanuts. Actually, that is unfair to the peanuts, it is literal pocket lint.
The average cost of a developer is more than $100/hr. mold needs to make you 0.05% (not 5%, 0.05%) more productive to directly pay for itself. If mold saves you 13 seconds a day in linking time it directly pays for itself. This ignores the knock on effects of reduced build time shortening the feedback cycle which completely dwarf the direct benefits.
It is ridiculous that such dirt cheap pricing for tools is viewed as a problem. As we saw in a post yesterday [2], this is why they say there is no money is tools.
Only $100/yr? Likely that is the problem then. Who's going to bother filing a P.O. with their mega company for only that amount?
It also wasn't paywalled, meaning tons of people probably just added it as a dependency in the CI and just ignored the license restrictions because it wasn't directly part of the shipped binary. There's no good way to enforce that.
> and just ignored the license restrictions because it wasn't directly part of the shipped binary. There's no good way to enforce that.
There's not anything to enforce. The AGPL allows you to run the application for any purpose. (If you modify the source, you have to make the modified source available to all users, including over the network users.)
At work in the 90s I was allowed to buy a brand new sparc workstation with a shrinkwrap compiler (and shrinkwrap sybase). One of the first things I did was compile gcc so I could install the whole gnu world.
They did the same at my first job out of college (Sun based company). It was my first introduction to free software. That was circa 1990... it's amazing that gcc is still going strong.
This is unfortunately still the norm in the FPGA world, which I think is a poor decision in the long term for Intel/Xilinx. I wanted to seriously get into FPGA dev a decade or so ago, I learned to write Verilog and everything but the tooling made me give up, it was too painful to deal with all the quirks and limitations of the free tools.
You sign up on their website and click ok on the long eula nobody reads and then click download. So yea, there are probably restrictions, but they don't have a paywall in front of them. This isn't legal advice at all.
GCC was largely ignored until the day Sun introduced the concept of UNIX user and developer editions.
Only then it started to gain momentum from people not willing to pay for Solaris Developer SDK.
In any case, I bet people using free beer tools enjoy being paid for their work, so just maybe they should also think in giving some money to the authors of those tools.
Over many years I've come across several discussions about fast linkers and how compile times sped up so much by replacing one linker with a faster one, but I've never found out what about them makes them normally so slow. Can anyone shed some light please?
This isn't the whole story, but linking is CPU-intensive and older linkers are mostly single-threaded. A lot of the performance gains come from doing work in parallel, which makes for a big improvement on beefy modern multicore CPUs.
Even single-threaded, modern linkers (lld, mold, even gold) are faster than BFD ld, which is notoriously slow, and sadly still the default you get on Linux.
Yep. A linker in the best case would run as fast as cat. Paste the binaries together, done. Disk I/O was a problem back when we used spinning rust but less so now.
What takes time is rewriting stuff as you go. Running the relocation tables to insert addresses into the code is cheap. Deadstripping sections is fairly cheap, deadstripping individual basic blocks within functions takes a lot more analysis and thus time.
Deduplicating constant strings is a good idea but involves streaming them all into a hashtable of some sort. Maybe you want them to share common suffixes, more work.
Deduplicating, deadstripping, rewriting debug information takes time. Debug builds can feature many gigabytes of dwarf to rewrite.
Oddly enough that the linker is scriptable, as in you can give it a program that it interprets, doesn't seem to be a significant cost. Probably because the script in question is quite short and somewhat limited in functionality.
Historically lld was very fast because it didn't bother doing any of the debug munging or other deduplication. Lld ran fast but the output binary was big.
I'm several years out of the linker performance game now so don't know the current status. In particular I don't know where mold or lld are in terms of quality of output vs their own performance.
"Once again, the goal is speed, in this case being faster than my second linker. That linker has been significantly slowed down over the years by adding support for ELF and for shared libraries. This support was patched in rather than being designed in. Future plans for the new linker include support for incremental linking–which is another way of increasing speed."
Just think of the apps that were written in early Unix days - simple single-purpose apps, probably just one source code file , just one obj and libc to link together, no shared libs et al.
The linker code just grew organically as new "must-have" features were added. Correctness of features was more important than speed esp. when spinning rust was a limiting factor.
I'm not sure how competitive it will be with lld, especially if we consider ThinLTO (which takes multiple minutes on 64-core machine) - it can make the advantages of mold insignificant.
> I'm not sure how competitive it will be with lld, especially if we consider ThinLTO (which takes multiple minutes on 64-core machine) - it can make the advantages of mold insignificant.
Mold is focused on (incremental) development builds where LTO is probably not what you want anyway. For actual release builds you shouldn't really care that much about the build time.
Anyone got any numbers/info on the impact to LTO optimizations? Some brief googling shows mold does support LTO, but is it as good/better than what you'll find in the LLVM/GCC linkers?
LTO for LLVM/clang and gcc is implemented by getting the compiler to emit internal compiler IR code rather than machine code to the object files. The linker's job is to call into the compilers at link time with the serialized IR code from the object files to produce machine code; the linker does not do the link-time optimization itself. Therefore LTO support in a linker is a pretty binary feature (does it support X compiler on Y platform?) without much of a "good/better" gradation. And when it comes to that, mold implements LTO support for both gcc and LLVM on Unix-like systems.
I'm subject to a moratorium on AGPL software at work. Some legal departments forbid or highly restrict use of AGPL licenses. They are concerned the licenses' viral nature causing problems for their software products.
I'm sure it is - but that leaves the engineer with the calculation to do of "time this commercial tool will save me" versus "time/goodwill I will burn trying to get the purchase approved".
There are plenty of ways to reduce friction. In this case, mold could have offered a 30 day commercial license to the company to demo the linker and see if the ROI was worth it.
Sometimes they do. But at best they support a small development team, and better if they help a niche to such an extreme that there isn’t a critical mass of people able to band together and make a free version. Something like hexrays is an example but IIRC open source tools are replacing them now.
It depends a lot on the project and your CPU. Incremental build (build, change 1 thing, rebuild) times are affected more than clean build times, because linking is a larger % of an incremental build.
When I benchmarked it about a year ago (with a fairly large Rust project on a 24-core machine), incremental builds went from ~8s to ~2.7s.
According to the developer, he has also written lld-link.exe[1] (i.e. lld that accepts link.exe flags for compatibility), which is already significantly faster than link.exe.
You configure with -DCMAKE_CXX_FLAGS='-march=native -ftlo' like any other cmake build. Or you `-G'Unix Makefiles' and export CFLAGS/CXXFLAGS before you build it if you really want to use make.
You should still be OK although I agree that CMake is really annoying to approach if you don't know the projects idioms particularly well e.g. there is a recurring bug in one of our builds where OMP initialization causes a deadlock, which can thus be disabled by not using openmp at cmake-time: finding how to do this when I went to disable it permanently took a good 20 minutes of guesswork because it's CMake magic versus make bullshit
I've recently been trying to speed up our project builds, and found that linking is absolutely a huge bottleneck. I've got 24 cores * 2 threads to build on, and maybe 30% of that power goes unused because of the linker.
I've made a previous attempt to build with mold but it didn't quite work at the time. I'll be giving it another try.