Good Christ. Imagine having decided that your price structures should be a JSON file instead of persisted in a database and then thinking that any decision made by that person/team is a good idea.
I look forward to when we see the article about breaking the monorepo nightmare.
Sometimes this sort of thing is not a bad idea. If it's a simple data structure that doesn't change very often, you get an admin interface (vi), change tracking, and audit trail for free. Just think of it as configuration rather than data and most folks would think it's normal to do this.
I’ve used pre-commit the tool and now prek for the better part of a decade and never had these issues even using rebase flows exclusively. This seems like an operator error.
The special hooks for context and arena (actually arena(s) can be part of context) should have eliminated the need to change signatures for threading context and arena handles through the chain of calls. Instead there should have been an API (both - internal and user accessible) to check and pick, if present, the closest one on stack (somewhat similar to how you can get ClassLoader and the hierarchy of them in Java)
I have done the same; it's not natural to do it this way. Go should actually express an explicit mechanism to do this. When I did it, it felt exactly like trying to use epoll from Go: you can do it, it just feels like crap.
Man this person is mediocre at best. You can do fully manual memory management in Go if you want. The runtime is full of tons of examples where they have 0-alloc, Pools, ring buffers, Assembly, and tons of other tricks.
If you really want an arena like behavior you could allocate a byte slice and use unsafe to cast it to literally any type.
But like… the write up completely missed that manual memory management exists, and Golang considers it “unsafe” and that’s a design principle of the language.
You could argue that C++ RAII overhead is “bounded performance” compared to C. Or that C’s stack frames are “bounded performance” compared to a full in-register assembly implementation of a hot loop.
But that’s bloody stupid. Just use the right tool for the job and know where the tradeoffs are, because there’s always something. The tradeoff boundary for an individual project or person is just arbitrary.
As someone who writes Go code that processes around 100B messages per day (which all need to be parsed and transformed), I can confirm that the author’s position is very much misguided.
And it also completely ignores the fascinating world of “GC-free Java”, which more than a few of the clients I work with use: Java with garbage collection entirely disabled. It’s used in finance a lot.
Is it pretty? No.
Is it effective? Yes.
Regarding Go’s memory arenas,
do you need to use memory arenas everywhere ? Absolutely not. Most high performance code has a hot part that’s centered (like the tokenizer example that OP used). You just make that part reuse memory instead of alloc / dealloc and that’s it.
Same. I'm genuinely confused by all the comments of 'ah man, this is holding me back' in this thread, and folks claiming it's not possible to do any arena tricks in Go.
I'm not sure if these are just passerbys, or people who actually use Go but have never strayed from the std lib.
This isn't true in practice because you won't be able to control where allocations are made in the dependencies you use, including inside the Go standard library itself. You could rewrite/fork that code, but then you lose access to the Go ecosystem.
The big miss of the OP is that it ignores the Go region proposal, which is using lessons learned from this project to solve the issue in a more tractable way. So while Arenas won't be shipped as they were originally envisioned, it isn't to say no progress is being made.
I had to fork go’s CSV to make it re-use buffers and avoid defensive copies. But im not sure an arena api is a panacea here - even if i can supply an arena, the library needs certain guarantees about how memory it returns is aliased / used by the caller. Maybe it would still defensive copy into the arena, maybe not. So i don’t see how taking arena as parameter lets a function reason about how safely it can use the arena.
I personally loved using Go 8 years ago. When I built a proof of concept for a new project in both Go and Rust, it became clear that Rust would provide the semantics I’m looking for out of the box. Less fighting with the garbage collector or rolling out my own memory management solution.
If I’m doing that with a lot of ugly code - I might as well use idiomatic Zig with arenas. This is exactly the point the author tried to make.
Your last paragraph captures the tension perfectly. Go just isn’t the tool we thought for some jobs, and maybe that’s okay. If you’re going to count nanoseconds or measure total allocations, it’s better to stick to a non-GC language. Or a third option can be to write your hot loops in one such language; and continue using Go for everything else. Problem solved.
Go made it explicitly clear when it was released that it was designed to be a language that felt dynamically-typed, but with performance closer to statically-typed languages, for only the particular niche of developing network servers.
Which job that needs to be a network server, where a dynamically-typed language is a appropriate, does Go fall short on?
One thing that has changed in the meantime is that many actually dynamically-typed languages have also figured out how to perform more like a statically-typed language. That might prompt you to just use one of those dynamically-typed languages instead, but I'm not sure that gives any reason to see Go as being less than it was before. It still fulfills the expectation of having performance more like a statically-typed language.
By the common usage of the term, it is most definitely a systems language.
Systems are the "opposite" of scripts. Scripts are programs that perform a one-off task and then exit. Systems are programs that run indefinitely to respond to events. We have scripting languages and we have systems languages. While all languages can ultimately be used for both workloads, different feature-sets gear a language towards one or the other. Go is does not exhibit the traits you'd expect of a scripting language.
This idea that Go isn't a systems language seems to stem from "Rustacians" living in the same different world which confused sum types with enums, where they somehow dreamed up that systems are low-level programs such as kernels. To be fair, kernels are definitely systems. They run indefinitely too. But a user land server program that runs continuously to serve requests is also a system as the term has been normally used.
Long before Rust or Go existed, “systems languages” were commonly the ones you can write a whole system in to run on hardware, like C, Pascal or C++. I’m not opposed to that definition changing, but it certainly hadn’t when Go came out.
I agree that Rust enums should have been called unions, though.
While that does not match my memory, it works too. An assembly language is part of Go, so it fits among the languages you mention. The only constraint it imposes is your imagination (and what the hardware is capable of). However, it remains that "systems language" was caveated as being for network servers specifically. But no matter how you slice it, I think we can agree that Go isn't a scripting language, so it must be a systems language.
Rust does use enums under the hood in order to implement sum types, so the name as it is used within the language is perfectly valid. It's just not clear how that turned into nonsense like Go not having enums (which it does).
It says "more like", not "like". Javascript now performs more like a statically-typed language, as one example. That wasn't always the case. It used to be painfully slow — and was so when Go was created. The chasm between them has shrunk dramatically. A fast dynamically-typed language was a novel curiosity when Go was conceived. Which is why Go ended up with a limited type system instead of being truly dynamically-typed.
> Or a third option can be to write your hot loops in one such language; and continue using Go for everything else. Problem solved.
Or use Go and write ugly code for those hot loops instead of introducing another language and build system. Then you can still enjoy nicety of GC in other parts of your code.
I can see that being an option in a small team that works closely with one another and wants to keep things simple.
Though it is my personal opinion that forcing a GC-based language to do a task best suited for manual memory management is like swimming against the tide. It’s doable but more challenging than it ought to be. I might even appreciate the challenge but the next person maintaining the code might not.
> If you really want an arena like behavior you could allocate a byte slice and use unsafe to cast it to literally any type.
A word of caution. If you do this and then you store pointers into that slice, the GC will likely not see them (as if you were just storing them as `uintptr`s)
I still would be wary, even in that case. Go does not guarantee that the address of an allocation won't change over the lifetime of the allocation (although current implementations do not make use of this).
If you really store just references to the same arena, better to use an offset from the start of the arena. Then it does not matter whether allocations are moved around.
Types with inner pointers add difficulty to be sure, but it’s still possible to use them with this pattern. You have to make sure of three things to do so: 1) no pointers outside of the backing memory; 2) an explicit “clear()” function that manually nulls out inner pointers in the stored object (even inner pointers to other things in the backing slice); 3) clear() is called for all such objects that were ever stored before the backing slice is dropped and before those objects are garbage collected.
> You can do fully manual memory management in Go if you want. The runtime is full of tons of examples where they have 0-alloc, Pools, ring buffers, Assembly, and tons of other tricks.
The runtime only exposes a small subset of what it uses internally and there's no stable ABI for runtime internals. If you're lucky to get big enough and have friends they might not break you, some internal linkage is being preserved, but in the general case for a general user, nope. Updates might make your code untenable.
> If you really want an arena like behavior you could allocate a byte slice and use unsafe to cast it to literally any type.
AIUI the prior proposals still provided automated lifetime management, though that's related to various of the standing concerns, so you can't match that from "userspace" of go, finalizers don't get executed on a deterministic schedule. Put simply: that's not the same thing.
As someone else points out this is also much more fraught with error than just typing what you described. On top of the GC issue pointed out already, you'll also hit memory model considerations if you're doing any concurrency, which if you actually needed to do this surely you are. Once you're doing that you'll run into the issue, if you're trying to compete with systems languages, that Go only provides a subset of the platform available memory model, in the simplest form it only offers acq/rel atomic semantics. It also doesn't expose any notion of what thread you're running on (which can change arbitrarily) or even which goroutine you're running on. This limits your design space quite significantly at the bounds your performance for high frequency small region operations. I'd actually hazard an educated guess that an arena written as you casually suggest would perform extremely poorly at any meaningful scale (lets say >=32 cores, still fairly modest).
> You could argue that C++ RAII overhead is “bounded performance” compared to C. Or that C’s stack frames are “bounded performance” compared to a full in-register assembly implementation of a hot loop.
> But that’s bloody stupid. Just use the right tool for the job and know where the tradeoffs are, because there’s always something. The tradeoff boundary for an individual project or person is just arbitrary.
Sure, reducto ad absurdum, though I typically would optimize against the (systems language) compiler long before I drop to assembly, it's 2025 systems compilers are great and have many optimizations, intrinsics and hints.
> Man this person is mediocre at best.
Harsh, I think the author is fine really. I think their most significant error isn't in missing or not discussing difficult other things they could do with Go, it's seemingly being under the misconception prior to the Arena proposal that Go actually cedes control for lower level optimization. It doesn't, and it never has, and it likely never will (it will gain other semi-generalized internal optimizations over time, lots of work goes into that).
In some cases you can hack some in on your own, but Go is not well placed as a "systems language" if you mean by that something like "competitive efficiency at upper or lower bound scale tasks", it is much better placed as a framework for writing general purpose servers at middle scales. It's best placed on systems that don't have batteries, and that have plenty of ram. It'll provide you with a decent opportunity to scale up and then out in that space as long as you pay attention to how you're doing along the way. It'll hurt if you need to target state of the art efficiency at extreme ends, and very likely block you wholesale.
I'm glad Go folks are still working on ideas to try to find a way for applications to get some more control over allocations. I'm also not expecting a solution that solves my deepest challenges anytime soon though. I think they'll maybe solve some server cases first, and that's probably good, that's Go's golden market.
4 ways to demonstrate that the author either knows nothing about closures, structs, mutexes, and atomicity OR they just come from a Rust background and made some super convoluted examples to crap on Go.
“A million ways to segfault in C” and its just the author assigning NULL to a pointer and reading it, then proclaiming C would be better if it didn’t have a NULL value like Rust.
>I have been writing production applications in Go for a few years now. I like some aspects of Go. One aspect I do not like is how easy it is to create data races in Go.
To me it looks like simple, clear examples of potential issues. It's unfortunate to frame that as "crapping on Go", how are new Go programmers going to learn about the pitfalls if all discussion of them are seen as hostility?
Like, rightly or wrongly, Go chose pervasive mutability and shared memory, it inevitably comes with drawbacks. Pretending they don't exist doesn't make them go away.
Go is a bit more of a low level language compared to actor languages where the language enforces that programming model. I think the point of the slogan is that you want to make the shared memory access an implementation detail of your larger system.
Threads share the same memory by definition, though. When you isolate these threads from a memory PoV, they become processes.
Moreover, threads are arguably useless without shared memory anyway. A thread is invoked to work on the same data structure with multiple "affectors". Coordination of these affectors is up to you. Atomics, locks, queues... The tools are many.
In fact, processes are just threads which are isolated from each other, and this isolation is enforced by the processor.
Goroutines aren't posix threads. They could've lacked shared memory by default, which could be enforced by a combination of the compiler and runtime like with Erlang.
Go's creators said "Don't communicate by sharing memory", but then designed goroutines to do exactly that. It's quite hard to not share memory by accident, actually.
It's not like it's a disaster, but it's certainly inconsistent.
I don't think allowing developers to use their discretion to share state is "certainly inconsistent". Not sure what your threshold is for "quite hard" but it seems pretty low to me.
Goroutines could've lacked shared memory by default, requiring you to explicitly pass in pointers to shared things. That would've significantly encouraged sharing memory by communicating.
The opposite default encourages the opposite behaviour.
Concurrent programming is hard and has many pitfalls; people are warned about this from the very, very start. If you then go about it without studying proper usage/common pitfalls and do not use (very) defensive coding practices (violated by all examples) then the main issue is just naivity. No programming language can really defend against that.
Also, these are minimal reproducers, the exact same mistakes can trivially happen in larger codebases across multiple files, where you wouldn't notice them immediately.
> Pretending they don't exist doesn't make them go away.
It's generally assumed that people who defend their favorite programming language are oblivious to the problems the language has or choose to ignore these problems to cope with the language.
There's another possibility: Knowing the footguns and how to avoid them well. This is generally prevalent in (Go/C/C++) vs. Rust discussions. I for one know the footguns, I know how bad it can be, and I know how to avoid them.
Liking a programming language as is, operating within its safe-envelope and pushing this envelope with intent and care is not a bad thing. It's akin to saying that using a katana is bad because you can cut yourself.
We know, we accept, we like the operating envelope of the languages we use. These are tools, and no tool is perfect. Using a tool knowing its modus operandi is not "pretending the problems don't exist".
> Using a tool knowing its modus operandi is not "pretending the problems don't exist".
I said that in response to the hostility ("crap on Go") towards the article. If such articles aren't written, how will newbies learn about the pitfalls in the first place?
While I agree with you in principle, there is a small but important caveat about large codebases with hundreds of contributors or more. It only takes 1 bad apple to ruin the bunch.
During the short time I was working on a Go project I spent a significant amount of time debugging an issue like the one described in his first example in a library we depended on, so it's definitely not a problem of “super convoluted example”.
I look forward to when we see the article about breaking the monorepo nightmare.
reply