This is how I think about Go slices (may help other understand them).
A slice itself is just a window into a backing array of fixed size. The slice carries three data members. The pointer to the backing array and its remaining capacity and the length of the slice data.
Typically slices are passed around by value but you can take their address and modify a "shared" slice.
The built-in append() returns a new slice by value.
What happens is simply that when appending data to a slice and there is no room in the backing array, a new backing array is allocated that the returned slice points into. The old "input" slice to append is still intact and if some code has access to it, it will look at data stored in the old backing array.
I've constructed similar utility types in C and find them quite convenient. It's very convenient to have the distinction between the backing memory (array) and a slice viewing a portion of it instead of just a dynamic array.
> I've constructed similar utility types in C and find them quite convenient. It's very convenient to have the distinction between the backing memory (array) and a slice viewing a portion of it instead of just a dynamic array.
Of course that's convenient, the issue of Go's slices is that they act as both a dynamic array and a slice viewing a portion of one. The two uses conflict with one another, and the interactions are full of traps.
If it were true in the typical sense that “append() returns a new slice by value”, then you would expect to be able to mutate the old slice and the new slice independently from each other. But in reality, you can only do this if append() decided to reallocate, which only happens at some implementation-defined exponential pattern of sizes.
package main
import "fmt"
func main() {
a := []int{0}
for i := 0; i < 40; i++ {
b := append(a, 0)
a[i] = 1
fmt.Printf("%d ", b[i])
a = b
}
}
Does that mean that a) the pointer to the single element is only invalidated on “append” if the slice has no more capacity (this is how C++ vectors work), or b) does the fact that there’s an active reference into the slice always cause a reallocation (copy-on-write style)? If it’s the former, we’re literally in the C++ iterator invalidation nightmare, only without the debugging tools.
The array is only reallocated when it runs out of space. There is no magic about the "old" array. The GC just won't collect it, as long as a pointer to it exists.
I don't think there are many reasons, if any at all, to keep a pointer to an array element of a slice around in Go. Usually I only get the address of an array element only when passing it to some C code or doing some low level manipulation, but then I don't keep the pointer around. In Go code you usually just keep the slice object - which contains the necessary pointer anyway.
Without knowing much Go I believe it's neither a nor b. The pointer to the single element will always stay valid, no matter whether reallocation happens or not (and having a pointer doesn't influence whether reallocation happens or not).
Re-allocation might be a confusing word here because afaik it's actually always a new allocation (the old one is not touched) and only if there are no more pointers to the old allocation will the next GC cycle deallocate it.
So there is never iterator invalidation like in C++ but of course you still need to be careful because you might accidentally share or not share the same underlying data.
This is correct. A pointer like `p := &x[0]` will always point at the original backing array even if an append on the slice causes the slice to allocate a new backing array. This means that you can update `x[0]` on the new slice without changing `*p`.
Since slice API is pass-by-value, in theory ANY method will invalidate the pointer. In practice only resizing methods actually NEED to reallocate the underlying array, but magic can happen. However, refcounting will make sure that a previously underlying array having pointers to it will remain allocated. This means that 1. pointers to single elements will always dereference 2. slice structure modification can leave pointers pointing to stale data
A slice itself is just a window into a backing array of fixed size. The slice carries three data members. The pointer to the backing array and its remaining capacity and the length of the slice data.
Typically slices are passed around by value but you can take their address and modify a "shared" slice.
The built-in append() returns a new slice by value.
What happens is simply that when appending data to a slice and there is no room in the backing array, a new backing array is allocated that the returned slice points into. The old "input" slice to append is still intact and if some code has access to it, it will look at data stored in the old backing array.
I've constructed similar utility types in C and find them quite convenient. It's very convenient to have the distinction between the backing memory (array) and a slice viewing a portion of it instead of just a dynamic array.