> Do you have any suggestions on what tools the people doing 'parlor tricks' with C++11 should be using to accomplish them, then? That is, what would you suggest to get to that place without the straightjacket?
Duh, Lisp of course. (Or Haskell.)
> Note that at a bare minimum, these supposed tools should give the ability for fine-grained manual resource control
Check. Lisp provides garbage collection, but you don't have to use it. It's perfectly possible to write Lisp programs that do manual memory management. It isn't often done because it's hardly ever a win, but if you really want to you can.
> zero-(runtime)-cost abstractions
a.k.a. macros
> and performance roughly on-par with C++.
The SBCL compiler is pretty good. But one of the reasons that Lisp code is not generally as fast as C/C++ is that Lisp code is safe by default whereas C/C++ code is not. You can make Lisp code unsafe (and hence faster) but you have to work at it, just as you can make C/C++ code safe, but you have to work at it. I submit that in today's world, being safe and a bit slower by default might not be such a bad place to be in the design space.
> these are all things that tend to be written in C++ or plain C.
There is a world of difference between C++ and plain C. C is actually not a bad language if you want to write fast code with relatively little effort and don't care about reliability or security. The value add of C++ over C is far from clear. (I don't know of any OS written in C++. Linus wrote a famous rant about why Linux is written in C and not C++. There are, however, examples of operating systems written in Lisp.)
> wouldn't it be more productive to ask questions about why people use it?
I know why people use it: it's fast, there is a huge installed base, and it's an excellent platform for studly programmers to display their studliness. That doesn't change the fact that C++ has deep design flaws which result in its being incredibly hard to use and extend. And the existence of coders studly enough to be productive in C++ does not change the fact that it imposes an extremely high cognitive load on its users.
No, macros are not what I'm talking about here: I mean that C++ provides abstractions that only impose runtime costs if you use them. For instance, the cost of vtable lookup is only paid if you are using virtual functions; otherwise, you don't have any overhead for function calls beyond what's imposed by the hardware.
As noted elsewhere in the thread:
> Unlike Lisp, C++ lets one write efficient, generic algorithms that can operate on several types of data. Lisp cannot, and falls back to dynamic type testing, which makes for slower code[1]. Basically your only option in Lisp is to specialize everything manually, or inline everything. Both approaches are extremely poor. As I hinted, Haskell and Standard ML do an even better job than both C++ or Lisp. This is talked about a good bit in this article [2].
That's what I mean when I say 'zero-cost abstraction'.
Lisp is exactly the same. You only pay the run-time cost of generic functions and dynamic type dispatch if you use them. What many people get hung up on is that in Lisp you get generic type dispatch by default, so to not pay that cost you have to do some work (declare types).
Operating systems: BeOS was written in C++. Genode[0] appears to be written in C++ too. (I'm pretty sure there are quite a few more, but that'll do as an existence proof.)
[0] Granted, that's an "operating system framework", but it's definitely at the same "level" as implementing an OS.
> Check. Lisp provides garbage collection, but you don't have to use it. It's perfectly possible to write Lisp programs that do manual memory management. It isn't often done because it's hardly ever a win, but if you really want to you can.
How does LISP work without a garbage collector? Closures without a garbage collector are pretty awful. Let's keep in mind that Rust gives us a pretty good idea of what a safe system without a GC looks like, and it doesn't look anything like LISP.
The same way any other language works without one: you allocate the storage you need and manage it yourself. It's not pretty, but it can be done. The resulting code ends up looking an awful lot like the code in any other imperative language. (Math gets a little tricky because you have to be careful not to inadvertently create bignums, but other than that it's pretty straightforward.)
> Closures without a garbage collector are pretty awful.
> Rust gives us a pretty good idea of what a safe system without a GC looks like
And yet, Rust has closures :-) (And indeed, they are pretty awful.)
Writing non-consing code in Lisp is no different from writing non-consing code in any other language. You can produce stack-allocated closures that get cleaned-up on function return, just as in Rust. If you want to write non-consing code in Lisp (or any other language) you just can't use first-class closures.
Right, I guess to me it's just almost not worth using a LISP if I can't use any of the features that make LISP enjoyable to use (LISP without conses, closures or any other interesting features). But I suppose in principle a very carefully written imperative LISP program could be pretty fast, sure :) I'd rather write in a language that supports that style of programming natively in that eventuality, though.
(I'd also add that there are some issues surrounding larger unboxed types, but I won't venture to posit how SBCL handles those).
You can have conses, you just have to allocate them all up-front. The when you want one, you don't call CONS, you call MY-CONS, which grabs one of the pre-allocated conses off your free list. Then when you're done with it, you push it back onto your free list so it can be reused. It's no different from having MALLOC and FREE, except that you have to write them yourself (but that's not hard).
But yes, it's a lot easier to write code with a GC than without one. That's true in any language.
But let's not forget that the original article was saying, essentially, "Hey, look, we can make C++ do Lispy things!". My point is just that if you want to do Lispy things it's a lot easier just to use Lisp than to try to shoehorn Lisp's features into C++.
It isn't really, though. If you write your own malloc and free like that (by the way, writing a performant, bugfree, concurrent malloc and free is not that easy :)), you're responsible for safety as well (e.g. use after free bugs) which LISP will no longer protect you against. That's not to mention that LISP has to interact with C on a regular basis for things like system calls, and comes with a runtime that prevents it from playing nicely as an embedded library (perhaps SBCL has a way of running without one, but I can't find it... and that's not really a product of functional-ness or lack of static compilation either either, I have heard from several people who have trouble using libraries built with ghc or Go). And in embedded contexts, you may need hard guarantees that, for example, no dynamic allocation of any sort is done, or that your program doesn't use the stack, etc. To the best of my knowledge, LISP has no facilities for either of these things.
(It does appear that SBCL lets you drop down to assembly, but again if you do that all the advantages of using LISP are gone. Anyway, what is the goal here? Do you really want to use a typed LISP with no lists, with large of featureless statically allocated memory, manually handling concurrency, mutability everywhere, inline assembly, and the inability to use even most of the C++ LISPy features because LISP has no support for using them without runtime costs? Writing a language without resorting to costly abstractions is hard and it was explicitly never a goal of LISP to be one. That's not to mention that in LISP it's nonobvious which features are costly and which ones aren't, so the abstraction it provides over hardware is only theoretical in this context).
In any LISP in a high performance context, you are always paying for things you don't use. You could probably argue that some of the above problems could be mitigated if everyone adopted SBCL as the standard, but unfortunately that's just the way it is in the real world. And while it is unfortunate, the fact is that even all the technical problems could be resolved (I have my doubts), it would be much more irritating to write such low-level systems code in LISP than in a language that wasn't so far removed from the workings of modern computer architecture.
I'm actually a big fan of Lisp and I've found it quite useful for a number of projects, but when you really need to do low-level programming, it is significantly easier in (modern) C++.
> If you write your own malloc and free like that ... you're responsible for safety as well (e.g. use after free bugs) which LISP will no longer protect you against.
That's right. There's no such thing as a free (no pun intended) lunch.
> (by the way, writing a performant, bugfree, concurrent malloc and free is not that easy :)),
The reason it's hard to write a malloc for C is that it has to manage variable-length blocks.
> That's not to mention that LISP has to interact with C on a regular basis for things like system calls, and comes with a runtime that prevents it from playing nicely as an embedded library
No, that's just wrong. There's nothing about Lisp that prevents it from being implemented as an embedded library, e.g.:
> You could probably argue that some of the above problems could be mitigated if everyone adopted SBCL as the standard
No, I'm saying use the right tool for the job. If you really need every last bit of speed and you don't care about safety or engineering cost then by all means use C or C++. But if you want safety, reliability, and the sort of run-time dynamism described in the original article you're better off using Lisp or its progeny.
I'm also saying that if you want performance and you also want to use Lisp, you can. But at the end of the day there are fundamental tradeoffs in computing between speed, safety, dynamism, and engineering cost that no language will save you from.
There are many contexts in which your malloc won't perform well, and many more where it will fall over in a concurrent environment (unless LISP uses atomic operations and locks by default, in which case you have much bigger performance problems to worry about). Concurrency without garbage collection is nontrivial, though I don't blame you for not thinking about it all that much if you rarely interact with such languages. It's great to learn about some of LISP's better-performing utilities (push and pop for example) but let's not get carried away. Also, allocating fixed-length blocks of memory is a perfectly reasonable allocation strategy in C.
> No, that's just wrong.
From the link, embeddable common LISP comes with a runtime, which makes it inappropriate in many contexts. I didn't say that LISP couldn't interact with C (obviously it can!) only that it's not particularly convenient. From the link, it supports inline C, which is great, but again you're not really using LISP at this point.
> No, I'm saying use the right tool for the job.
Oh, sure, I don't think we're disagreeing on that. Certainly most of the prominent Rust developers will immediately point you to a language like Haskell, Nimrod or Python if they will satisfy your usecase. It's just that some of your posts suggested that you think LISP could in principle be used in all the places C++ is used, which I don't think is necessarily true, and certainly it wouldn't be convenient to do so. For people who do have to use C++, I think these LISPy features are a nice way to make the experience more tolerable, and I think that's all the article was getting at.
Good point (but you have that problem in any language). However, PUSH conses, so my code is wrong in that regard (you have to use a pre-allocated free vector, not a free list). So I concede the point: writing your own allocator in Lisp is not trivial. But it can be done.
Duh, Lisp of course. (Or Haskell.)
> Note that at a bare minimum, these supposed tools should give the ability for fine-grained manual resource control
Check. Lisp provides garbage collection, but you don't have to use it. It's perfectly possible to write Lisp programs that do manual memory management. It isn't often done because it's hardly ever a win, but if you really want to you can.
> zero-(runtime)-cost abstractions
a.k.a. macros
> and performance roughly on-par with C++.
The SBCL compiler is pretty good. But one of the reasons that Lisp code is not generally as fast as C/C++ is that Lisp code is safe by default whereas C/C++ code is not. You can make Lisp code unsafe (and hence faster) but you have to work at it, just as you can make C/C++ code safe, but you have to work at it. I submit that in today's world, being safe and a bit slower by default might not be such a bad place to be in the design space.
> these are all things that tend to be written in C++ or plain C.
There is a world of difference between C++ and plain C. C is actually not a bad language if you want to write fast code with relatively little effort and don't care about reliability or security. The value add of C++ over C is far from clear. (I don't know of any OS written in C++. Linus wrote a famous rant about why Linux is written in C and not C++. There are, however, examples of operating systems written in Lisp.)
> wouldn't it be more productive to ask questions about why people use it?
I know why people use it: it's fast, there is a huge installed base, and it's an excellent platform for studly programmers to display their studliness. That doesn't change the fact that C++ has deep design flaws which result in its being incredibly hard to use and extend. And the existence of coders studly enough to be productive in C++ does not change the fact that it imposes an extremely high cognitive load on its users.