Oh can we coin a term like this for drunk driving? You know, when it’s more convenient to drive home drunk than leave your car out? That’s just being taxi-averse
They model a system which they know to be constrained by a closed-form equation called the Hamiltonian. They (cleverly, IMO) force the network’s predictions to be constrained by the Hamiltonian, by choosing the right output and loss function.
I don’t see a way to generalize this to the procedural rule-based systems you describe, unless they too are governed by a fairly simple continuous function Like the Hamiltonian.
I don’t know if it was “dramatic”, but it made me really happy.
I think software engineering is depressingly, suboptimally stable. The space of all possible ways to write programs is so, so large and almost completely unexplored. Lisps, Hindley-Milner type systems, Cox, neural networks with autodifferentiation ... whenever we substantially depart from the local optima around C-like programming, we find cool shit.
However, the vast majority of programming is done to build nearly indistinguishable CRUD SaaS apps. If we could just spend half our time doing real, interesting exploration of programming-space to satisfy our curiosity, maybe we could all agree to build apps with a more boring tech stack. However, that would require a large scale change in incentives that is hard to imagine shy of a Space Race style event.
... or Jonathan Blow was right and C was more or less the global max, and everything else was a mistake. I really hope that isn’t the case.
Part of me wants to ask “but who investigates the Inspector of the LECC?” But I’ve dealt with Elixir and I know that supervision trees rarely need to be very deep in practice
I think this is a good observation. One thing I’ve been thinking about is probabilistically evaluating traditional software. Every un-caught exception, for example, is usually a false negative when evaluating “did the code correctly handle the input?”
Not that ML code is bug-free. But it seems like most traditional software is probabilistically correct, despite not being thought of that way.
First off, I totally agree. Not as easy as it should be to write an async web server in python. FastAPI is probably your best bet. I usually use Sanic. Easy to accidentally block though.
That said, it sounds like you’re serving a large model. No amount of async/await or goroutines can solve this problem. A non-blocking web server is a godsend for I/O-bound tasks, but a large model is just a deep call stack - lots of multiply, nonlinear function like RELu, then add, times a billion. This would still block, even if you had perfect async/await code.
I made some assumptions here, but if I’m right, the answer is “shrink your model” and/or “buy more compute”. Neither of which are easy. But if you’re trying to shrink a model, check out Distiller https://github.com/NervanaSystems/distiller
Edit: the restriction I talk about is for event-loop based servers using something like uvloop or asyncio under the hood. Maybe this restriction doesn’t hold for other concurrency modes.
> No amount of async/await or goroutines can solve this problem. A non-blocking web server is a godsend for I/O-bound tasks
In the past, we were told that threads were cheap and to use them heavily, especially to achieve parallelism. Now with the advent of async models, we're being told that threads are expensive, and often that a single processor/thread async model is better than a multi-threaded blocking one.
I'm not a luddite, I do agree that async is often better. But I wonder how we got tricked into thinking more and more threads were the answer and how we avoid such trickery again.
A little off-topic, but I've built a small backend server in Node.js and Tensorflow.js to run a previously Pyhton-built model and was amazed by how performant and non blocking it runs.
The model can do around 10k predictions/s and does it with async, which allows Node to respond to web requests in the meanwhile.
I guess it's a matter of using the right tool for the task, whenever possible, Python for data science, Nodejs for a web backend.
Async doesn’t help with CPU-bound tasks, but yes, you can rig up systems that involve running multiple Python processes behind a load balancer. It’s just more work.
Oh can we coin a term like this for drunk driving? You know, when it’s more convenient to drive home drunk than leave your car out? That’s just being taxi-averse