When they suddenly need a top-level engineer to solve a hard problem, they hire one who consults for three months, works remotely, and visits once or twice.
You've essentially described every job I've had since college. That's exactly my experience of how companies actually work in practice.
I'm constantly learning, and seeing when things become "possible" with the current state of knowledge. Whenever I see that, on my own, I'll sketch out how the solution would work.
Then, I look around for companies that need a problem I already have a solution for solved, and I pitch my solution to them. Sometimes, I'll take jobs to solve problems I haven't considered before, but that's rarer.
If they like my solution, I set up a contract to implement it for them.
I'd say about 50% of the time I pitch a pre-existing solution, I get work. What's great is I'm essentially picking my own work and I get paid well (because the problems are hard).
The downside is you pretty much always have to be coming up with new solutions to things, and stay very current on technology.
I'm generally into something long before the "early adopters" get there, and I'm gone by the time it hits the mainstream. For example, I was working on a Google Spanner-like database while Google was doing the same.
At the time, no-one thought something like that was possible (outside of Google -- and they didn't tell anyone). I implemented different parts of the solution with three different clients over a two year period, none of whom hired me to create something like Google Spanner, but all of whom needed one part of the whole package solved.
So, in the best interview tradition, what do you read to keep up to date with the state of the art?
I have a couple of times found myself inventing solutions to problems I had in the business, only to see them appear as cutting edge open source at the same or a little later (ie Python deployment tools)
I have always assumed one needs to be working in an area, pushing hard and then finding that "what no-one has an answer to this!?" moment.
I would have trouble believing I could find a solution at the cutting edge, and find the people who needed it, without being knee-deep myself.
As you suspected, I'm developing solutions in the areas I work in specifically (stuff I'm "knee-deep" in).
I read the usual CS papers, plus lots of PhDs thesis. I follow the references in the bibliographies like crazy, and I also look up the writer's on the 'Net and see what other stuff they've done/written.
It feels like detective work, actually: the most-cited papers are frequently not the best. There's tons of researchers working in obscurity but who nevertheless have useful results.
I see my role as taking advances from research and bringing them into practical use.
It feels like detective work, actually: the most-cited papers are frequently not the best. There's tons of researchers working in obscurity but who nevertheless have useful results.
What are some examples of the best research that is obscure?
Sven Havemann's PhD on "Generative Mesh Modelling"[0] is brilliant. At this point, it could be implemented on top of Blender 3Ds newest modelling kernel.
In general, stack languages are great for graphics in both 2 dimensions (PostScript) and 3 dimensions (GML)[1].
If you have experience solving hard problems that helps. I had the good fortune of doing so in graduate school. That lead to a job solving another totally unrelated hard problem, with a technology stack totally unrelated to my graduate work. Once I had two gigs like that, the only people who responded to my resume's were people looking for someone to work with immature technology, or had a very hard problem to solve.
Also once you decide that you want to have a professional career jumping on technology grenades, do not work as an employee. I made that mistake. The nature of grenade jumping is that your are asked to solve a hard problem that needs to be solved in a short time, if you are an employee this amounts to massive hours, (70-90 hour work weeks), of unpaid overtime. Work as a contractor and be sure to bill for every hour. Or be sure to get a very healthy option grant if it is a promising start up. My life has been much happier once I started doing that.
Also be sure to work out, this sort of work takes a severe toll on your health if you do it for 20-30 years.
I've been in more or less the same situation, but for longer periods of time.
I'm curious given your experience. My biggest issue is that I don't know what a 'hard problem' is, I know what comes naturally to me, and what takes me some time to grok, but my experience is that those problems that come natural to me seem to be extremely obtuse to others.
So what types of problems do you solve that allow you to market yourself in such a manor?
I assume you have NDA type structures in place so I expect very general responses.
You've essentially described every job I've had since college. That's exactly my experience of how companies actually work in practice.