It's too early. People are trying all of the above. I use all of the above, specifically:
- A well-structured folder of markdown files that I constantly garden. Every sub-folder has a README. Every files has metadata in front-matter. I point new sessions at the entry point to this documentation. Constantly run agents that clean up dead references, update out of date information, etc. Build scripts that deterministically find broken links. It's an ongoing battle.
- A "continuation prompt" skill, that prompts the agent to collect all relevant context for another agent to continue
- Judicious usage of "memory"
- Structured systems made out of skills like GSD (Get Shit Done)
- Systems of "quality gate" hooks and test harnesses
For all of these, I have the agent set them up and manage them, but I've yet to find a context-management system that just works. I don't think we understand the "physics" of context management yet.
On your first point, one unexpected side effect I’ve noticed is that in an effort to offload my thinking to an agent, I often end up just doing the thinking myself. It’s a surprisingly effective antidote to writer’s block… a similar effect to journaling, and a good reason why people feel weird about sharing their prompts.
Came here to say something similar. For me, the craft aspect is now even more exciting because I can craft more ambitious things without getting bogged down in the details. For me, refining my conceptual model, drawing diagrams, finding the right way to think about something was the craft.
Maybe that's another way of saying: I was trained as a designer, and now the distinction between design (read: architecture, service-design, product, ux, cx) and programming is blurring.
Heck yeah! Love that way of putting it. Agree. Now there’s more time to focus on making the right architecture and carrying it out. It’s no longer a days long task to do a big refactor to remove code smells.
I am solidly in this "curious" camp. I've read HN for the past 15(?) years. I dropped out of CS and got an art agree instead. My career is elsewhere, but along the way, understanding systems was a hobby.
I always kind of wanted to stop everything else and learn "real engineering," but I didn't. Instead, I just read hundreds (thousands?) of arcane articles about enterprise software architecture, programming language design, compiler optimization, and open source politics in my free time.
There are many bits of tacit knowledge I don't have. I know I don't have them, because I have that knowledge in other domains. I know that I don't know what I don't know about being a "real engineer."
But I also know what taste is. I know what questions to ask. I know the magic words, and where to look for answers.
For people like me, this feels like an insane golden age. I have no shortage of ideas, and now the only thing I have is a shortage of hands, eyes, and on a good week, tokens.
But that knowledge was never hidden or out of reach. Why not read books, manuals, or take online classes? There is free access to all these things, the only cost is time and energy.
Everyone has tons of ideas. But every good engineer (and scientist) also knows that most of our ideas fall apart when either thinking deeper or trying to implement it (same thing, just mental or not). Those nuances and details don't go away. They don't matter any less. They only become less visible. But those things falling apart is also incredibly valuable. What doesn't break is the new foundation to begin again.
The bottleneck has never been a shortage of ideas nor the hands to implement them. The bottleneck has always been complexity. As the world advances do does the complexity needed to improve it.
> Why not read books, manuals, or take online classes? There is free access to all these things, the only cost is time and energy.
Sure, but it's just way faster now. I can get the exact right piece of knowledge I need to advance my understanding on demand, rather than having to spend minutes or hours tracking down the right information, then scanning around the text to filter out all the irrelevant stuff.
There's also a motivational effect: the activation energy of bothering to do this was such that in many domains, it didn't seem worth it.
> Everyone has tons of ideas
Most people have profoundly bad ideas
> Those nuances and details don't go away. They don't matter any less. They only become less visible. But those things falling apart is also incredibly valuable. What doesn't break is the new foundation to begin again.
Agree, however that's the challenge of this time. Things are becoming less visible. On the other hand, you can implement and get that feedback ten times faster, or point ten minds at stress-testing a concept in 3 minutes. For many of my projects, that's the difference between getting anything done vs idly fantasizing. For others, it could easily be irrelevant.
> The bottleneck has never been a shortage of ideas nor the hands to implement them. The bottleneck has always been complexity. As the world advances do does the complexity needed to improve it.
I don't think this is a coherent statement. How could you possibly surmount complexity with anything other than better ideas and more hands?
> I can get the exact right piece of knowledge I need to advance my understanding on demand
This is where I disagree. It would be different if these LLMs were acting as instructors and pushing you through courses designed for learning things, but this is more akin to looking at the section of a textbook that contains the exact paragraph you need. Or doing the same thing with a manual. I do not think this is the best way to learn and I actually think it is a good way to perpetuate misunderstandings. There are plenty of bad textbooks and docs, I don't want to dismiss that, but that extra information in the chapter, the previous chapters, or leading up to that paragraph are important. They are designed to be learned in order for a reason. Skipping that other information more often harms you than helps you. It gets you done with a task faster, but doesn't make you learn faster. Two different things. For review, that's different though, but the other knowledge is already there.
I think there's this belief that there are shortcuts to learning. That's a big mistake. You can't learn programming by reading, yet so many people try to do something similar with different domains. It is exactly the same thing that leads people to conspiracy theories. They have such "swiss cheese knowledge" that they don't understand how things actually are connected. How people use LLMs is typically to take the direct route to the information they want, which is only logical, but misses all that other important stuff that you wouldn't know is important to understanding those things until you have mastery of that knowledge.
If there was a shortcut, people would be writing manuals and textbooks very differently. We've been doing it for thousands of years and iterating it for just as long. It's converged to the place it has for a reason.
> Most people have profoundly bad ideas
Yes, and why? One of the most common reasons is people are missing the surrounding context. All those little details. It's exactly what I said before about figuring it out as you go. This is part of why the "doing" matters. Why that stuff that doesn't seem important to the novice actually is, and why experts include it in their teachings.
> you can implement and get that feedback ten times faster,
You can, but you can also dig yourself into a 10x deeper hole 10x faster. The LLM doesn't make you an expert. The LLM doesn't make context appear. I'm sorry, but all that nuance doesn't go away. The LLM only makes it less visible as it does some things for you and every time it does something you don't know how to do you're that much deeper into the water. It's okay to be in a little over your head (that's how we learn) but these tools also make it very easy to get into much deeper waters than you can handle. When that happens, you are unable to do anything and are at the mercy of the LLM. So good luck.
> I don't think this is a coherent statement
Because complexity is complex. There are many different types. Sit with the idea longer, I promise it is coherent. But I'm not going to give you a shortcut. Maybe the LLM will, I'm fairly confident it will be able to figure it out.
Honestly, based on where this is going, I suspect we might agree more than it appears in this short exchange.
As for the point about complexity, I will meditate on it.
My first cut is that what you're getting at is something like: you're using "complexity" in the technical sense of the word, in which case we're in the realm of so-called "wicked problems" and such. There is some kind of 'making sense' that goes beyond 'mere ideas,' or at the very least something like "throw more minds and hands at the problem," in this context, is the wrong attitude.
In such a world, a machine that amplifies human effort (while obscuring it) is not the right tool for the job—more than likely you find yourself spinning around an attractor basin.
I personally have a tentative conclusion that I specifically am able to avoid this by amplifying certain strategies I've developed over my life, but really, I don't have solid confirmation yet.
In any event, I'm enjoying the experiment, but I'll reflect on why you seem to be certain about it.
I agree and think we're mostly on the same page fwiw. But there are parts I disagree with. We are more aligned than misaligned. But I do want to clear some things up
> you're using "complexity" in the technical sense of the word
Unfortunately complexity is a bit more complex than this and I think it's leading to some miscommunication. Even in technical settings complexity is overloaded. Something can be computationally complex but easy to understand. Something can be conceptually complex, but trivial to compute. I do really mean "complexity is complex".
But I am using it in both senses. I'll make a pretty straight forward argument here. As we advance on any given topic it becomes harder to advance, right? Just take something simple like playing a video game. You start from nothing and then you get a lot better simply by figuring out the controls. But as you progress towards the "professional" level it both gets computationally harder to advance (it requires more time and training) as well as conceptually harder (as little details and the number of things you need to consider increases). You have to process more information and you have to account for more information. Another way to see this is with a mathematical example: a Taylor Series. A first order approximation is relatively easy to compute but it quickly grows as you want your accuracy to increase. And the relationship isn't linear...
> I personally have a tentative conclusion that I specifically am able to avoid this by amplifying certain strategies I've developed over my life
I can't answer this. I'm just some dude on the internet and have no idea about you. So my response shouldn't be taken in that way.
What I can do is ask an important question. Are you able to confirm you are unique or are you just tricking yourself?
Either could be true, right? I mean we put mental defenses around us all the time. There's so much crazy shit we treat as mundane. I mean just a solar flare being the size of a dozen earths and we treat it like our little minds can comprehend that and make it mundane. We like to convince ourselves that we understand things much better than we actually do. It's normal, but recognition of it is important to help us continue moving forward, right? To question what we believe. Since our understanding is never perfect, there's always more, but our brains like to feel done as that feels accomplished.
It certainly does not help that we're working with tools that are trained for human preference. I use them too. I am an AI researcher even. But I think we also need to be careful. The same objective we use to train these systems to be accurate and sound natural in their speech also maximizes their ability to deceive us. It is a really tricky thing, so I think it is also something we need to be extremely cautious about. If we don't believe we could be the fool then that makes us the biggest fool of them all, right? So how do we find out if we've been fooled or not? I don't think that's so easy and I honestly don't have an answer. There's a lot of complexity to this as you dig in and... complexity is complex.
Yeah, I know what you mean. I do my best to check myself and also run things by people I trust, but there's an ever-present risk I'm going insane.
As a test, I've been attempting an incredibly complex project that goes far beyond my abilities as a kind of deliberate worst-case-scenario. It's more or less a programming language for a very specific purpose that compiles to a custom bytecode and runs on a custom runtime with specific performance guarantees.
I've spent part of the last month iterating on a formal model of the system and various specifications. Along the way, I teach myself how to understand and critique the part of the system I'm working on, however I also deliberately keep things just beyond my understanding by opportunistically pulling in concepts from various sources ... algebraic topology, obscure corners of PL, concepts plucked from similar systems. It's a complete monstrosity with, now, hundreds of supporting documents, research spikes, processed references, critique passes, etc.
If I'm able to complete this project and have it work as expected, I think I'll have learned a lot about what is or isn't possible. If the current design does in fact work, I'm fairly confident I'll have advanced the state of the art in the niche field I'm working in.
Yeah, you're absolutely right. I was just thinking yesterday ... that because the majority of reading I do now is output from chatbots, I'm starting to think and write like a chatbot.
A little terrifying. Probably the solution is to read 19th century literature before bed.
You think you know what taste is. Have you been cranking on real systems all these years, or have you been on the sidelines armchairing the theoretics? I'm not trying to come across as rude, but it may be unavoidable to some degree when indirect criticism becomes involved. A laboring engineer has precious little choice in the type of systems available on which to work on. Fundamentally, it's all going to be some variant of system to make money for someone else somehow, or system that burns money, but ensures necessary work gets done somehow. That's it. That's the extent of the optimization function as defined by capitalism. Taste, falls by the wayside, compared to whether or not you are in the context of the optimizers who matter, because they're at the center of the capital centralization machine making the primary decisions as to where it gets allocated, is all that matters these days. So you make what they want or you don't get paid. As an Arts person, you should understand that no matter how sublime the piece to the artist, a rumbling belly is all that currently awaits you if your taste does not align with the holders of the fattest purses to lighten. I'm not speaking from a place of contempt here; I have a Philosophy background, and reaching out as one individual of the Humanities to another. We've lost sight of the "why we do things" and let ourselves become enslaved by the balance sheets. The economy was supposed to serve the people, it's now the other way around. All we do is feed more bodies to the wood chipper. Until we wake up from that, not even the desperate hope in the matter of taste will save us. We'll just keep following the capital gradient until we end up selling the world from under ourselves because it's the only thing we have left, and there is only the usual suspects as buyers.
You seem to be saying two things. For me, the answer is: I've been somewhere in the middle—working on real projects, sure. I've been employed as a software developer in the past, and I've worked with startups and corporations. I've also worked in academia.
Have I spent years, personally grinding directly in the belly of the beast? No. I managed a small dev team in small startup once. Yeah, it's not the same thing. I know I don't know everything.
Yes, I'm familiar with the critiques of capitalism. I went to art school. Art school is like studying philosophy, but only the social critique parts (for better or worse).
Yes, I'm aware that I'm being ingested by machinery that serves capital. I've read Nick Land.
We're all doing our best to navigate this, but don't forget that poets, mathematicians, artists, and musicians really exist. They contact the cold realities of real life too, and many of them still succeed, still live beautiful lives. And no matter how bad things are, they still write history in the long-run.
I envy you to be honest. My muse doesn't bring me visions of sublime beauty. No melodies, landscapes, scenes or plays in the mind's eye. Just blueprints, system diagrams, feedback loops, crystalized reasoning, probable futures, things going wrong, breaking. Never in a way that seems to be communicable to galvanize more than a couple people who probably already know what I'm seeing deep down in their hearts. But... I'm trying. I flubbed the middle. The point was, no one at the bottom can unseat the people at the top or their optimization function from inside the system, playing by it's rules. Unless we all align on something to force a change; the system's stable & will remain so. There's just one outcome, and it ain't pretty for most of us. I can't even bring myself to throw my weight in as much as I used to, because I just can't not see the patterns everywhere I look now. Didn't mean to be rude. Just. Tired. So, so, very tired. The kind sleep won't fix. Whatever your art form... May you get good inspiration, and may the work to realize lighten your soul in the doing, and everyone's around you. Mine... Certainly doesn't.
Appreciate this very direct and real share. I pray that you find your way to a more merciful part of this life game, or that somehow all that you're going through now is worth it for something later.
So from my perspective as a professional programmer, my feeling is good on you, like, you're empowered to make things and you're making them. It reminds me of people making PHP sites when the web was young and it was easier to do things.
I think where I get really irritated with the discourse is when people find something that works for them, kinda, and they're like "WELL THIS IS WHAT EVERYONE HAS TO DO NOW!" I wouldn't care if I felt like "oh, just a rando on the internet has a bad opinion", the reason this subject bothers me is words do matter and when enough people are thoughtlessly on the hype train it starts creating a culture shift that creates a lot of harm. And eventually cooler heads prevail, but it can create a lot of problems in the meantime. (Look at the damage crypto did!)
Ok fella. But show me something then. This is all talk.
Personally I have been able to produce a very good output with Grok in relation to a video. However, it was insanely painful and very annoying to produce. In retrospect I would've much preferred to have hired humans.
Not to mention I used about 50 free-trial Grok accounts, so who knows what the costs involved were? Tens of thousands no doubt.
I know it's not anyone's fault exactly, but the current state of systems in general is an absolute shit show. If you care about what you do, I'd expect you to be cheering that we just might have an opportunity for a renaissance.
Moreover, this kind of thinking is incredibly backward. If you were better than me then, you can easily become much better than I'll ever be in the future.
I know I'm just adding to the noise here, but it seems like half the time I look at comments on posts (anywhere) I see a claim like this. I think you're probably just trying to warn people so they don't waste their time, but for me this type of comment is not helpful at all.
I had a very similar experience. I have different preferences, but ultimately, my takeaway was that if I want to follow my own version of their philosophy, I should just create my own thing.
In the meantime, the codex/cc defaults are better for me.
- A well-structured folder of markdown files that I constantly garden. Every sub-folder has a README. Every files has metadata in front-matter. I point new sessions at the entry point to this documentation. Constantly run agents that clean up dead references, update out of date information, etc. Build scripts that deterministically find broken links. It's an ongoing battle.
- A "continuation prompt" skill, that prompts the agent to collect all relevant context for another agent to continue
- Judicious usage of "memory"
- Structured systems made out of skills like GSD (Get Shit Done)
- Systems of "quality gate" hooks and test harnesses
For all of these, I have the agent set them up and manage them, but I've yet to find a context-management system that just works. I don't think we understand the "physics" of context management yet.
reply