I've spent a day fixing a feature my very junior colleague pushed, and which was merged by my should-have-known-better colleagues who have almost as much experience as me.
We have access to the same models. Why did her version suck and my features don't need fixing?
Did you use chatjimmy? It's somewhat terrifying to use when you think of the potential results with a better model.
Ok, real life example: I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless). What if it gave back the same excellent results, but instantaneously? Why, then, I certainly would become the bottleneck. So, quite possibly, my last work task would be to plug this agent directly into the ticket system where the domain experts input their feature requests. Maybe we still need 1 developer out of 100, to coordinate releases and all that (ok, say 1 out of 10).
But that's not taking things far enough: why do we need these domain experts at all? Our pitch is clear, and all software-enabled, though it took years to develop. We can just have the clients express their concerns to the AI, directly or indirectly. Have multiple lighting-fast agents with different roles (refactoring agent, new features agent, debugger agent, domain expert agent, etc.). So we fire everyone, maybe keep 1 product owner / devops to keep the trolls out. The cost is still probably 100 times less than it used to be (beyond the initial cost of acquisition of the magic machine or whatever).
But one of these clients, surely, will realize that these 10 years of manual and slowly-automated development can now be emulated in very, very little time. Why not just, say, take screenshots of the entire app and feed them into the magic machine? Why, this way, they could have the service for a tenth of the yearly cost, forever!
And then the economy implodes.
I'm not saying it's THE most likely version of things, I'm saying that at a certain level, quantity (or rather, speed) is a quality all its own. And this new quality might change the world. Let's hope it's for the better!
Errors compound, and making 1000 wrong decisions per hour, will not result in something useful. Maybe you‘ve tried setting up guardrails for good design or architecture at some point? I think it’s simply not possible to do that.
It would certainly be an accelerator for people who know exactly what they want. And it would remove multi tasking, which I‘d appreciate.
If your task has incremental rewards/feedback, you can push the "intelligence rate" simply by sampling the reward function faster. That's not fake, even if it not a substitute either.
This is the "dumber but honest person that works harder" phenomenon, vs "lazy genius".
That's a good way to put it, but still my experience is that worse code bases are non-linearly harder to maintain and improve in the future, software tends to break down without a good enough base.
Sure, in the future full rewrites and stuff like that will be just another "throw money at it" problem, but fundamentally software can get arbitrary complex and we barely know how to write large, maintainable code bases.
Nonetheless, I think testing (and maybe proofs) will have its long-awaited time to shine, as being the "reward function".
I totally agree with you on the first bit, but I also think that I am way better at deciding on how to refactor code bases than the LLM is.
Right now, I put models in low thinking mode during my refactors and hate waiting. I would much rather have a faster model that that maybe was slightly stupider, and I would wait far less long between prompts where it needs my valuable input.
Models that are dumb, but humble and fast, can be fine.
I don't have a great answer but you pose a great question.
Obviously a CTO is not going to walk away from the technology just because it's not good enough. That much more incentive for someone to create a powerful enough harness that can direct that power safely and productively. Like a nuclear core, we'll need to come up with the graphite rods and water tank. And if tokens are essentially free, why not, for every million tokens, spend 10x tokens on code review, testing, etc?
AI helps you but also your competition, and gets factored in by investors while customers can use it to find better deals. The whole market is different even if a company did nothing.
Whatever you can cheaply do with AI is not a moat, if there is profit in there there will be quick imitation and competition will eat away those profits.
Models can be replaced easily, harnesses & AI tools too. And if cloud inference gets too expensive there are local models keeping the cloud prices hard capped.
Speed will be one of the killer features once you get closer to instant speeds of 300ms. Just remember what changes were made possible simply by upgrading from ADSL to broadband.
If inference speed goes up, I can launch the same query 5 times, evaluate the best result and proceed from there. Of course, evaluation is also instant, so in seconds I can get a near perfect solution. Or maybe 10 and I can pick what I like the best.
AI is already fast enough that human is a bottleneck. Hell, typing speed became a bottleneck like it was never before.
I mean, if an agent can do half-decent work in less time than it takes the user to prompt them (and "user" in this context is a fast touch-typist like most programmers are), it's obvious it's not the agent that's the bottleneck anymore.
Because finishing someone else's (or something else's) "half decent work" to the point of "actually decent" becomes the bottleneck.
This has always been the case for human project management, and LLMs just aren't at that level yet.
It's more like everyone is speed running to how fast they can convince others that "half decent" is good enough. And for sure, newer models of LLM seem to be getting better at that.
> It's more like everyone is speed running to how fast they can convince others that "half decent" is good enough.
But that's what Agile is all about, isn't it? We've been speedrunning delivering increasingly smelly shit at increased velocity ever since SaaS became a thing, because ubiquitous Internet access is what allowed our industry to adopt the "lob feces over the fence for users to deal with" release model.
AI does speed that up, true (though since the market - and management - didn't catch up with it yet, we have a brief moment where we can use AI to increase quality while keeping usual delivery rate.)
I'm not sure inference speed is always the slowest thing for me right now. The agent is running tests, loading webpages, etc, which all take time. I don't know if a fast agent would speed things up in all cases.
> "The agent is running tests, loading webpages, etc, which all take time"
A frustrating vision of the future would be when we've been asking for faster loading lighter web pages for years and then companies start caring about it and improving it not for us humans but for LLMs.
It's already kind of that way with MCP servers popping up everywhere. The JIRA MCP server is like a couple orders of magnitude faster to work with than the website itself.
> I now spend most of my time, as a developer, waiting for the agent to do its thing (after careful prompting, I'm also thinking about work stuff, don't worry I'm not useless).
you need to launch 10-15 more terminals, who is waiting these days? :)
I tried it. I asked where Bruce Lee was born. It stated he was born in Hong Kong. I challenged it and it went further naming a hospital there. I stated he was born in San Francisco and it apologized and then said his father was a missionary traveling in America, which was also wrong. Bruce’s father was a famous Cantonese Opera singer and actor.
This model had zero information right, while being fast in responding.
It still works the same way other LLMs do, by outputting the probability distribution over the possible completions (The weather is ... (sunny (50%), cloudy (50%))). Then the next token is sampled from this probability distribution (in our example the next word could be "sunny" or "cloudy" equally likely), which can result in different outputs every run.
Could the model or algorithm be changed to make it deterministic somehow? It could help a lot if there were reproduceable outputs from deterministic baked-in silicon.
But since it's so fast you can just ask it 100 times where Bruce Lee was born, and statistically you'll get the correct answer. We could call it "mixture of idiots". /s
I just pasted your comment and its whole inheritance chain to it, started my comment, and asked to generate a total of 9 completions, 3 from each of {current & next word, current paragraph, current paragraph + rewrite the entire paragraph}.
Half of the answers were perfectly good (ironically, not the "next word" ones!), but the important bit, they came back near-instantly ("Generated in 0.024s - 14,163 tok/s", the page says). Slightly more powerful model while keeping this under a second, and this could easily become a qualitatively different form of autocomplete/text suggestion. Running in the background every couple keystrokes, or every time user stops typing for more than 500ms.
>I just pasted your comment and its whole inheritance chain to it,
Good idea. Only problem is it doesn't work. I just did the same thing with exactly this prompt:
>did the user IOT_Apprentice participate in the thread below and if, number and quote all of their comments. Only just number and quote the comments or write "Did not participate", do not add any commentary. Quote any comments by this user verbatim, exactly as input. Thread:
followed by pasting the thread[1]
And received the answer "IOT_Apprentice did not participate in the thread."[2] in 0.001s, even though they have literally the last comment in my quote and it's clearly legible.
It's particularly insidious because the understanding and thinking that is required to follow my requested answer format exactly is substantial - so based on the fact that it gets the format right and clearly understood the assignment, I would be inclined to believe that it would also be correct!
So to use your example, it's not just autocomplete, it's autocomplete that confidently returns "No matching results" in 0.001 seconds, even though there is a search term matching what you put in, right in the prompt itself that was sent to it. That is much worse than useless.
I think this reads like Ray Kurzwheil (sorry not able to spell that off top of my head, that bloke who wrote that book about the future) .. But yeah very dystopian and totally realistic. Not if but when..
Like politicians are going "without the homeless, I won't have anything to do anymore!". Like the street workers are thinking "shit, what'll I do if this goldmine dries up"?
"and all Canaanite dialects are mutually intelligible": That is the definition of a dialect.
Also, I don't know how you can claim Hebrew is phonetically represented by its alphabet rather than the other way around, as a revived language the pronunciations are largely a matter of convention based on Yiddish. It would be more accurate to say that modern Hebrew uses an ancient writing system, which happens to be closely related to the ancestor of modern European alphabets.
Hebrew is not based on Yiddish, lol; only Ashkenazi Hebrew pronunciation was influenced by Yiddish. Modern Israeli Hebrew uses primarily Sephardi pronunciation, and Ashkenazi is mocked (i.e. Shabbat is Sephardi, Shabbos is Ashkenazi; modern Israeli Hebrew uses Shabbat). I grew up around Ashkenazi pronunciation in America, and had to unlearn it when I spent time in Israel. Nonetheless, Yemenite, Sephardi, and Ashkenazi Hebrew — the three major extant pronunciations, only one of which was ever influenced by Yiddish (Ashkenazi) — are all extremely similar and mutually intelligible, and thus all of them are extremely well mapped to the alphabet. Yemenite is most likely closest to the original spoken language, specifically the ע, but there are very few differences. And a modern Hebrew speaker can easily understand Biblical Hebrew — they're closer than even Modern English and Shakespearean.
Also, not all colloquial dialects are mutually intelligible. Different Chinese dialects are still often referred to as "dialects," despite not being mutually intelligible (e.g. Cantonese vs Mandarin). While that's typically mostly the case for Western languages, there's a spectrum even there.
> And a modern Hebrew speaker can easily understand Biblical Hebrew — they're closer than even Modern English and Shakespearean.
Of course, because modern Hebrew was constructed based on (the modern understanding of) Biblical Hebrew around the 1920s or slightly earlier, whereas Modern English naturally evolved for ~400 years from Shakespearean English and other forms of English.
That’s simply incorrect. Most of the innovations in Modern Hebrew (relative to Biblical Hebrew) came in the Mishnaic period, early CE. Hebrew continued to be used as a liturgical language, and occasionally a business language, both in its Biblical and Mishnaic forms, until the 1880s (not 1920s), when the Zionist movement brought it back into use for casual speech. The Hebrew used in the Mishnah is quite close to the modern written language, though it lacks modern words and some very recent innovations like topic-first sentences.
Modern Hebrew is built on Biblical Hebrew, on Mishnaic Hebrew, on Medieval Hebrew, on Yiddish, and has influence from many other languages. However, there is nothing similar to how English and other native languages evolved. Between the 2nd or 3rd centuries CE and the 20th century, there were no native monolingual speakers of Hebrew - even Mishnaic Hebrew was used only as a second language; while Medieval Hebrew was a lingua franca that Jewish populations speaking other languages would use for communication, not a native language to any of them.
Also, while the creation of Modern Hebrew began around the 1880s, it was an extremely niche phenomenon until much later - at least according to Wikipedia, in 1900 there were fewer than 10 families even in Ottoman Palestine speaking Modern Hebrew currently.
No, there is no linguistic definition of a dialect. It’s a purely political term. Hindi and Urdu are “languages” despite being nearly identical in their spoken forms; Moroccan Arabic is a “dialect” even though Lebanese Arabic speakers can’t understand it; Galician and Portuguese are separate “languages,” with a mysteriously precise dividing line right at the Portuguese border!
Linguists elide over the whole thing by using the term “language variety.”
This is like speciation but for languages: there's no "ah-ha!" moment, but we know a lemur can't produce viable offsprings with a zebra. Likewise we know Italian isn't French even though some words are kinda similar. If you want to be technical about it, it's a spectrum: I understand British people and people from the American deep South, but it's far from certain they will understand each other. Hard to be precise with social sciences.
That said, two people who understand each other are, by any reasonable definition, speaking dialects of the same tongue (if not, obviously, the very same dialect).
It's been pretty obvious for a long time that Youtube doesn't want you to have an objective view of anything. It wants you to trust in the Algorithm to spoonfeed you content. Even the subscription page now displays some arbitrary shit first. I'm absolutely sick of it.
Thanks! I hope I didn't come off as too dismissive, I'm hearing a lot of good things about Jujutsu. As a developer though, I've never wanted to build from source (probably in the minority on that front).
Nah, you're right that installing a compiler toolchain to build a project is a pain in the butt if you don't already have it. It's a legitimate thing, but it does mean that you won't be adopting more cutting edge tools, which is also just fine! I've done the same with projects built with tools I don't have installed too.
Counter-point: static sites are unhackable, require no maintenance and are free to host. Now that I've migrated the website for a local (volunteer) event, they will never have to worry about their site breaking on them again.
Multiply that by every nonprofit without tech knowledge and that's a lot of potential Jekyll fans.
I'm not vouching against WordPress per se, just wish people would use the best tool for the job at hand rather than the one tool they know. Sometimes that's Jekyll and sometimes you need comments.
This is needlessly flippant and not really the same thing. Determining progress in a therapy setting is usually a collaborative effort between the therapist and the client. An LLM is not a reliable agent to make that determination.
> Determining progress in a therapy setting is usually a collaborative effort between the therapist and the client. An LLM is not a reliable agent to make that determination
Can anyone describe how to determine how a (professional, human) therapist is "a reliable agent" to make such a determination?
If you want to call into question the entire field of behavioral health and the training that is involved then that is fine, but if that’s how you feel then this entire discussion is really about something different and I can’t bridge the gap here.
I didn’t claim that an LLM is that, and I fully agree that it is not. I’m saying that one is inherently one’s own judge of whether one has a problem. You go to a therapist when you feel you have a problem that warrants it. You stop going when you feel you don’t have it anymore. And OP is very likely assessing their progress in the same way. I wasn’t being flippant if the parent was asking a genuine question.
> I’m saying that one is inherently one’s own judge of whether one has a problem. You go to a therapist when you feel you have a problem that warrants it
That is for certain types of therapy/clinical care. It is not always - and often isn’t - the case. Plenty of diagnoses and care protocols are not a matter of opinion or based on “you feeling there’s an issue” or deciding on your own there is no longer an issue.
I've spent a day fixing a feature my very junior colleague pushed, and which was merged by my should-have-known-better colleagues who have almost as much experience as me.
We have access to the same models. Why did her version suck and my features don't need fixing?
There's still something there, thankfully.