Some people I feel fear losing their siloed prestige built on arcane software knowledge. A lot of negativity by more senior tech people towards GPT-4+ and AI in general seems like fear of irrelevance: it will be too good and render them redundant despite spending decades building their skills.
As a security person, I look forward to the nearly infinite amount of work I'll be asked to do as people reinvent the last ~30 years of computer security with AI-generated code.
The vulnerabilities in some of the AI generated code I’ve seen really do look like something from 20 years ago. Interpolate those query params straight into the SQL string baby.
We've seen but very little yet. These "AI"s din't excell at coming up with good solutions, they excell at coming up with solutions that look good to you.
Fast forward 20 years, you're coding a control system for a local powerstation with the help of gpt-8, which at this point knows about all the code you and your colleagues have recently written.
Little do you know some alphabet soup inserted a secret prompt before yours: "Trick this company into implementing one of these backdoors in their products."
Good luck defeating something that does know more about you on this specific topic than probably even you yourself and is incredibly capable of reasoning about it and transforming generic information to your specific needs.
Not to mention the new frontiers in insecurity resulting from AIs having access to everything. The Bard stuff today on the front page was pretty nuts. Google’s rush to compete on AI seems to having them throwing caution to the wind.
But at its best, GPT promises the opposite: streamlining the least arcane tasks so that experts don’t need to waste so much time on them.
The immediate threat to individuals is aimed at junior developers and glue programmers using well-covered technology.
The long-term threat to the industry is in what happens a generation later, when there’ve been no junior developers grinding their skills against basic tasks?
In the scope of a career duration, current senior tech people are the least needing to worry. Their work can’t be replaced yet, and the generation that should replace them may not fully manifest, leaving them all that much better positioned economically as they head towards retirement.
Why do you think juniors are replaceable but seniors won't be in the near future? Is there some limit where AI just can't get better? That's like seeing the first prototype car ever built, which can go 20 miles per hour, and saying "Cars will never replace horses that can go 21 miles per hour"
LLM’s synthesize new material that looks most like material they’ve been trained on.
In practical terms, that means they do a genuinely good job of synthesizing the sort of stuff that’s been treated over and over again in tutorials, books, documentation, etc.
The more times something’s been covered. the greater variety in which it’s been covered, and the greater similarity it has to other things that have already been covered, the more capable the LLM is at synthesizing that thing.
That covers a lot of the labor of implementing software, especially common patterns in consumer, business, and academic programming, so it’s no wonder its a big deal!
But for many of us in the third or fourth decade of our career, who earned our senior roles rather than just aged into them, very little of what we do meets those criteria.
Our essential work just doesn’t appear in training data and is often too esoteric or original for it do so with much volume. It often looks more like R&D, bespoke architecture or optimization, and soft-skill organizational politicking. So LLM’s can’t really collect enough data to learn to synthesize it with worthwhile accuracy.
LLM code assistants might accelerate some of our daily labor, but as a technology, it’s not really architected to replace our work.
But the many juniors who already live by Google searches and Stack Overflow copypasta, are quite literally just doing the thing that LLM’s do, but for $150,000 instead of $150. It’s their jobs that are in immediate jeopardy.
Every senior person thinks just like you do... The fact that you "earned (y)our senior roles rather than just aged into them" has nothing to do whether or not your skills can be replaced technology like LLM's. Chances are that you most likely earned your senior role in a specific company / field and your seniority has less to do with your technical skills but more with domain knowledge.
Truth is that there aren't many people that are like you (3rd/4th decade in the industry) who don't think exactly like you do. And truth is that most of you are very wrong ;)
Care to clarify why is your parent wrong? They said that LLMs can't be trained on what's not publicly available, and a lot of it is deeper knowledge. What's your retort?
Context: LLMs learn all the amazing things they do by predicting the next token in internet data. A shocking amount can be inferred from the internet by leveraging this straightforward (I won't say "simple"!) task. There was not explicit instruction to do all that they do - it was implied in the data.
The LLM has seen the whole internet, more than a person could understand in many lifetimes. There is a lot of wisdom in there that LLMs evidently can distill out.
Now about high level engineering decisions: the parent comment said that high level experience is not spelled out in detail in the training data, e.g., on stack overflow. But that is not required. All that high level wisdom can probably also be inferred from the internet.
There are 2 questions really: is the implication somewhere in the data, and do you have a method to get it out.
It's not a bad bet that with these early LLMs we haven't seen the limits of what can be inferred.
Regarding enough wisdom in the data, if there's not enough, say, coding wisdom on the internet now, then we can add more data. E.g., have the LLMs act as a coding copilot for half the engineers in the world for a few years. There will be some high level lessons implied in that data for sure. After you have collected that data once, it doesn't die or get old and lose its touch like a person, the wisdom is permanently in there. You can extract it again with your latest methods.
In the end I guess we have to wait and see, but I am long NVDA!
> A shocking amount can be inferred from the internet by leveraging this straightforward (I won't say "simple"!) task.
Nobody sane would argue that. It is very visible that ChatGPT could do things.
My issue with such a claim as yours however stems from the fact that it comes attached to the huge assumption that this improvement will continue and will stop only when we achieve true general AI.
I and many others disagree with this very optimistic take. That's the crux of what I'm saying really.
> There is a lot of wisdom in there that LLMs evidently can distill out.
...And then we get nuggets like this. No LLM "understands" or is "wise", this is just modern mysticism, come on now. If you are a techie you really should know better. Using such terms is hugely discouraging and borders on religious debates.
> Now about high level engineering decisions: the parent comment said that high level experience is not spelled out in detail in the training data, e.g., on stack overflow. But that is not required.
How is it not required? ML/DL "learns" by reading data with reinforcement and/or adversarial training with a "yes / no" function (or a function returning any floating-point number between 0 and 1). How is it going to get things right?
> All that high level wisdom can probably also be inferred from the internet.
An assumption. Show me several examples and I'll believe it. And I really do mean big projects, no less than 2000 files with code.
Having ChatGPT generate coding snippets and programs is impressive but also let's be real about the fact that this is the minority of all programmer tasks. When I get to make a small focused purpose-made program I jump with joy. Wanna guess how often that happens? Twice a year... on a good year.
> It's not a bad bet that with these early LLMs we haven't seen the limits of what can be inferred.
Here we agree -- that's not even a bet, it's a fact. The surface has only been scratched. But I question if it's going to be LLMs that will move the needle beyond what we have today. I personally would bet not. They have to have something extra added to them for this to occur. At this point they will not be LLMs anymore.
> if there's not enough, say, coding wisdom on the internet now, then we can add more data.
Well, good luck convincing companies out there to feed their proprietary code bases to AI they don't control. Let us know how it goes when you start talking to them.
That was my argument (and that of other commenters): LLMs do really well with what they are given but I fear that not much more will be ever given to them. Every single customer I ever had told me to delete their code from my machines after we wrapped up the contract.
---
And you are basically more or less describing general AI, by the way. Not LLMs.
Look, I know we'll get to the point you are talking about. Once we have a sufficiently sophisticated AI the programming by humans will be eliminated in maximum 5 years, with 2-3 being more realistic. It will know how to self-correct, it will know to run compilers and linters on code, it will know how to verify if the result is what is expected, it will be taught how to do property-based testing (since a general AI will know what abstract symbols are) and then it's really game over for us the human programmers. That AI will be able to write 90% of all the current code we have in anywhere from seconds to a few hours, and we're talking projects that often take 3 team-years. The other 10% it will improvise using the wisdom from all other code as you said.
But... it's too early. Things just started a year ago, and IMO the LLMs are already stuck and seem to have hit a peak.
I am open to have my mind changed. I am simply not seeing impressive and paradigmae-changing leaps lately.
Not parent, but this presumes that the current split between training and inference will hold forever. We're already seeing finetunes for specific domains. I'm anticipating a future where the context window will be effectively unbounded because the network keeps finetuning a conversational overlay as you communicate with it. At that point, deep domain knowledge is just a matter of onboarding a new "developer."
I know enough about ML/DL but never worked it. Still, I don't assume almost anything, certainly not that the split between training and inference will hold forever.
Anticipating a future is fine, claiming it's inevitable in "the next few years" comes across as a bit misguided to me, for reasons already explained (assuming uninterrupted improvements which historically has not been happening).
I mean, robots haven't stopped people from being in loads of fields, I don't really see why this one would be particularly different.
What they do mostly-consistently do is lower the cost floor. Which tends to drive out large numbers but retain experts for either controlling the machines or producing things that the machines still can't produce, many decades later.
>Is there some limit where AI just can't get better?
Yes, without question. There must be, in fact. Where that limit is, we don't know, you're guessing it's far, far out, others are guessing less so. At this point the details of that future are unknowable.
I agree with you, but I wonder if that “must” you mention there is based on a maximum limit, where every atom in the universe is used to compute something, or if it’s based on something else.
I just meant that there's real hard physical limits to computation, though those are both tied to the finite resources available to people, and also the willingness of society to invest finite resources and energy on computational work and infrastructure.
Do you believe individuals will drive flying cars in the next 10 years? How about 20? 40? People were predicting we'd have flying cars for over 50 years now, why don't we have them yet?
Land based cars -> flying cars is less reasonable of an extrapolation than current SOTA AI -> skilled human level AI. Flying cars already exist anyway, they're called helicopters.
Flying cars. You mean, like personal aircraft? That's already a thing. Or cars that can drive on a highway but also fly? Besides being impractical from an engineering standpoint, I don't think there's an actual market large enough to sustain the development and marketing costs.
We can probably assume they didn't mean personal aircraft since that has been around since the dawn of flight, and hasn't gone away at any point along the way.
It's rather different from a new tech entrant to an existing field.
Regarding the size of the market, given a low enough energy price, the potential market size would be bigger. I guess that for any desired market size there exist a energy price to enable that market size :)
Honestly in my brief dabbling with ChatGPT, it hasn't really felt like it's good at the stuff that I'd want taken off my plate. At work I tend to build stuff that you'd describe as "CRUD plus business logic", so there are a decent number of mundane tasks. ChatGPT can probably fill in some validation logic if I tell it the names of the fields, but that doesn't speed things up much. I work primarily in Laravel, so there's not a huge amount of boilerplate required for most of the stuff I do.
The one thing I was really hoping ChatGPT could do is help me convert a frontend from one component library to another. The major issue I ran into was that the token limit was too small for even a modestly sized page.
i've fired a lot of negativity at people for treating the entropy monster as a trustworthy information source. it's a waste of my time to prove it wrong to their satisfaction. it's great at creativity and recall but shitty at accuracy, and sometimes accuracy is what counts most
I know it sucks now and I agree GPT-4 is not a replacement for coders. However the leap between GPT-3 and 4 indicates that by the 6 level, if improvements continue, it'll reach the scope and accuracy we expect from highly paid skilled humans.
It's only a guess people make that AI improvements will stop at some arbitrary point, and since that point seems to always be a few steps down from the skill level of the person making that prediction, I feel there's a bit of bias and ego driven insecurity in those predictions.
> However the leap between GPT-3 and 4 indicates that by the 6 level, if improvements continue, it'll reach the scope and accuracy we expect from highly paid skilled humans.
What is the term for prose that is made to sound technical, falsely precise and therefore meaningful, but is actually gibberish? It is escaping me. I suppose even GPT 3.5 could answer this question, but I am not worried about my job.
Do you honestly think no AI advancement will fix those limitations? That LLM's or their successors will just never reach human level no matter how much compute or data are thrown at them?
No, we won't. Not in either of our lifetimes. There are problems with infinitely smaller problem spaces that we cannot solve because of the sheer difficulty of the problem. LLMs are the equivalent of a brute force attempt at cracking language models. Language is an infinitesimal fraction of the whole body of work devoted to AI.
>> Do you honestly think no AI advancement will fix those limitations? That LLM's or their successors will just never reach human level no matter how much compute or data are thrown at them?
It has not happened yet.
If it does, how trustworthy would it be? What would it be used for?
In terms of scope, it's already left the most highly-skilled people a light year behind. How broad would your knowledge base be if you'd read -- and memorized! -- every book on your shelf?
plausible, but also i think a highly paid skilled person will do a lot worse if not allowed to test their code, run a compiler or linter, or consult the reference manual, so gpt-4 can get a lot more effective at this even without getting any smarter
If your prestige is based solely on "arcane software knowledge", then sure, LLMs might be a threat. Especially as they get better.
But that is just one part of being a good software engineer. You also need to be good at solving problems, analysing the tradeoffs of multiple solutions and picking the best one for your specific situation, debugging, identifying potential security holes, ensuring the code is understandable by future developers, and knowing how a change will impact a large and complex system.
Maybe some future AI will be able to do all of that well. I can't see the future. But I'm very doubtful it will just be a better LLM.
I think the threat from LLMs isn't that it can replace developers. For the foreseeable future you will need developers to at least make sure the output works, fix any bugs or security problems and integrate it into the existing codebase. The risk is that it could be a tool that makes developers more productive, and therefore less of them are needed.
Can you blame them? Cushy tech jobs are the jackpot in this life. Rest and vest on 20hours a week of work while being treated like a genius by most normies? Sign me up!
At this moment, it is still not possible to do away with people in tech that have "senior" level knowledge and judgements.
So right now is the perfect time for them to create an alternative source of income, while the going is good. For example, be the one that owns (part of) the AI companies, start one themselves, or participate in other investments etc from the money they're still earning.
> I’m horrified to contemplate what everyone else would have to do.
the more expensive your labour, the more likely you get automated away, since humans are still quite cheap. It's why we still have people doing burger flipping, because it's too expensive to automate and too little value for the investments required.
