Side note, but I hate that we're moving to a world where coding costs a subscription. I fell in love with coding because I could take my dad's old Thinkpad, install Linux for free - fire up Emacs and start hacking without an internet connection.
It doesn't though. You can still code the old fashioned way, and you are even likely to become a better programmer for it.
Personally, I tried copilot when I got it for free as a student and it didnt make a difference. The reason I know is that I was coding on two devices, one which had copilot installed the other didnt, and I didnt care enough to install it on the latter through an entire semester.
Its just slightly better autocomplete, by a questionable standard of "better".
> There’s literally nothing an llm can write or tell you that you can’t write yourself or find in a manual somewhere.
That's like saying, there's literally nothing a service business can do for you that you can't do yourself. It's only true in a theoretical sense, if neither time nor resources are a constraint.
In such hypothetical universe, you don't need a dentist - you only need to spend 5+ years in medical school + whatever extra it takes to become proficient with tools dentists use + whatever money it takes to buy that equipment. You also don't need accountants, lawyers, hairdressers, or construction companies. You can always learn this stuff and do it yourself better!
Truth is, time and attention is finite. Meanwhile, SOTA LLMs are cheap as dirt, they can do pretty much anything that involves text, and do it at the level of a mediocre specialist - i.e. they're literally better than you at anything except the few things you happen to be experienced in. Not perfect, by no means error-free - just better than you. I feel this still hasn't sunk in for most people.
> No, it’s not. You’re making my statement abstract for the sake of arguing.
> I’m not a cook, doctor, or a lawyer. I can’t prepare meals for a party of more than 2.
They are demonstrating how over-broad your own statement was with an *equivalent* statement to show how it only passes on an unhelpful technicality.
Immediately after your quotation is this:
> you only need to spend 5+ years in medical school + whatever extra it takes to become proficient
LLMs pass the bar exam and the medical exam. These are things which I assume I would be able to do myself if only I were willing to dedicate 5 years of my life to each.
> I can write exactly all code an llm could write.
I can often see many errors in the code that ChatGPT produces. Within my domain, it's just a speed-up, a first draft I have to fix. Outside my domain, it knows what I *can't* Google because I've never heard the keyword that would allow me to.
On legal questions, ChatGPT (despite passing the bar exam) seems to make up cases. I belive this because I can google the cases and fail to find them. Is this because they don't exist, or because they're not indexed on Google? I don't have the legal background necessary to know — and it would take me years to get the knowledge necessary to differentiate "it's worse than first glance" from "it's better than second glance".
Even if LLMs were "a memorization machine" (they're bad at that), the statement is obviously false because actutal literal memorisation machines (books, video recorders, Google) cannot pass these exams.
LLMs only started to because they could follow the questions.
But even that aside, it doesn't matter why LLMs can do what they can do or what else can also do that, what matters is that it would take most humans several years to get to the level of current LLMs in a subject that human isn't already familiar with.
I don't know how you compare, but ChatGPT's English grammar and vocabulary are significantly better than mine. And when I prompt it appropriately, it also seems to be a better creative writer than I am, at least for short pieces.
> Chatgpt (and other LLMs) are awful at creative prose.
As are most humans.
Don't get me wrong, what I've seen from even the better LLMs have a certain voice and tropes and sacherine worldview that isn't dark enough where it needs to be for the story to work; but on the other hand, what I see on some fiction writing subreddits… the AI is often a genuine improvement over amateur writers, even in cases where the AI contradicts itself about plot elements.
Which is frustrating, because I have the feeling the novel I've been trying to finish writing for the last decade may be usurped by AI before I get my final draft.
> what I see on some fiction writing subreddits… the AI is often a genuine improvement over amateur writers
What point are you trying to make here? That amateur writers are amateurs?
That AI is only "often" an improvement over an amateur?
> Which is frustrating, because I have the feeling the novel I've been trying to finish writing for the last decade may be usurped by AI before I get my final draft.
This statement shows such a warped attitude towards art and the creative process. What do you mean "usurped?" Do you actually believe that LLMs will overtake humans when it comes to creative works?
If so, you don't really understand what is compelling about the written word or what makes for good writing and reading and it's no wonder you feel as though your own writing is so substandard.
I highly doubt your writing is that bad. Especially if you've been working on it for a decade.
Just so we're clear, you're not sure what I'm saying and nevertheless think that I'm a better writer than you believe that I believe myself to be and you are confused by an imaginary "only" that wasn't in what I wrote and that you are skeptical of the idea that an LLM might overtake humans in creative works generally and writing in particular?
I'm not sure if the following statement will help your confusion, but most who judge the quality of a story do so without being able to write that story. Critiquing and writing are different skills.
It's giving a reply that "sounds right to a human" in context. Which, on small scale, is exactly what they, me and you are also doing when writing (or speaking), except for the infrequent cases when we force ourselves to reason through stuff very. slowly.
(This is why I believe LLM performance is best judged against human inner voice/system 1 reasoning, not the entirety of human thinking. When thinking with system 1, people don't really have an idea what they're doing either - they're just doing stuff that feels right.)
Also note that "sounds right to a human" is literally the loss function on which LLMs are trained, so between heaps of training inputs and subsequent extensive RLHF, the process is by its very construction aiming optimizing for above-human-average performance across the board.
Even documented libraries can be a struggle, especially if they are not particularly popular. I'm doing a project with WiFi/LoRa/MQTT on an ESP32. The WiFi code was fairly decent, but the MQTT and especially LoRa library code was nearly useless.
Sonnet 3.5 fails to generate basic JetpackCompose libraries properties properly. Maybe if somebody tried really hard to scrape all the documentation and force feed it, then it could work. But i don't if there are examples of this.
Like general LLM, but with complete Android/Kotlin pushed into it to fix the synapses.
Of course, why wouldn't it? It's a generative model, not a lookup table. Show it the library headers, and it'll give you decent results.
Obviously, if the library or code using it weren't part of the training data, and you don't supply either in the context of your request, then it won't generate valid code for it. But that's not LLM's fault.
Even if you absolutely have to use an LLM for some reason, there are already perfectly good LLMs for code generation that you can comfortably run on commodity hardware.
For now. It’s not clear what the monetisation strategy is, but probably it will be paid in future (alongside whatever other strategies they may have, like selling data, etc)
I think you are approaching this with the wrong mindset. I see it as I'm paying somebody to type and document for me. If you treat LLMs like a power tool, it is very easy to do a cost benefit analysis.
That is what happens when developers want to be paid for their work, but refuse to pay for the tools they use, regardless of little they may cost.
So we're going back to the last century, but given we are in a different computing context, only the stuff that can be gated via digital stores, or Web Services, gets to have a way to force people to pay.
You are not alone. The only future for these sorts of AI coding helpers is for them to use 100% free software AIs. On the bright side good progress is being made in that area and the main sticking point seems to be the expensive hardware to run them on (and integration). Costs on that hardware will hopefully drop over time so they won't still be mostly limited to 1st world (like the subscriptions).
But I am glad we now have more paid options available. Tooling is important and people that do good work should be able to charge for high quality tools.
I would be much happier in a world full of tools licensed like Sublime Text, where I can purchase a license and just run it without the need to constantly phone home though.
Use an aggregator like nano-gpt.com. You get access to all the top models (including o1 pro which usually requires a $200/month subscription) on a pay per use rate. Short on cash ? Use deepseek models for .1cents.
"install Linux for free - fire up Emacs and start hacking without an internet connection." that still works you know. Nobody is forcing you down this subscription path (except Microsoft)
Just get a graphics card and run a prompt-compatible llm yourself. Recent models like phi-4 show decent results (relative to your general amazement baseline) even on medium quantization. I’m running q4_k_m (8gb) with custom “just print and stfu” characters and rarely reach Claude anymore.
> I fell in love with coding because I could take my dad's old Thinkpad, install Linux for free - fire up Emacs and start hacking without an internet connection.
I'm not understanding what it is about a private company launching a product that changes that?
>but I hate that we're moving to a world where coding costs a subscription
I mean you don't need to if you don't want to. I am gainfully employed as a software developer and what I do everyday is literally just fire up Emacs on my Linux machine and write code. To this day I haven't figured out what llms are supposed to do that a bunch of yasnippets don't.
Just like five years ago most of my day is reading and debugging code, I'm not limited by how fast I can type.
Understand probably not, but they can read bizarrely fast and write a summary of what things do very fast ; maybe not 100% accurately but close enough to be of value when trying to understand a large bag of code fast.
Was going to say it's mostly been great for me. At least until recently trying to try out deepseek r1 it's not been great. I can't tell if it's the router or the provider..
If you pay out of pocket it means it's not an approved tool by your company, which means you can be fired and sued for leaking their intellectual property.
Also 20$ per month is way less than what it costs them to run it. Eventually they will need to charge way more to cover their costs, and the people who can't code without an AI assistant will need to pony up :)
It likely doesn’t matter if it is for training in this case. I can’t send our codebase to a friend even if they would never look at it or use it.
The issue would be based on the terms of employment and the software license. There’s likely a provision that just says “don’t share” regardless of what the other party will use it for.
To OP’s point, if your company is paying for the sub, then sending the codebase data would be an approved use of the codebase as part of your job.
I do a lot more work with cline and aider than without. For now that translates to a lot more money. In the end it's probably just going to be part of your normal job; you will just deliver a lot more in your 40 hour week than you did before. At this stage in the AI hype, I would insist on more or code side projects or do multiple projects. I write a lot more code and still have more time than before sonnet came out.
If it improves your performance by 1%, then for a salary of $2000/mo, that $20/mo is breakeven.
If it benefits you more than 1%, then you're in profit.
Of course, if you're in a job that doesn't actually care about performance, and performing won't lead to better salary at some point, then it may not matter.
Only if it increases your compensation by that 1%?
I get making economic/stats based analysis like this, but is your boss going to notice that 1% to give you a raise they otherwise wouldn’t? Probably not…
Your company culture can be performance-minded and this still be true.
What a weird way of thinking. If you act so pedantic anyway, you can stop coding simply 1 percent earlier each day, since your boss would not notice anyway. Personally I live in Belgium and I would be glad to pay 20 euros per month if it increases my productivity, as noticed by my boss or not.
The real problem however is, I cannot simply share my employer's repos to be absorbed by any LLM out there. So I use only the tools that my employer provides and approves of. Currently that is Microsoft Copilot chat/RAG via my work account. It takes some copy/paste and adaptations of problems/solutions but it is much more efficient than using SO. It is also a great teacher that never gets tired of my plenty why/how questions.
In my view, the future is that LLMs can train on entire private code repos until it understands its ins and outs. Currently it would need to fit in the context window, hence you need to babyspoon it, as I understand things.
I was only embodying the mindset in the comment I was replying to. Thinking in terms like “My time is worth $X, so if I do Y thing that takes Z minutes, that’s D dollars saved.” It gets really hairy and leads people down false paths where they think they’re being super “rational” all the time.
In this case it tricks you because it assumes that the LLMs increase productivity and launders that into the calculation. For me, and many people, LLM usage decreases my productivity.
We're truly building walls everywhere.