I've had a ceiling fall on me once and once to a friend while on vacation. Just because it hasn't happened to you doesn't mean it hasn't happened to other people.
They're only sharing an annecdote because they are responding to your annecdote about not seeing a ceiling collapse.
> I don't think it changes the point of the metaphor.
If their anecdotes is moot, than your anecdote is also moot; if the anecdotes can only confirm a conclusion and never disconfirm, then we've created an unfalsifiable construction with the conclusion baked into it's premises.
Sure, I suppose that's something that someone who doesn't understand the discussion might say.
A person who better comprehends what they read might properly contextualize within the larger conversation, where the point that stands is that LLMs and ceilings are both useful, neither are doomed such that no one should use them, and that individual instances of failures are somewhat uncommon and not a reason for others to avoid the category.
> Sure, I suppose that's something that someone who doesn't understand the discussion might say.
I'm going to be frank, you are the person who misunderstands (and are being rather rude about it). You are responding to an argument no one is making.
To put a fine point on it, you said this:
> Entropy may mean all ceilings collapse eventually, but that doesn't mean we aren't able to make useful ceilings.
But you were responding to a comment saying this:
> Except your ceiling can and will fall on you unless you take preventative measures, entirely due to molecular interactions within the material.
Emphasis added. They are saying maintenance is necessary, not that a safe ceiling is unachievable. It's obviously achievable, we've all seen it achieved.
They further say:
> It boggles the mind to let an LLM have access to a production database without having explicit preventative measures and contingency plans for it deleting it.
Emphasis added. When they say it boggles the mind to deploy an LLM without the proper measures, the implication is that it does make sense to deploy it with the proper measures.
> ...the point that stands is that LLMs and ceilings are both useful, neither are doomed such that no one should use them, ...
I have not seen a single person in this subthread say that LLMs aren't useful or that they are doomed. People say that. But the people you're talking to haven't.
I try to avoid these petty "I brought the receipts" comments, but I don't like the way you're being snarky to people who's crime is engaging with the premises you set up. The faults you are finding are faults you introduced. I'd appreciate if you would avoid that in the future.
If that's what you got out of the above conversation that is about as fundamental a misunderstanding as the one at the top of this thread saying "It is fundamental to language modeling that every sequence of tokens is possible". I could say something rude here about both mistakes being made by the same person, but since you brought it up I won't.
If you want to take a comb to it, the comment saying this:
> Except your ceiling can and will fall on you unless you take preventative measures, entirely due to molecular interactions within the material
Was already off the plot. What was being discussed wasn't some specific molecular process, it was the false premise "oh molecules move around randomly so your ceiling might just collapse of its own accord because the beam decided to randomly disintegrate". That's not something that happens.
You said "The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use". This is analogous to "the ceiling could just collapse on you due to random molecular motion, no matter how much maintenance you do or what materials you use".
Make sense now?
Your edit at the bottom of your top comment does better than your original statement.
> What was being discussed wasn't some specific molecular process, it was the false premise "oh molecules move around randomly so your ceiling might just collapse of its own accord because the beam decided to randomly disintegrate". That's not something that happens.
Except it does happens. That’s why buildings get condemned and buildings eventually turn to rubble.
To the exact point; I have a product from a couple years ago using an old model from OpenAI. It’s still running and all it does is write a personality report based on scores from the test. I can’t update the model without seriously rewriting the entire prompt system, but the model has degraded over the years as well. Ergo, my product has degraded of its own accord and there is nearly nothing I can do about it. My only choice is to basically finagle newer models into giving the correct output; but they hallucinate at much higher rates than older models.
> I could say something rude here about both mistakes being made by the same person, but since you brought it up I won't.
I'd encourage to desist from rudeness, not just when people point it out to you, but at all times.
> You said "The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use". This is analogous to "the ceiling could just collapse on you due to random molecular motion, no matter how much maintenance you do or what materials you use".
If prompt engineering is effective (analogous to performing the necessary maintenance and selecting the correct materials), I'm curious what your explanation is for the incident in the article?
> I'd encourage to desist from rudeness, not just when people point it out to you, but at all times.
I desire neither to be inauthentic, nor to suppress my emotions.
> If prompt engineering is effective (analogous to performing the necessary maintenance and selecting the correct materials), I'm curious what your explanation is for the incident in the article?
Keeping with the analogies, the original article doesn't say whether they built the roof properly or if the just used some screws to hold up a piece of quarter inch plywood and called it a day.
It's no surprise that a terribly built roof may fall down. It's possible to get shoddy materials from a supplier without knowing.
Calling a curl command isn't something that would be within the model's training as "this deletes things don't do it". The fact that this happened is not, to me, evidence that the model might have equally run `sudo rm -rf --no-preserve-root /` under similar circumstances.
It sounds like the phrase "NEVER FUCKING GUESS!" was in the prompt as well, which could easily encourage the model towards "be sure of yourself, take action" instead of the "verify" that was meant.
As mentioned elsewhere in this thread, the fact that the article focuses so strongly on "the model confessed! It admitted it did the wrong thing!" doesn't lead me to put a ton of stock into the capability of the author to be cautious.
Yeah, also seems very US-centric with "outstanding shares". Other countries don't allow you to have outstanding shares. Also, where is the "reinvest" option during series rounds. Like, what founder also doesn't reinvest to keep their equity during fund raising?
This is US centric, it's mentioned on the first page, along with the notes and links.
Reinvesting is a minority, edge case. I can't think of a single VC company in the Bay area that can have founders reinvest shares and not cause issues raising.
"...reinvest shares." I don't understand this. I'm guessing it's my lack of knowledge.
I'd expect "reinvest" to mean "supply more capital to obtain more shares" in an attempt to maintain ownership percentage. The founder would have to already be rather wealthy, or be savvy enough to negotiate some other way to maintain their ownership.
You can borrow against your shares if you have a good relationship with the bank. That will allow you to be wealthy enough to buy some shares in the round. Not enough that you won't get diluted, but not as much as being broke. And of course, you have to pay back the loan.
I didn’t read tfa, but can we also have it be able to distinguish when a vulnerability doesn’t apply? As an open source contributor, people open nonsensical security issues all the time. It’s getting annoying.
Sounds like what my teachers used to say: “a personal problem”. Literally nobody outside FB knows what they’re missing and until they fix that, literally nobody cares.
reply