Of course you can generate text. You just need to run it in an autoregressive loop as a sort of reverse of all the fun Jev-like papers that have come out in the last couple days. Ask it to predict the next letter in a string, then sample at your favorite temperature, then predict the next letter, etc. This will be quite expensive, and it may work terribly. I’m not personally inclined to try it. I am, however, curious whether it would work less horribly if you correctly guess what tokenizer the input uses and request a choice over next tokens consistent with the tokenizer in question.
It would be absolutely hilarious if you did this, asked it which model it was, and it gave a recognizable answer that wasn’t Jev.
It seems to me that Jev cannot be usefully calibrated out of the box, in a fairly strict sense. Suppose I have a pull request and the state is the title, description, etc. The question is “Will it be merged?” (it doesn’t really matter whether it’s a choice or a “noul” [0]).
Now consider that two different projects may have radically different criteria for accepting a PR. And the two projects may have different probabilities for acceptance of a random PR from the distribution of PRs they get (i.e. the overall fraction of PRs that are accepted). So what could Jev possibly return that is “calibrated” for both? It doesn’t even have an “I have no idea” option because the output schema cannot distinguish between “I am confident that there is a 50% probability that the answer is yet conditioned on the state” and “there is no useful information contained in the state that I can extract and therefore you should assume that you posterior distribution is the same as your prior”.
For fun, I gave Jev some irrelevant state and asked it various questions for which the state was useless (I picked sporting outcomes), and it was 0-for-3 at giving yes/no probabilities that were particularly close to the obviously correct no-information answers or close to 0.5 in cases where the prior was far from 0.5.
This is a silly test, but I’ve personally encountered genuine production situations where the best classifier available (or at least the best one available at any cost remotely close to what it was worth) was, drumroll please, a constant. But it was a calibrated constant: we measured it! And there is no way to feed this sort of information to Jev. (Yes, I tried it. Even literally stating the distribution in the state does not work well, although it does appear to have some effect on the outputs.)
[0] Is “noul” even a word? I know what a binary classifier is…
Other animals do get heart cancer, which is fascinating. It's still very uncommon compared to other cancers, but not virtually absent as it is in people.
Uhh thanks for sharing I guess? The third paragraph just nukes the idea of divulging anything, it's obviously written entirely for specialists in the field (chromatin remodeling? the Nesprin-2 protein? mechanotransduction?). What I'm trying to say is no layperson would walk away from this article with a slight idea of why there's no cancer in the heart.
I’m fascinated by this thing and by the way it’s interpreting Jev. It’s very cool, but is it actually a classifier?
IIUC they took an already-trained “frozen” LLM and trained a little model on top that takes both a question and the hidden states after processing the input data and produces answer “probabilities”. (In contrast, the original LLM would have been run in AR mode to generate multiple output tokens representing its answer.) But then they used it for a purpose that isn’t really classification.
IMO there is a rather large difference between “is this email spam” and “what character should I type in this agentic workload”. The former is classification: there is hopefully a ground truth (is the email spam?) and the model is trying to classify the email. You would score it with a proper scoring rule. The latter is a strategy: there usually isn’t a correct answer, now or in the future. The model is playing a game consisting of repeated rounds, and the only way to evaluate it is to see how well it plays. You can’t even usefully compare it to the optimal solution because you may not know the optimal solution and you don’t actually need the model to produce an optimal solution.
I do think this approach is really cool, and it does suggest that one might be able to use a modern LLM to process an input and then extract the model’s next agentic step in a very fast, non-AR manner, with results comparably good to the usual AR decoding. And I think it’s very interesting to decouple the tokenized input representation from the model output representation, both because prefill tends to be faster and cheaper than AR output and because it’s never seemed particularly sensible to me that a model should be constrained to generate outputs at the cadence of one run through the model per output token. (AFAIK the main reason that models work on the same input and output token space is that this is how the pretraining process works.)
I wonder how to fit “reasoning” into this framework. Maybe have the question be something like “do you need to think further and, if so, what is your first thinking token”. But maybe something more clever is possible.
> The latter is a strategy: there usually isn’t a correct answer, now or in the future. The model is playing a game consisting of repeated rounds, and the only way to evaluate it is to see how well it plays
This is self contradictory. You can tell a correct answer as you said, by looking at the score. In my $DAYJOB I am making hundreds of RL environments that produce a score for each intermediate state.
Sure, you are scoring them. But I doubt that you are scoring them in the sense that you are minimizing loss where the loss is a proper scoring function that treats the model as a classifier where that treatment as a classifier actually means something. Unless you happen to be RLing a problem where there is either exactly one correct rollout or there is actually a bona fide natural distribution over rollouts independently from the model.
My environments don't score actions, they score state, so no matter how the agent chooses to solve a task it gets scored correctly. It's called Potential Based Reward Shaping and its main benefit is that it does not introduce reward hacking.
Right, and I’d add that in some games there is actually a “win” state that serves as a binary indicator. Given this, you can actually view the model as the thing being classified into two states: (1) consistently wins the game and (2) doesn’t consistently win the game.
I'm increasingly of the opinion that stuff like Jev and this are overfitting and producing illusory but confident "probabilities" that are complete bullshit. You can't see the underlying reasoning... but it's incredibly tempting for people who want to place faith in them. Rather than even trying to understand the complex system at play, it's easy to give up on trying to find reason, and just accept the second- or third-tier outputs of massively complex things that, on the first tier, are not necessarily reliable sources of truth.
Let's give an example: Suppose you ran a Jev that tried to determine when another Jev was wrong about something ...baseball games, let's say. The second Jev would come up with a perfect list of when the first Jev was wrong and when not to believe it. So now you have a second-order system that you believe more than the first-order one.
Only after 100 baseball games, the second-order Jev is only as good as the first one, it just inverted a bunch of games that could have gone either way. So hell, you make a third-order Jev that analyzes the first two...and its results are AMAZING when you look at the historical record! Only, you know, that's what's called overfitting.
Honestly, [edit: Fuck, I just wrote "honestly". I've been brain damaged by you-know-who] the invention of the "noul" is a bit of a giveaway. Imagine what junk bond traders could've done with that in the '80s. Not-not-not-falsy is how we all like our stock picks served up, right?
Reading the raw statistical output of an LLM as if it were an oracular source of truth is literally idol worship and gambling in the purest sense.
Take it from a guy who lost $10k on baseball this season having Claude rewrite my original code to reverse and re-reverse underdogs and favorites; there is no fucking "probability" of anything coming out of an LLM, even if your source of truth isn't an LLM but an evolutionary algorithm you designed yourself. Which would've performed better before the LLM started interpreting ways to bucket it and make up bullshit probabilities around it.
Anyone going down the Jev path is deeply misguided, but will see the light once they realize they have re-invented the magic 8 ball. Or that executive decision maker cube from the 80s with 6 random answers.
Here's your Jev. Note the preponderance of nouls ;)
If you happen to gamble or just watch baseball, you'll see that reality is not so easily reduced to one number. On the most basic level, a single probability number leaves out volatility, without which it's utterly useless for predicting anything other than a sequence it's already trained and fitted for. But as a gambling addict and occasional patron of oracles and gurus, it's very easy and tempting to mistake a clear-cut "probably" for whatever you want to hear. Listening to an LLM's heartbeat for probabilities is just a gambler's fallacy taken out to the 19th degree.
Yes, Jev is calibrated "from factory" on a bunch of tasks, but we can be almost sure our own bespoke tasks are not covered. So the model does not really know how to produce calibrated confidence scores.
What it would need is a calibration dataset on which to align. There is no calibration in the abstract, only relative to a set of test examples. A model with an uncalibrated output probability can be recalibrated using conformal prediction. You run the model over your calibration examples, get the probabilities.
Assume the new example's answer is y, and calculate its nonconformity score, higher means a worse fit. Count how many calibration examples have a score at least as high as that. Add one to this count, then divide by the total number of calibration examples plus one.
I can tell you though, I'm not alone... a completely degenerate cokehead at my local bar just showed me a baseball prediction app he had Claude build for him on his phone, which looked suspiciously similar to mine. (Mine might be hand coded and backed by a symbolic regression A-Life engine I've been working on since 2005, but so what? Claude has helped me flip most of the predictions for various reasons, to equal "success"). Crude, and yet his app somehow included wind direction forecasts for each inning of every game. This guy is smart as a punter but has probably never used a computer for anything in his life beyond downloading STD test results.
Check out the storied history of RM2. Taxpayers literally voted to raise sales taxes to fund a rail corridor, and the money was reallocated to a different county for an unrelated project that did not benefit the counties in question. Never mind that an actual functional Dumbarton Rail could be a huge improvement to life on the Peninsula if done well.
Imagine a genuinely fast transfer between any one of the Caltrain stations (say Palo Alto or Menlo Park) and one of the East Bay BART stations. Of course, getting the western portion past all the expensive neighborhoods in the way sounds like a real mess.
JWT uses a famously misdesigned signature scheme. It’s kind of amazing that this article uses JWT as an example of being better than SAML — SAML really is awful!
I’m morbidly curious whether the (supposedly) superior compaction support in recent GPT models with an appropriate harness has anything to do with this. A conventional LLM with conventional attention is, of course, wildly unsuitable to continuous tasks like driving, but maybe as the technology advances it will improve in its ability to sort-of work.
Not unless you want to cheat the attention mechanism or do extra computations running prefill in a front-truncated version of the conversation.
Also, to the extent that the model reasons and thus learns something, if you blindly truncate the front, you will lose that knowledge. In the OP, the LLM that actually navigated the course successfully only did it on the second try. It it forgot the first failed try, it might not have succeeded :)
Might I gently suggest that you have a model at least as capable as Opus 5.5 translate that from Claudish to English? Or, even better, have an actual human work on the docs a bit? As it stands, they are fairly egregious, and they seem to devote at least as much space to little AI-generated quips that convey no meaning than to actually explaining what’s going on.
Also, maybe a human should decide whether these are “function” hooks or “module” hooks. All of this marketing calls them “function” hooks, but the json config seems entirely unaware of this.
(Has anyone else noticed that half the sentences in Claudish aren’t merely weird: they are noun phrases and not sentences at all? I’m pretty sure that any decent pre-LLM NLP-based grammar checker would correctly flag half the sentences in Claudish. Also, whatever variant of Claude wrote this thing can’t even capitalize around semicolons consistently with itself, let alone consistently with how English has been written for at least a century.)
Ouch! Much of this verbiage was dictated by me personally; I just have a fairly distinct register some might consider inscrutable. My English teachers in grade school always said the same :)
Rest assured I'll inject a bit less soul into the official docs once Mods are launched; re your feedback on the JSON key, what would you recommend?
Well regardless of the origins of Claude-speak, I'm sure millions of humans are picking up some of its idioms, and at least some fraction of it will enter standard English in some decades
Out of curiosity, I ran the PDF through pangram and it seemed to flag nearly the entire thing as AI. I recognize these aren't perfect detectors, but they can give signals if nothing else. Perhaps it needs more soul injected!
I know it's a joke but if you look at many of the key AI researchers papers (written pre-LLM mass adoption) you can see some similarities with how LLMs structure their output.
Well, I've tried to get around it for our corporate SEO blogging strategy, and I've found that it's just not possible to do it (even with really crazy techniques where you have different models combined to produce the output, or you change your prompting style massively). The minute I take the same article that the LLM gave me and I rewrite it in my own voice (meaning just take the information and topics, but rewrite it from scratch), the post works immediately and passes as 100% human on Pangram.
The only thing that gets around that is StealthGPT, but it mangles your intent in the output so badly that it's easier to just write the thing yourself.
> Ouch! Much of this verbiage was dictated by me personally; I just have a fairly distinct register some might consider inscrutable.
I do notice that the more Claude-generated stuff I read, the more it creeps into my daily usage, and I suspect I don't read nearly as much of it as an Anthropic Employee would, so I could believe that. Though "much of" isn't the same as "written with zero AI generation/rephrasing/etc."
Anyway, that aside, something I would like to see is an example of something that's possible with this system that isn't, or isn't easily, possible with existing Claude hooks. Right now it reads like "Here's a cool thing! Here's how it works! And here's this other similar system we have! Here's how that works!" But there's no summary of the differences, why I'd use this over regular hooks, and so on. Or at least not that I saw on quick skim, and I'd be looking for that to be mentioned as early as possible, so I know if the rest is worth reading.
Your comment here is written in perfectly fine English :)
Maybe "function_hook_modules" for the JSON key?
Does this finally fix the limitation that there is no supportable way to replace the implementation of a native tool in Claude Code? codex-rs has an (undocumented but likely supportable if kind of awkward) mechanism, and I’m part way through a project that needs this. I was going to release it with a recommendation not to use Claude Code, but it’s at least plausible that this will change my mind. (I’m not sure yet — the $ mechanism might be too restrictive. It’s a start at least.)
Don't trust my formatting -- I typed this in the comment box and I have not run it through a prettifier or anything.
The point here is that there's a world adjacent to but not actually contained within Claude Code in which not everything blindly trusts everything else, and maybe it's time to start remembering some lessons from the 90's on. For example:
- Don't sniff content types and preferably don't infer them from filenames either.
- Don't trust your subresource to arrive intact. Pin those hashes and verify them!
- Leave some room for extensibility.
- Maybe don't require everyone to mix their code with the security policy that controls edits to that code? I realize that anyone cursed with GitHub Actions is forced to violate this rule in the most outrageous ways imaginable, but that doesn't mean that Claude Code needs to follow suit forever. Once there's subresource integrity, the actual subresource can be put somewhere else where it belongs better. If there's "source_file", in the future there can be "source_url" and such without trying to infer it from the formatting of the string.
(I've involved absolutely no AI in the preparation of this comment. I make plenty of use of AI, but I like my brain to stay exercised, and I like to think that I remain MUCH better at this kind of thing than even the best current models.)
Contact me at the email address in my HN account and we can arrange for a human with a soul and feelings to write the docs from scratch assuming you have a price in mind that values the nature of my talent and experience.
Your writing is fine, documentation is hard and nobody is perfect at launch. I can all but guarantee I have more editing experience than the commenter and trust me, you’re fine.
It's a register that would be great for some tabletop games or pop science writing, but in this context it could really use a technical editor to aggressively remove phrasing like "keeping the fold neatly uniform" and "affordance surface", as well as change things like "hooks live in a hooks.json file, today of four extant types" into plain technical language ("there are four types of hook that can be defined in hooks.json").
Writing documentation is one of the most often mentioned uses of LLMs. I suppose if Anthropic wouldn't be doing it it would put into question why anyone else would.
I’m curious… would you talk to them like this in person? The only ‘egregious’ thing I read here was your reaction. Maybe you could consider acting more human?
> they are noun phrases and not sentences at all? I’m pretty sure that any decent pre-LLM NLP-based grammar checker would correctly flag half the sentences in Claudish. Also, whatever variant of Claude wrote this thing can’t even capitalize around semicolons consistently with itself, let alone consistently with how English has been written for at least a century.
Probably just a case of a company hoping their scale can change the societal standard faster than they can be bothered to match the standard.
You'll talk like 2023 unsupervised TikTok generators and you'll be happy.
This is so many orders of magnitude above the “good” AQI threshold that it’s almost comical.
Now, sure, different particles have different health characteristics, but the stuff coming out of a vape isn’t water, and you’d better hope that it is many orders of magnitude less damaging than the fine particles that are found in the environment if you actually want to believe that your lungs are prepared to handle it.
(Okay, it’s obviously orders of magnitude less damaging than, say, wildfire smoke or brake dust or restaurant kitchen exhaust on a per-microgram basis, but the question is how many orders of magnitude less dangerous.)
P.S. I strongly suspect that there is one common dry aerosol that is genuinely harmless in moderate amounts: sodium chloride from sea spray. It’s highly soluble and found all over inside your body. I’m not sure to what extent air quality impact studies try to separate out the AQI impact from sea spray from everything else.
It would be absolutely hilarious if you did this, asked it which model it was, and it gave a recognizable answer that wasn’t Jev.
reply