Turns out OpenAI's llms are pretty decent at coding x86_64 bios bootloaders in assembly, but as soon as you go off script from the two main examples online, it falls apart really quickly, as it's crystal clear it has no idea what is actually going on or the limitations of how bootloaders (and 2, and 3 stage bootloaders) work.
A while back I wanted to learn more about how direct threaded Forth implementations work. Just explanations, not code. I went back and forth with Claude for a while, and I noticed that its ability to give me a coherent explanation was terrible. However, it was able to give me perfectly fine x86 assembler to implement portions of it -- I knew it was perfectly fine because it was reproducing code from jonesforth, which I had open in another tab.
Turns out OpenAI's llms are pretty decent at coding x86_64 bios bootloaders in assembly, but as soon as you go off script from the two main examples online, it falls apart really quickly, as it's crystal clear it has no idea what is actually going on or the limitations of how bootloaders (and 2, and 3 stage bootloaders) work.