I think this needs to be established instead of being assumed. There is plenty of knowledge out there in the world that can easily be accessed and combined to accomplish bad things. A motivated actor will find the information they need, regardless of whether ChatGPT helps them.
>A motivated actor will find the information they need, regardless of whether ChatGPT helps them.
This is true, of course, but ChatGPT still shouldn't do it. A motivated actor may find out how to make a bomb, but that doesn't mean HN should feature posts about the best bomb-making techniques and materials.
I was about to go and find an article about the best techniques and materials and submit it to see how it does. On the other hand I'll wait until I'm not at the airport.
I think this is the key here. The protections in place are good enough: someone motivated can jailbreak chatGPT or use TOR. At least in the former case it would be easier to detect and find him.
I think this needs to be established instead of being assumed. There is plenty of knowledge out there in the world that can easily be accessed and combined to accomplish bad things. A motivated actor will find the information they need, regardless of whether ChatGPT helps them.