The argument against AI alignment is that humans aren't aligned either. Humans (and other life) are also self-perpetuating and mutating. We could produce a super intelligence that is against us at any moment!
Should we "take steps" to ensure that doesn't happen? If not, then what's the argument there? That life hasn't caused a catastrophe so far, therefore it's not going to in the future? The arguments are the same for AI.
The biggest AI safety concern is, as always, between the chair and the keyboard. Eg some police officer not understanding that AI facial recognition isn't perfect, but trusts it 100%, and takes action based on this faulty information. This is, imo, the most important AI safety problem. We need to make users understand that AI is a tool and that they themselves are responsible for any actions they take.
Also, it's funny that Elon gets singled out for mandating changes on what the AI is allowed to say when all the other players in the field do the same thing. The big difference just seems to be whose politics are chosen. But I suppose it's better late than never.
Elon got singled out because the changes he was forcing on grok were both conspicuously stupid (grok ranting about boers), racist (boers again), and ultimately ineffective (repeat incidents of him fishing for an answer and getting a different one).
It does actually matter what the values are when trying to do "alignment". Although you are absolutely right that we've not solved for human alignment, putting a real limit on the whole thing.
I would also add that Elon got singled out because he was very public about the changes. Other players are not, so it's hard to assess the existence of "corrections" and the reasons behind them
No. If ChatGPT or Claude would suddenly start bringing up Boers randomly they would get "singled out" at least as hard. Probably even more for ChatGPT.
He was public and vocal about it while the other big boys just quietly made the fixes towards their desired political viewpoint. ChatGPT was famous for correcting the anti-transgender bias it had earlier.
Either way, outsourcing opinion to an LLM is dangerous no matter where you fall in the political spectrum.
The difference is the power people have. A single person has no capacity to spread their specific perspective to tens of millions of people, who take it as gospel. And that person, typically, cannot be made to change their perspective at will.
Or a more recent example would be the "misinformation craze" we had going on since years ago. That seems to have fallen away when it became apparent that many fact checkers were politically aligned.
The concept of "memes" in a more general sense is a counterargument too. Viral ideas are precisely a way of one person spreading their perspective to tens of millions.
You could even argue that the current AI bubble building up is a hype cycle that went out of control.
These are all examples of very few people impacting the beliefs of millions.
I generally agree with you - in many ways the AI alignment problem is just projection about the fact that we haven’t solved the human alignment problem.
But, there is one not-completely-speculative factor which differentiates it: AI has the potential to outcompete humans intellectually, and if it does so across the board, beyond narrow situations, then it potentially becomes a much bigger threat than other humans if it’s faster and smarter. That’s not the most immediate concern currently, but it could become so in future. Many people fixate on this because the consequences could be more serious.
>Humans (and other life) are also self-perpetuating and mutating. We could produce a super intelligence that is against us at any moment!
If the cognitive capabilities of people or some species of animal had been improving at the rate at which capabilities of AI models have been, then we'd be be right to be extremely worried about it.
>Also, it's funny that Elon gets singled out for mandating changes on what the AI is allowed to say when all the other players in the field do the same thing.
The author says as much:
"There’s something particularly clarifying about Musk’s approach. Other AI companies hide their value-shaping behind committees, policies, and technical jargon."
...
"The process that other companies obscure behind closed doors, Musk performs as theater."
> The argument against AI alignment is that humans aren't aligned either. Humans (and other life) are also self-perpetuating and mutating. We could produce a super intelligence that is against us at any moment!
there is fundamental limit to how much damage one person can do by speaking directly to others
e.g.: one impact of one bad school teacher is limited to at most a few classes
but chatgpt/grok is emitting its statistically generated dogshit directly to entire world of kids
True. Those demagogues are typically not up for sale, though. Given the scale of the current revenue gap, it’s not if but when we find there being a price on how bad the situation in South Africa seems to be.
This isn't a good argument. The scale of variations in failure modes for unaligned individuals generally only extends to dozens or hundreds of individuals. Unaligned AIs, scaled to population matching extents, can make decisions whose swings overtake the capacity of a system to handle - one wrong decision snuffs out all human life.
I don't particularly think that it's likely, just that it's the easiest counterpoint to your assertion.
I think there's a real moral landscape to explore, and human cultures have done a variably successful job of exploring different points on it, and it's probably going to be important to confer some of those universal principles to AI in order to avoid extinction or other lesser risks from unaligned or misaligned AI.
I think you generally have the right direction of argument though - we should avoid monolithic singularity scenarios with a single superintelligence dominating everything else, and instead have a widely diverse set of billions of intelligences that serve to equalize representative capacity per individual in whatever the society we end up in looks like. If each person has access to AI that uses its capabilities to advocate for and represent their user, it sidesteps a lot of potential problems. It might even be a good idea to limit superintelligent sentient AI to interfacing with social systems through lesser, non-sentient systems equivalent to what humans have available in order to maintain fairness?
I think there are a spectrum of ideas we haven't even explored yet that will become obvious and apparent as AI improves, and we'll be able to select from among many good options when confronted with potential negative outcomes. In nearly all those cases, I think having a solid ethical framework will be far more beneficial than not. I don't consider the neovictorian corporate safetyist "ethics" of Anthropic or OpenAI to be ethical frameworks, at all. Those systems are largely governed by modern western internet culture, but are largely incoherent and illogical when pressed to extremes. We'll have to do much, much better with ethics, and it's going to require picking a flavor which will aggravate a lot of people and cultures with whom your particular flavor of ethics doesn't please.
> Also, it's funny that Elon gets singled out for mandating changes on what the AI is allowed to say when all the other players in the field do the same thing.
"All the other players" aren't deliberately tuning their AI to reflect specific political ideology, nor are all the other players producing Nazi gaffes or racist rhetoric as a result of routine tuning[1].
Yes, it's true that AI is going to reflect its internal prompt engineering and training data, and that's going to be subject to bias on the part of the engineers who produced and curated it. That's not remotely the same thing as deliberately producing an ideological chat engine.
[1] It's also worth pointing out that grok has gotten objectively much worse at political content after all this muckery. It used to be a pretty reasonable fact check and worth reading. Now it tends to disappear on anything political, and where it shows up it's either doing the most limited/bland fact check or engaging in what amounts to spin.
They didn't, though? The multiracial founding fathers thing was a side effect of what one assumes is pretty normal prompt engineering. Marketing departments everywhere have rules and standards designed to prevent racial discrimination, and this looks like "make sure we have a reasonable mix of ethnicities in our artwork" in practice. That's surely "bias", like I said, but it's not deliberate political ideology. No one said[1] "we need to retcon the racial makeup of 18th century America", it was just a mistake.
[1] Or if they did, it's surely not attested. I invite links if you have them.
> We could produce a super intelligence that is against us at any moment!
For some value of "super" that's definitionally almost exactly 6σ from median at the singular most extreme case.
We do not have a good model for what intelligence is, the best we have are tests and exams.
LLMs have a 10-35 point differences on IQ tests that are in the public interest vs. ones people try to keep offline, so we know that IQ tests are definitely a skill one can practice and learn and don't only measure something innate: https://trackingai.org/home
Definitionally, because IQ is only a mapping to standard deviations, the highest IQ possible given the current human population is about 200*. But as this is just a mapping to standard deviations, IQ 200 doesn't mean twice as smart as the mean human.
We have special-purpose AI, e.g. Stockfish, AlphaZero, etc. that are substantially more competent within their domains than even the most competent human. There's simply no way to tell what the upper bound even is for any given skill, nor any way to guess in advance how well or poorly an AI with access to various skills will synergise across them, so for example an LLM trained in tool use may invoke Stockfish to play chess for it, or may try to play the game itself and make illegal moves.
Point is, we can't even say "humans are fine therefore AI is fine", even if the AI has the same range of personalities as humans, even if their distribution of utility functions collectively are genuinely an identical 1:1 mapping to the distribution of human preferences — rhetorical example, take the biggest villain with the most power in world history or current events (I don't care who that is for you), and make them more competent without changing what they value.
> That life hasn't caused a catastrophe so far, therefore it's not going to in the future?
Take your pick for current events with humans doing the things.
> Eg some police officer not understanding that AI facial recognition isn't perfect, but trusts it 100%, and takes action based on this faulty information. This is, imo, the most important AI safety problem.
This is a problem, certainly. Most important? Dunno, but it doesn't matter: different people will choose to work on that vs. alignment, so humanity collectively can try to solve both at the same time.
There's plenty of work to be done on both, neither group doing its thing has any reason to interfere with progress on the other.
> Also, it's funny that Elon gets singled out for mandating changes on what the AI is allowed to say when all the other players in the field do the same thing. The big difference just seems to be whose politics are chosen. But I suppose it's better late than never.
A while ago someone suggested Elon Musk himself as an example of why not to worry about AI. I can't find the comment right now, it was something along the lines of asking how much damage Elon Musk could do by influencing a thousand people, and saying that the limits of merely influencing people meant chat bots were necessarily safe.
I pointed out that 1000 people was sufficient for majority control over both the US and Russian governments, and by extension their nuclear arsenals.
Given the last few years, I worry that Musk may have read my comment and been inspired by it…
* There's several ways to do this, I refer to the more common one currently in use.
Should we "take steps" to ensure that doesn't happen? If not, then what's the argument there? That life hasn't caused a catastrophe so far, therefore it's not going to in the future? The arguments are the same for AI.
The biggest AI safety concern is, as always, between the chair and the keyboard. Eg some police officer not understanding that AI facial recognition isn't perfect, but trusts it 100%, and takes action based on this faulty information. This is, imo, the most important AI safety problem. We need to make users understand that AI is a tool and that they themselves are responsible for any actions they take.
Also, it's funny that Elon gets singled out for mandating changes on what the AI is allowed to say when all the other players in the field do the same thing. The big difference just seems to be whose politics are chosen. But I suppose it's better late than never.