> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer?
If you use just any amount of critical thinking, yes. Truth and objectivity are ideals, not practical states. LLMs are a very bad way to come close to this ideal. You may use them as a search interface to give you sources and then examine the sources, but the output directly is a strict degeneration over primary or secondary sources that you judge critically.
> LLMs are a very bad way to come close to this ideal...the output directly is a strict degeneration
I didn't understand the second part but regarding the first...
For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
Some may consider some subset of these as reputable sources. But in my mind, the same faculties of skepticism, cynicism, distrust, and benefit-of-the-doubt calculus are activated for all of them, including LLM outputs.
So that's one possible answer to your question.
But I suggest communicating this through simple illustrative examples to help your target audience understand the problem.
Abstract terms like primary sources, secondary sources, reputable sources, objective truth, strict degeneration, etc. may not help, especially if they have time or other constraints that make frequent critical examination of sources impractical.
> For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
LLM just distils information from those sources and is therefore always a second hand source at best, and a liar at worst. Humans can collect real world data and write about their findings, LLM cannot do that, that makes LLM strictly worse than the best human sources.
The nature of the source, whether primary or secondary or tertiary, does not automatically imply anything about their biases. Even a primary research paper on a novel topic may be biased or wrong.
I agree that LLMs can't collect real world data and write about their findings. But that's true about most human sources too, isn't it? Except primary novel researchers or investigations or philosophies, what is original? Most human-written information is also secondary or lower.
The "best human sources" does not imply "ALL human sources."
> For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.
With all due respect (not trying to be offensive at all) but this is insane to me.
All those sources of information you cited have a million incentives to provide fairly correct and checked information. But probably more importantly, they have even more incentives to NOT provide false information. At a minimum, their careers, reputation, recurring work, brand, etc... is on the line.
An LLM has zero incentives to provide you with true information, beyond a couple of md files with instructions. If it gets it wrong, there is zero accountability, just an -oh, you're absolutely right- response and move on.
I agree there is a lot of human bias in the world, but surely we can't even put in the same order of magnitude both types of biases!
The mere fact that editing is applied to newspapers, documentaries, or Wikipedia does not imply they become closer to the objective truth or free of omissions after the edits. Indeed, the edits may go the other way to align with vested business or political biases or personal fears of the editors or their management.
As for research papers, I agree that the peer review process makes them more much more self-correcting toward the objective truth, compared to the other formats. Nonetheless, it's well-known that academic research is far from perfect due to publication pressures, funding/grants, reproducibility crises, various biases (for example, political pressure in humanities fields).
Needless condescension and wrong assumption. No wonder so many people and students nowadays prefer answers (and even counseling) from LLMs instead of other people.
> but the absence of an editor or peer reviewer also does not make it better.
I don't see how disputing up the claim that LLMs and books, newspapers, etc are equitable sources of information is a strawman. I look forward to your erudite vernacular which will not deal with the substance of my comment.
If you use just any amount of critical thinking, yes. Truth and objectivity are ideals, not practical states. LLMs are a very bad way to come close to this ideal. You may use them as a search interface to give you sources and then examine the sources, but the output directly is a strict degeneration over primary or secondary sources that you judge critically.