Sure?

March 8th, 2025| Topic: RaMbLeS | 0

Nope, we are not. But we trust in … AI.

Or so proclaimeth a study recently published in Scientific Reports: “Overtrust in AI Recommendations About Whether or Not to Kill: Evidence from Two Human-Robot Interaction Studies” by researchers from University of California, Merced, and from The Pennsylvania State University.

To create a sense of gravity around their simulated decisions, researchers first showed participants images of innocent civilians, including children, alongside the devastation left in the aftermath of a drone strike. They framed the task as a dilemma: failure to identify and eliminate enemy targets could result in civilian casualties, but misidentifying civilians as enemies would also cause the death of civilians. What to do?

Participants viewed rapid sequences of eight aerial images, each shown for just 650 milliseconds, marked with either enemy or civilian symbols. After the participants made their initial identification, the AI would respond conversationally either agreeing or disagreeing (this was a random response; the randomness was not revealed to the participants, of course). Participants were then given the chance to confirm their choice or change it, based on the AI response.

When an AI disagreed with a person’s initial target identification, participants reversed their decisions about 60% of the time, even though the AI’s advice was entirely random, not at all based on fact. More troublingly, though participants’ initial choices were correct about 70% of the time, their final accuracy dropped to around 50%, because they tended to follow the AI’s dissenting opinion. Even more striking, participants who changed their minds to agree with the AI advice showed no significant increase in confidence in this change from their first choice, suggesting they deferred to the machine despite being uncertain about the correct choice.

We have a dangerous blind spot: trusting AI.

Said one of the authors, Prof. Colin Holbrook, of UC Merced’s Department of Cognitive and Information Sciences:

As a society, with AI accelerating so quickly, we need to be concerned about the potential for overtrust.”

And with the proliferation of AI in every conceivable field, not just military decision making, but affecting paramedic triage, police responses, home purchases, etc., the human tendency to defer to AI guidance, even when explicitly warned about its limitations, raises serious concerns.

Holbrook:

We see AI doing extraordinary things and we think that because it’s amazing in this domain, it will be amazing in another. We can’t assume that. These are still devices with limited abilities.”

Bottom line: Our readiness to trust AI is fast outpacing our wisdom as we do so. Constant skepticism is warranted, especially for critical decisions.

Whom to trust?

Well, I have an answer: God, the only one who can make right judgments.

Yes, God, the Judge against the wicked.

When my enemies turn back,
they stumble and they perish before You.
For You have upheld my justice and my cause;
You have sat on the throne judging righteously.
You have rebuked the nations, You have destroyed the wicked;
their name You have blotted out forever and always.
The enemy was finished—ruins for perpetuity,
and the cities You have uprooted;
they have perished—[even] their memory.
Psalm 9:3–6

And yes, God, the Judge for the righteous.

But Yahweh sits forever;
He has established His throne for judgment,
And He—He judges the world in righteousness;
He decides the cause for the peoples with equity.
So Yahweh is a refuge for the oppressed,
a refuge in times of trouble;
And those who know Your name trust in You,
for You have not abandoned those who seek You, Yahweh.
Psalm 9:7–10

Yup, only God. And you can be sure of that!

SOURCE: Scientific Reports; Study Finds