How to Spot a Scam

The transcript below may be for an earlier version of this episode. Our transcripts are provided by various partners and may contain errors or deviate slightly from the audio.

Shankar Vedantam:

This is Hidden Brain. I'm Shankar Vedantam. Nearly half a century ago, volunteers in a psychological experiment were shown a slideshow of a small red car getting into a crash. Some volunteers were asked, how fast was the car traveling when it passed the yield sign? There was a catch. The original slides showed the car passing a stop sign, not a yield sign. Psychologist Elizabeth Loftus and her colleagues later asked volunteers to identify images they had seen before. Some volunteers were shown the earlier slide with a stop sign. Others saw an edited slide where the stop sign was replaced by a yield sign.

Volunteers who had been asked the question about the yield sign mistakenly remembered seeing the picture with the yield sign. In the intervening decades, hundreds of experiments have shown that what we think we see is really an amalgam of what is out there, how we feel about what we see, our conversations with others about what we see and so on. Attention, memory, and emotion all play a role in shaping what we notice and what we fail to observe. In the late 1990s, the psychologists Dan Simons and Christopher Chabris devised an experiment that produced a nearly unbelievable result. They had volunteers watch a group of people passing basketballs. Volunteers were told to count how many times the ball was passed between members of a group. It was tricky, but if you focused, you could do it. But the researchers were after something more interesting than counting basketball passes in a video.

Dan Simons:

A person wearing a full gorilla suit walks into the middle of the video, turns to face the camera, thumps its chest, and then walks off the other side about nine seconds later.

Shankar Vedantam:

The gorilla seems impossible to miss. It's a gorilla. But as psychologist Dan Simons points out, the volunteers were not looking for gorillas. They were intently counting basketball passes.

Dan Simons:

What we find is about half the people who view this video didn't notice the gorilla, and when you ask them about it, they're shocked that they could have possibly missed something so obvious.

Shankar Vedantam:

This week on Hidden Brain, why we miss things hiding in plain sight and how grifters and cheats take advantage of our blind spots. We'll also explore how to keep from falling prey to scams.

In daily life, all of us deal with a barrage of emails, texts, and calls. Most are from coworkers, friends, and family. But every so often, someone reaches out with a more sinister agenda. At the University of Illinois Urbana-Champaign psychologist Dan Simons studies how our perception, attention, and memory are fallible and vulnerable to exploitation. Dan Simons, welcome to Hidden Brain.

Dan Simons:

Thanks for having me on.

Shankar Vedantam:

I want to play you a clip from a story that appeared some time ago on NPR. Take a listen to the clip.

NPR clip:

You were in an orchestra that played music on tour 2002 to 2006, but the music you played was not what The audience heard.

That's right. I am a pretty good amateur high school violinist, but when I performed for this orchestra, the microphone in front of me was off, and a CD recording of a much more talented violinist was being blasted towards unsuspecting audiences.

Shankar Vedantam:

Dan, what is this story about?

Dan Simons:

Well, this is a story that's based on a memoir from Jessica Hindman, who's a creative writing professor and also an amateur, quasi-professional violinist that we first read about in an article that was in The Guardian, a newspaper that talked about ... I Faked in a World-class Orchestra was essentially the headline of the article. And in the original story in The Guardian, she talked about how she performed as part of this ensemble that would travel around, and it was performed with somebody who was a famous composer and she would pretend to play the violin when in reality the music that was being played was a CD being played over a set of speakers.

Shankar Vedantam:

I want you to stop a moment and imagine the scene. An amateur violinist somehow infiltrates a world-class orchestra. In front of live audiences, she pretends to play the works of a famous composer while music from a CD blasts out on speakers. Picture the scene in your mind. Where is this happening? What would you see if you were in the audience? What would it take for you to notice the fake bow syncing? Dan was fascinated by the story in part because of his interest in scams, but there was also something about it that didn't seem to add up.

Dan Simons:

My first thought when I read that was no way. If you know anything about musical performances, orchestras are loud. The music comes from all over the place and a world-class orchestra would be performing in a world-class concert hall. So it's really hard to imagine that many musicians faking in sync with each other to a CD and having nobody in the audience, often very musically knowledgeable people, be able to tell. Think of it this way. It's hard to fake cymbals clashing. The timing has to be perfect or you notice the discrepancy. So that immediately for me raised red flags to say, okay, what's going on here? Maybe this is an unreliable narrator. This is a creative writing professor, writing about a memoir. We know that memoirs often have distortions in them. So we started doing a lot of digging into this story and identified the composer as Tim Janis who is a New Age composer. We did a little digging into him, and it turns out that Tim Janis had an ensemble. He actually had about a dozen copies of this ensemble traveling around to state fairs and mall courtyards performing New Age music that sounds like Titanic.

So it was often a violinist, maybe a keyboardist and a penny whistle and nothing like a world-class orchestra. So we then said, okay, well, we've got to go read the original. Go read the book. And it turns out the book is very straightforward. It really never claims that this is an orchestra. There was nothing in it that said, I performed in a world-class orchestra.

Shankar Vedantam:

The term world-class orchestra, that came from the story in The Guardian. And as for the famous composer running this traveling ensemble of musicians ...

Dan Simons:

It said the composer was famous, which is true. He's a very famous new age composer. He's performed on PBS specials with George Clooney narrating, but it's new age music. It's not orchestral music.

Shankar Vedantam:

And of course, if you're doing this at a state fair, presumably with three participants, and there's a lot of other stuff going on, people are walking around, there's a lot of noise, now it's much easier to play a piece of music from a CD player. And even if people knew that it was coming from the CD player, they probably wouldn't look twice.

Dan Simons:

They wouldn't look twice, and they probably wouldn't care. It's also the case that in a mall context, with all of that background noise, you can blast the speaker loud enough that you really probably couldn't tell whether the music was coming from that one violin and that one penny whistle player.

Shankar Vedantam:

Dan could now see how one piece of suggestive but inaccurate information, the use of the word orchestra, triggered a whole cascade of assumptions in his head. Those assumptions, of course, turned out to be inaccurate.

Dan Simons:

This was a case in which we were completely fooled, not by the work itself, but by the headline. You hear famous composer and immediately assume classical composer. And if it's a world-famous classical composer, of course a world-famous classical composer is going to be leading a renowned orchestra. So it makes it reasonable to make that conclusion and to draw in the most readers possible. And then that gets perpetuated over time. So as we heard in that brief clip from NPR, the opening question is, so you performed in an orchestra? And that didn't get immediately corrected. Not surprising. It's hard to do that in real time. And that possibly continued to lead to this dissemination of what's essentially distorted information.

Shankar Vedantam:

What I find fascinating about that story is that it's a little bit like a magic trick, which is that the real action with the magic trick is not happening while the magician is actually waving his or her arms or waving the wand dramatically. It's happening with some small thing that you're not noticing. And of course, that's exactly how this worked, which is when you say famous composer and orchestra, I have in my mind, I'm thinking Carnegie Hall. I'm imagining Carnegie Hall, and I'm like, wow, 150 musicians sitting on a stage are faking it and they're playing the music out of a CD player. That's incredible. But of course it comes from those associations which are so quick and automatic and we construct entire castles of meaning inside our heads that we never go back to question.

Dan Simons:

Yeah. And I think what happens in cases like this is people will come to this assumption based on reading one term that's very precisely stated, and then end up because of that assumption, not realizing that they distorted information, that they've spread something that's actually not accurate and it's easy to do. But I think it's also just as easy for us to get really concrete specific information that we believe and forward and pass along to others because it's consistent with what we believe.

Shankar Vedantam:

This natural tendency of the mind is the same one that operates when we get taken in by scams.

Dan Simons:

So we don't even really need these large associations to have us fooled. We can just get something in our email or on Facebook or on social media that sounds right to us, and we tend to just assume that it's true before finding out otherwise.

Shankar Vedantam:

I understand some time ago you decided to make an investment in a company called American Eco. Tell me about the company and how that came to be.

Dan Simons:

This is a classic case of how not to do investments. This was one of the first times I ever invested in an individual stock, which I don't do at all anymore. I just go for very safe index funds. But this was a case in which I was really excited about this company because a friend of mine had told me about it. And this friend of mine had invested in it, had made a lot of money, it had been rising, and had lots of stories about what the company does. It was an environmental cleanup company. It was led by somebody who purportedly was a minister, which gave it this additional credibility. And initially, it was doing well and I was monitoring it really closely, and then everything came crashing and burning down. I don't know the reasons behind why it came crashing down, but it just went bankrupt to drop to zero value.

But it was a perfect case in which I heard from a friend who I trusted who was very interested in this sort of investment, had dug into it a lot, was listening to their conference calls and was equally snowed by it. He'd made a lot of money because he got in early, but this is the sort of mechanism that leads us to make a lot of terrible mistakes. So the same thing happens now with crypto investments. Many of which turn out to have been frauds or scams. You hear from a friend who has a way of making a quick buck. It sounds enticing. You don't know enough to check it out. You probably don't fully understand how blockchain works and how the mechanisms of this crypto coin function. But because you had somebody who was familiar to you and trusted, you said, "Okay, I'll do that too." And then it comes crashing down.

Shankar Vedantam:

Well, I'm sorry you lost your investment in this company, but it does reveal an important lesson here, which is that we trust people whom we know, and in some ways there's a very good reason for that. Our lives would be impossible if we didn't trust our friends and coworkers and family. And what you did was you extrapolated that trust probably into a domain where you should not have done so. The problem is not with the trust, but in assuming that trust carried over when it came to the land of investments.

Dan Simons:

Exactly. We have to trust people, and we trust people often because they're familiar to us. So familiarity is one of these hooks that we find really appealing. If something's familiar to us, we tend to give it more credibility. We tend to trust it more. And that makes a lot of sense. Because as you've gotten to know people, family, friends over time, people who've led you astray or lied to you repeatedly are no longer people you're going to associate with. So the people who are still in your circle, who are still familiar with you, who you interact with regularly are people who've earned your trust. And we've learned to do that over probably many, many generations. The problem now is that we have a much, much larger circle of people who seem familiar to us than we might have in our past. So you might have a ton of "friends" on social media that you've never actually met and you've only interacted with virtually. But because you've interacted with them in some form, you assign them that same trust you might to a closer friend, and that can lead you really wrong. So if you look at most romance scams, for example, they gradually build up that familiarity until you fully trust somebody who you've never met at all and probably looks and sounds nothing like the person you interact with virtually.

Shankar Vedantam:

There's also the question of domain familiarity here, which is that let's say for example, I'm speaking to you and you're an expert in psychology, you're a professor at a university, but we suddenly started talking about theoretical physics, and you very confidently told me something about theoretical physics. Now, I might tell myself, Dan Simons, smart guy, clearly he's an academic, he's an expert in psychology, maybe I should follow him and trust him in what he says about theoretical physics. And this is the other thing here, which is that expertise in one domain doesn't necessarily translate over to the next, but it has the feeling like it does when we talk to people whom we like and respect.

Dan Simons:

Yeah. When it veers into punditry. When people don't actually know something about a topic but are speaking with absolute certainty about it the same way they would their own content domain. I tend to find that it goes the other way around. That theoretical physicists talking about psychology garner more trust than they probably should. But yeah, that's a danger for anybody who has become an expert in something. One thing that happens as you gain expertise is you tend to trust your intuitions and your instincts more because you've built up a lot of experience with them. And a lot of really successful people believe that they can trust their gut about everything because they've had a number of judgments happen to work out.

Shankar Vedantam:

I was talking some time ago with Abhijit Banerjee. He's an economist at MIT, and he won the Nobel Prize in economics some time ago. And he was telling me that after he won the Nobel Prize, it's very hard for him to be in a social setting where people don't defer to him in everyday life. People ask him his opinion and now the Nobel Prize winner has spoken so clearly which movie should we go to? Which restaurant should we go to? I'm sure Abhijit knows the right answer. And so there's this domain creep that allows you to say, because I'm an expert in one thing, I'm seen as an expert in everything.

Dan Simons:

Yeah. And it's a really strong temptation probably to have all of these people who are looking up to you as the expert on some topic and recognizing when that expertise is limited is a real challenge. The true experts, the true really insightful people are the ones who say, "I don't know. I have no idea. I don't know anything about that." As opposed to just blustering away.

Shankar Vedantam:

When we come back, the most common types of scams and the psychological tricks they employ. You're listening to Hidden Brain. I'm Shankar Vedantam.

This is Hidden Brain. I'm Shankar Vedantam. One of the oldest legends revolves around a clever scam. In the Trojan War, Greek warriors supposedly laid siege to the city of Troy for 10 years. After much bloodshed, the Greeks pretended like they had lost the war. They sailed away on their boats, leaving behind a large wooden horse. The Trojans dragged the horse into their city as a trophy, not realizing that Greek soldiers were concealed in sight. Once inside Troy, the Greeks laid waste to the city and won the war. At the University of Illinois Urbana-Champaign, psychologist Dan Simons has studied the psychology of scams along with the psychologist Christopher Chabris. Dan and Chris are co-authors of the book, Nobody's Fool: Why We Get Taken In and What We Can Do About It. So going back about a hundred years, Dan, a man named Charles Ponzi orchestrated a massive financial fraud in the 1920s. Tell me the story of what he did.

Dan Simons:

Ponzi's idea was that by investing in postal coupons and dealing with exchange rates across countries, he could profit by buying in one country, selling in another country. Turns out that wasn't really doable for lots of reasons. But he'd gotten enough people to invest in this scheme that he didn't stop it right then. He continued to return money to some investors and then try and pull in more and more investors to use their money to pay the next people. So each person who was paying into this scam was paying out to the people who were in before them, which is now what's known as a Ponzi scheme. It's a pyramid scheme where the people who are in first get paid out, but then you have to bring in more and more people as investors to pay off the ones who came before you. It eventually crashes and burns when you run out of investors. But Ponzi promised huge returns in a very short window of time.

Shankar Vedantam:

I understand that at the time he took in something like $15 million, which must have been a boatload of money in the 1920s.

Dan Simons:

Yeah. And in fact, at one point I believe he owned a bank in order to do a little bit of the money laundering directly because it wasn't as closely regulated as it is now. What's interesting about Ponzi's scheme is that nowadays most people would know that a promise of 50% returns in six months with no risk whatsoever is impossible. It's way too good to be true. But people do still fall for variants of that. We've seen that with a lot of crypto offers. They'll offer gigantic returns in a very short timeframe. Anytime you get that sort of an offer, it's almost certainly a scam of some sort, whether or not it's actually a Ponzi scam.

Shankar Vedantam:

More recently we had the example of Bernie Madoff. I'm not quite sure if Bernie Madoff was running a pyramid scheme, but in some ways his plan had similarities to what Ponzi was doing.

Dan Simons:

Yeah. It was a Ponzi scheme in some form. He was taking in new investments to pay out old investments, but it differed in a really fundamental way. What Madoff offered wasn't a guarantee of 50% returns in six months, but what he offered was something different that investors really wanted. They wanted safety, security, and consistency. So his "fund", fund in quotes there because it wasn't actually a real fund, promised and returned eight to 14% every single year with never a down year and almost never a down month. That's really enticing to a lot of people who want that regular return without the risk associated with it. He took away all the volatility and he claimed he could do this because he was a master at watching the market and the trading floor, and he could time his trades to take advantage of that in a way that would've been superhuman.

Shankar Vedantam:

Talk about the fact that Bernie Madoff was a pillar of the community. A lot of people looked up to him. People had deep ties to him, and in many ways, all of these social connections ended up exacerbating the scam.

Dan Simons:

Yeah. In many ways the Madoff scam is what's known as an affinity scam. In that he was preying on people that were part of his social community. The Jewish philanthropic community in New York especially. And he was the head of NASDAQ before this all happened so he was a very prominent financial figure. He was a titan of the industry. And prior to his scam, he actually ran a legitimate business that had he just done that was highly successful, which may have made it harder for people to doubt him because he could speak with a lot of authority. He didn't come across as a brash conman. He came across as somewhat humble and understated and was very good at getting people to trust him, especially people who were associated with the groups that he associated with. He also was selective. He wouldn't allow just anybody to invest with him. He selected it and made it this accomplishment to have been allowed to invest with him.

Shankar Vedantam:

I'm picking up on two psychological themes here. One is the principle of scarcity, which is you make something scarce and it becomes, people want it more than they did before. But also we're returning again to the same theme that we discussed earlier when you fell for something because a friend told you that an investment was a good investment, which is in this case too, people were investing with Bernie Madoff because they trusted him. They thought Bernie was my friend. Bernie's not going to do anything to harm me.

Dan Simons:

Exactly. And in fact, when you know somebody and you're somewhat familiar with somebody, that's when they're going to be most able to take advantage of you if they happen to be a con artist of that scale. And the assumption is so firmly held that it becomes a commitment, something that we no longer question. There's a great example of a financial investor who was attuned to what was going on with the Madoff Ponzi scheme, knew it was a scam, tried to warn a friend of his whose parents had been heavily invested in Madoff and said, "Look, you should tell them to get out of this. This is not a good thing." And their response was, "Well, he just doesn't know Bernie. Bernie would never screw us." That idea that he's beyond reproach is what allowed him to continue to operate the scam without being questioned.

People were fully committed to it. Even when they heard evidence to the contrary, wouldn't back away from it. And that's something that happens to all of us in many contexts, and that can lead to a lot of problems. It can send you down a rabbit hole that's very hard to crawl back out of because in order to undo all of the logical steps and hoops you've jumped through to get down to that hole, you have to take a reevaluation of the place you started and question it. Something that we're generally not that prepared to do.

Shankar Vedantam:

Here's another example along those lines. Some time ago, an entrepreneur named Elizabeth Holmes spoke at the Clinton Global Initiative about an extraordinary breakthrough that her company was working on.

Elizabeth Holmes clip:

Our work is in being able to make lab testing accessible to people in a way in which every person, irrespective of their insurance status, irrespective of where they live, can afford the ability to get a test done. So that means in our case, we've invested the last 12 years in now developing hundreds of tests, many of which are less than $10.

Shankar Vedantam:

There's something very seductive about this idea, Dan. Because of course, if you in fact can test people for a number of disorders for very little money, that really could revolutionize the treatment of medicine, especially in poor countries, and I can see why the Clinton Global Initiative, which is interested in helping people in poor countries, why this could be very appealing to hear a message like this.

Dan Simons:

Yeah. It's a beautiful vision, right if it were true. And one of the things that was a hallmark of their claims was that they could, with a single small amount of blood, much less than you would typically take in a blood draw to do traditional testing, could test for hundreds or possibly even more than a thousand different things with a miniaturized testing machine that could be moved around. Whereas other traditional testing would require giant machines, larger amounts of blood, which in some extent explains why venture capitalists wanted to jump right in. It explains why former military leaders were excited about it. Because here's the potential to do something where you could do blood testing in the field as opposed to having to send it back to a traditional lab.

Shankar Vedantam:

So Elizabeth Holmes' company, Theranos, eventually went out of business. But again, I want to stay with for a moment with the psychological idea here, which is in this case, the idealism of the vision of the company was one reason why people were not as skeptical as they should be. Because of course, you didn't want to question whether this was possible because in fact, it was going to help so many people, including poor people. It felt like something that was noble to do.

Dan Simons:

Well, and it would be something that was noble to do. The idea of it was noble, which of course is also a hallmark of a lot of scams. It's going to be this amazing result. All you have to do is put in a little money here. And it also fit into the Silicon Valley hype cycle of promising products that don't actually eventually do what they claim to have done. So it fit more into that mold as opposed to the medical devices mold and the biological testing mold, which doesn't tend to have that hallmark where you normally have to prove it before you can get a lot of money. So one thing that's interesting about this is, yeah, it had a tremendous amount of appeal to people who thought about how they could use this technology, how much benefit it would be, but not so much to biotech experts.

So if you look at the board of directors of Theranos, it was filled with retired generals, retired admirals, former secretaries of state, really accomplished successful people, but people who had no specific training in biotech, and most of the money that's funded Theranos came from Silicon Valley venture capital firms, not from the East Coast biotech world and investment firms. And there's a good reason for that. The people who invest in medical technologies, who are experts in biotech would thoroughly check those claims out, and most of them probably knew that's pretty far out there to be able to do all of this with a pinprick of blood and big claims with big consequences require really strong evidence.

Shankar Vedantam:

You talk about two other scams in the book that are really interesting. Tell me about the president scam.

Dan Simons:

Oh, yeah. This is one of my favorites. This was a con initiated by a man named Gilbert Chikli, who's a French Israeli con artist. And he has admitted to essentially creating this scam. What he would do is he would call up, say, a middle manager at a company. So let's say the manager of a regional bank. And he would pretend to be the president of the entire bank chain. So that regional bank manager probably knew who the president was, but wasn't a close colleague. They were far enough down the hierarchy that they wouldn't necessarily know that person well. But what he'd do is he'd call them up and tell them, "Hey, I'm working with the French government and we're investigating people who are aiding terrorists, and the Secret Service has identified a couple of accounts at your bank branch that may be tied to funding of terrorists. So you're going to get a call from a Secret Service agent in the next hour or so, and I want you to help them in any way that they need."

Of course, it wasn't the president of the bank branch. It was Gilbert Chikli. But the bank branch manager, if they trusted that this was who he said he was, and we tend to have a default bias to accept what we hear is true unless we have reason to question it, then they're lost. So the Secret Service agent calls, and it's probably Chikli again. Calls himself Paul, and says, "Okay. We're going to need you to go get a new phone because yours is not secure." And involves the mark in the con. They become a participant in this idea that they're helping the government with a secret operation and they can't tell anybody about it.

So in the end, he gets this bank manager to withdraw hundreds of thousands of euros from the bank claiming that they're being withdrawn from a known account, taking it to a cafe, handing over a bag of cash to somebody so that that person can then go and mark the bills and bring them back, and then of course they're gone. So if you're the bank manager, if anybody stopped and told you from the outset, "Hey, look, how odd would it be for a bank manager to withdraw 300,000 euros from your own bank, stick it in a bag, and then hand it to somebody outside of the bank in a cafe?" Any bank manager would say, "Well, that's got to be a scam." But in the moment when you have bought into that initial statement that this was the president, then you've been hooked.

Shankar Vedantam:

If you are the middle manager, it would take some gumption to say, "I don't know who you are. Let me call you back to confirm that you are the president." That would be very rude if you are a middle manager speaking to the president of your bank.

Dan Simons:

Exactly. And that's the real challenge. That's why it's so effective. It's coming from a position of authority and typically a trusted authority. There's a modern variant of the president scam that's much dumber, that's pervasive now. And if you work in an organization, you may well have already gotten emails from your company's CEO or president or your department head saying, "Hey, I'm in a meeting, but I need you to finalize this transaction," or, "I need you to pay this invoice really quick." If you happen to be the business manager who gets those sorts of messages or emails regularly, you probably won't think twice about it. But once the business email has been compromised so that they know who to send it to in the right way, it's very hard to question that. I have many friends and colleagues who have gotten those sorts of emails and have also had them sent from them, supposedly to their underlings. Because in that case, all you have to do is send a message that looks like it's from the right person and have it happen to go to the right person. And one way to do that is to send a ton of messages, and all you need is a couple of them to go through out of every thousand, and you're still going to profit from it without having to do any work at all.

Shankar Vedantam:

I understand that you once clicked on a link without thinking about whether the link was a phishing or a hacking attempt. Tell me what happened then.

Dan Simons:

Right. I got a message from a friend, a close friend, who had emailed me in the past when they'd seen me on TV on some interview or something. And the message was, "Hey, I saw you in this video," with a link. So in this case, I didn't think twice about it. Said, "Oh, I wonder what video they saw." Click. And almost immediately I realized, wait, that probably wasn't legitimate. Probably their account has been hacked, and it was sending emails to everybody in their address book, and I just happened to be the one who sometimes is on videos. And I was the target, just like that business manager who pays the invoices was the target. Fortunately, I realized it quickly and quickly changed all my passwords and I was fine, but if you are targeted in the right ways that appeal to the right information to things you're looking for and predict and expect to have happen, you can fall for it.

Shankar Vedantam:

When we come back, other techniques to combat our propensity to fall for scams. You're listening to Hidden Brain. I'm Shankar Vedantam.

This is Hidden Brain. I'm Shankar Vedantam. Psychologists Dan Simons and Christopher Chabris are co-authors of the book, Nobody's Fool: Why We Get Taken In and What We Can Do About It. They've explored a number of different scams, but also ways to spot scams and to protect ourselves. Dan, in 1943, US warplanes conducted a raid over Nazi Germany, but of the 291 B17 bombers taking off from Britain, only 33 returned undamaged. Tell me the story of what happened and the important scientific principle that the story produced.

Dan Simons:

This is a story about a prominent statistician, Abraham Wald. A statistics prodigy. And in 1943, what the military needed to try and figure out was, is there something they could do to help these planes survive? Because it's obviously very costly to lose that many planes on a mission. So the question they asked Wald was, "How could we, say, reinforce the planes to prevent the damage from taking them down?" So they sent Wald all of the evidence that they had from all of these damaged planes that they'd gotten looking at where they'd been hit by the anti-aircraft fire and flack, and he did an analysis where he looked at where the planes had been hit as a function of how big the area of the planes were. So the bigger the area of the plane, the more times it should be hit if the damage is just random.

And the question was, is the damage random or is there something systematic about it? If it's random, you're out of luck because you can't reinforce the whole plane, it becomes too heavy, can't travel as far, has to carry more fuel, can't carry as much munition so it doesn't work. So the hope is that you could reinforce parts of the planes in a way that would increase their survivorship. So Wald did some very sophisticated analyses that are actually still in use today in statistics and what he showed was that the damage wasn't random. Some areas of the planes were more damaged than others and more damaged than you would expect based on how big they were. So the take home message that you might take from Wald's analysis is, oh, okay, hey, let's say the fuselage was damaged more than you'd expect by chance so let's reinforce the fuselage.

Shankar Vedantam:

Because you can see that's where all the shrapnel and the anti-aircraft gunfire is going.

Dan Simons:

Yeah. But if you did that, you would've done exactly the wrong thing. And the reason for that is we think about the evidence we have right in front of us, which is all of these damaged planes and how much they've been shot up. But the parts on the planes that were damaged on those planes that survived presumably weren't critical to the planes surviving. They could get hit there a lot of times and they still made it back. What you have to think about are the planes that didn't return. The ones that you don't have any data from. And what you should do instead of reinforcing those areas that were damaged a lot, is reinforce areas that were almost never damaged on the returning planes because presumably that's where the planes that were shot down were hit. This is a principle, a habit that we call focus. We tend to focus on the information we have right in front of us, and we tend not to think about what information we're missing. In this case, in order to make the right decisions, you have to think not just about the planes you have, but about all of the planes that went down and where they were likely to have been hit. And Wald of course didn't make this mistake. Wald knew what he was doing.

Shankar Vedantam:

This is like the invisible gorilla experiment in some ways, which is you're so focused on counting the passes among the people passing the basketball that you fail to notice the person in the gorilla suit.

Dan Simons:

That's exactly right. It's a form of focus. It's the same idea. That's a visual form of focus, whereas we can have a thinking form of focus where we focus on some information and not other information. But the real parallel there isn't just that focusing on some things and ignoring others, it's that our intuitions are wrong. So in the gorilla basketball video, if I showed you that video and you didn't happen to see the gorilla and I never asked you about a gorilla, you'd continue going through life assuming that of course you would see a person in a gorilla suit, right? Because we're not aware of the amount of things we miss. We're only aware of the things we noticed. So we'd build up this intuitive experience that, hey, all these things I've noticed, that's what builds your intuition. The things you didn't notice don't build up your intuitions. And that works well most of the time, but it also allows us to be fooled really easily.

Shankar Vedantam:

So we're in the domain now of ... I'm not sure we should call these scams necessarily, because when you have a scam, usually you have someone who is trying to fool you or trying to pull the wool over your eyes. But in many ways, these are really important distortions in daily life because I see stories all the time that say, think about people like Bill Gates and Steve Jobs and Mark Zuckerberg. They dropped out of college. How valuable could it be to go to college? And in some ways, that's like the story of the B17 bombers.

Dan Simons:

Yeah. Because you have to think about, you're only getting those cases that are right in front of you. You've got the Bill Gates and Mark Zuckerberg cases. But in order to know whether college matters for success, you have to look not just at those three examples, which are prominent examples of not going to college, but think about what percentage of CEOs of those billion dollar, trillion dollar companies actually dropped out and what percentage of them went to college. And if you actually do that, the vast majority of these unicorn companies that were successful quickly with a very large value, almost all of the CEOs of those companies not only went to college, many of them went to graduate school. So to think about the evidence, you actually need to think about the rate. How often and how often did they not do that to get a sense of whether it's important.

And that's one thing that's really hard for us because we love success stories. Hey, here's Bill Gates. He was so successful. Let's look at all of the things he did in his past and then maybe emulate them. But the thing you have to keep in mind is, okay, yeah, he did those things, but how many other people did those things and failed miserably? And how many people did other things and succeeded, and how many people did other things and failed miserably? And it may turn out that the things he did are the worst things that you could possibly do, and he just got really, really lucky. And we don't know that unless we think about the evidence we don't have. The things we're not focusing on.

Shankar Vedantam:

Dan, we talked earlier about the story of Bernie Madoff, and we talked about how part of the seductiveness of that story was that he was offering returns that were extremely predictable. Not necessarily sky-high returns, but very predictable returns. Talk about why consistency in what we see day in and day out can sometimes be a flag that what we are seeing is problematic.

Dan Simons:

Yeah. Consistency is something that we call a hook, and hooks are kinds of information that we find appealing. In part because consistency usually is a sign that somebody has a great understanding in order to be able to produce the same level of performance over and over and over again. So usually consistency is a great thing. If you can get the same scientific result over and over and over again, that means you probably have a pretty good understanding of how it works. The problem comes in when people use that consistency as a tool to deceive us. So scientific fraudsters will also produce the same result over and over and over again with almost no variability or almost no noise in their outcomes. And in normal science, that's a great sign. In fraudulent science, it's a red flag. It's a problem. And sometimes the consistency is so great that it makes it implausible that it could be real.

Bernie Madoff's case, the consistent returns he offered of that eight to 14% were impossibly consistent. If you look at all other money managing funds, anything that's producing returns of like eight to 14% over that time window would have a lot of variability. Some years would be up 20%, some years would be down close to 20%. It would bounce around. And that's true of almost any complex human-driven activity, including big systems like the stock market that involve a lot of moving parts. You want there to be that volatility because it's what happens in any natural system. And when it's missing, then you've got to worry.

Shankar Vedantam:

It's interesting throughout our lives in so many different ways, we value and demand consistency from others. So it's not just when it comes to investments, when it comes to friends, partners, coworkers. Consistency is always seen as something that's good.

Dan Simons:

It is.

Shankar Vedantam:

But of course we know from our own lives when we think about ourselves, we're up one day, we are down the next. Things change.

Dan Simons:

Yeah, absolutely. But consistency is a good thing, and that's the thing that in reviewing all of these scams and cons from so many different areas that you realize is all of them take advantage of habits that we have that usually serve us really well. So things like consistency are usually a great sign of understanding. Things like familiarity, knowing the people around you and trusting them more, is usually a great thing to do because we've built up relationships. Usually if somebody produces a result that's really, really precise, that's a sign that they have a great understanding of how to do it. If they can make a really good accurate prediction and say, it's going to rain at 1:35 today, and it does, wow, their model must be really good. But somebody who's trying to fool us or trying to deceive us is going to use exactly those same clues to good things, but to hide what they're actually doing.

Shankar Vedantam:

In our current times, Dan, not just in the United States, but in many countries around the world, we live in a hyper-partisan era where people are deeply connected to their political parties and deeply antagonistic toward their opponents. And all of us are extremely vigilant to misinformation and disinformation on the side of our opponents. Any of us can look at people who don't belong to our political party and say, "These people are idiots. How can they possibly believe what they believe?" Can you talk a moment about the role of partisanship in our ability to be taken in by misinformation and scams?

Dan Simons:

Yeah. I think partisanship of that extreme form relies on a couple of different habits we have that lead us to have a very hard time taking a step back and viewing things from a different perspective. One of them is a commitment to the belief that your side is in the right. And that can take the form of idolizing a political figure and thinking they can do no wrong to assuming that the other side's agenda must be malicious. But whatever that assumption is, once you've held onto it so firmly that you don't question it, it becomes a commitment, then you can think incredibly logically about all of the evidence you're seeing, but it's always going to be twisted by that initial assumption.

It's in many ways the way cults work. If you start with a premise that the cult leader is infallible or a super genius or everything that they say must be true, then they can lead you down a path of doing stranger and stranger things because you have to make them consistent with that initial assumption. From the outside, people in a cult are doing some really bizarre things. From the inside, they're completely logical and coherent. And as you said, we're all great at tearing apart beliefs we disagree with. When you get some information, somebody passes along something that they're sharing that you completely disagree with because you're coming from the other side of it, it's really easy to take it apart. We have a much harder time doing that about things that we do believe, and it can lead us to actually being fooled by misinformation that's consistent with our beliefs.

Shankar Vedantam:

I don't know if you recall this, but I think during the 2016 presidential election, there were reports of hacking where supporters of the democratic candidate, Hillary Clinton, were told, you can't really trust the system. The system is really rigged against you. Powerful special interests are taking over the country. Why bother showing up to vote on election day? The idea really was that something that might appeal to people with a progressive bent of saying there are powerful special interests that are controlling everything. Ordinary people have much less of a say. But you're using that message to basically encourage people not to show up to vote, which might actually not be in their best interest. If I'm a political operative, rather than tell you about the beliefs about your political opponents, which you're going to shoot down in a heartbeat, it's almost easier to come from the other end of the spectrum and reinforce what you already believe and try and twist that to get me what I want you to do.

Dan Simons:

Yeah. No. I think that's exactly right. That a lot of the time what people will try and do is tap into other beliefs that are maybe tangential to the ones that you're actually trying to direct as a way of building up a sense of trust and building up a sense of agreement. Standard technique and persuasion. You get people to agree with things that are tangential to the thing that you really want to persuade them of and only come at them for the exact demand later. So this is something that's used, for example, in some of the crypto scams. One of the hallmarks of cryptocurrencies is that they have this feel of being anti-establishment, anti-government. Trying to tap into people who might be distrusting of institutions as a way of getting them to invest in crypto, which is probably something they shouldn't be doing

Shankar Vedantam:

Very often, Dan, when I read stories about a scam, when I read about the Bernie Madoff scam or what happened with Theranos, there's a part of me that says, how could people have been so dumb? How could they have fallen for this too good to be true story? Talk about how our belief that we can spot scammers is part of the advantage that scammers have over us.

Dan Simons:

Yeah. The idea that if something is too good to be true, it probably is sounds great in principle, but the reason we come to this mistaken belief that only gullible people or clueless people or the hopelessly naive people fall for these scams is that we usually only hear about them and view them from the outside, either in hindsight or watching in the moment and knowing that it's a scam. So if you're watching a great movie about a con artist, you know that they're a con artist. You can see how they're manipulating people because you're not immersed in it. You're not part of the story. They're not taking advantage of you and feeding you those kinds of information you find appealing. But the way scams and cons work is they take advantage of those cognitive tendencies to target you, and all of us can be victimized by this. No matter how cynical or skeptical or clear thinking you think you are, we all will fall for the right scam and the right targeting.

Here's a great one in a terrible way, of course, but here's a great one that targets professors. Professors often are giving talks at conferences, and quite often the arrangements for those conferences are made by the conference organizers. So the organizers will book the hotel room, they might book the travel. And academics who do a lot of traveling are used to this. What normally happens though, is you're working directly with the conference organizers, but sometimes you'll work with a travel agency that will handle it. Normally when that happens, the organizers will tell you to call the travel agency, make the reservations. So what happens when the travel agency calls you and you're busy and you don't think much about it, and they call and say, "Hey, we're working on behalf of the conference organizers and we're booking your hotel and your travel. So what's your frequent flyer number? What's your preferred type of room? And why don't you give us your credit card so we can put that down for incidentals at the hotel?" And they've just stolen your identity. Because they aren't actually the conference organizers. The scammers just look at who's speaking at the conference and call all the speakers. And you should be never, ever giving credit card information to anybody over the phone if they called you. Ever. Because that's the entry point for a scam.

Shankar Vedantam:

I'm wondering, after doing all of this research and thinking about all these scams, are you walking around being hyper cynical and skeptical all the time, Dan?

Dan Simons:

I hope not. We have to believe that other people are being truthful with us most of the time. You can't have a world in which you distrust everything you encounter. It wouldn't be productive. We couldn't hold a conversation. We couldn't have a society. We have to be trusting to some extent, and the key is trying to figure out when to be skeptical and critical and when not to be. So the message is to try and identify those cases when you're at serious risk and think about, if I were a scammer, how would I take advantage of somebody right now? Or you get an offer that sounds good, ask yourself, okay, if I were trying to scam me, how would I do it? Asking that question allows you to ask more questions.

Shankar Vedantam:

Well, so much of the message of this book, as I think perhaps with much of your field, Dan, is the message of humility and the idea that we should in fact be cautious about leaping to conclusions, and we should be cautious about the things that we feel certain about. Do you feel like this fits in with that larger message that in some ways we should actually be more humble about the things we know and the things we think we know?

Dan Simons:

Yeah. I think the thing to trust ... I trust experts who say I don't know. That's a good thing. People who have been very successful in their lives tend to assume that they can read other people, that they can judge the situation and they go with their gut, and that's how they get taken for big dollars. Because they assume that they would spot those red flags because they're only aware of the gorillas they've seen. They're not aware of all of the ones that they missed. They're not aware of the planes that didn't come back. They're not paying attention to those. The metaphor I really like for this is of a bullfighter, a matador. And a matador shows this red cape to the bull, and the bull finds it irresistible. They charge that red cape not realizing that there's a blade behind it. Well, the sorts of information hooks that we find really appealing, that consistency in the really potent things, the familiarity, are the sorts of things that lead us to charge right in without stopping to think, and those are the times we really need to stop and ask more questions.

Shankar Vedantam:

Dan Simons is a psychologist at the University of Illinois Urbana-Champagne, along with psychologist Christopher Chabris, he is the co-author of the book, Nobody's Fool: Why We Get Taken In and What We Can Do About It, and The Invisible Gorilla: How Our Intuitions Deceive Us. Dan, thank you for joining me today on Hidden Brain.

Dan Simons:

Thanks for having me on. It was a fun conversation.

Shankar Vedantam:

Hidden Brain is produced by Hidden Brain Media. Our audio production team includes Brigid McCarthy, Annie Murphy Paul, Kristin Wong, Laura Kwerel, Ryan Katz, Autumn Barnes, and Andrew Chadwick. Tara Boyle is our executive producer. I'm Hidden Brain's executive editor. Special thanks this week to sound designer Nick Woodbury.

Our unsung hero this week is Tayla Burney. Tayla is the Director of Network Programming and Production at NPR. Hidden Brains radio show is distributed by NPR and there are many different tasks involved in making sure public radio shows run smoothly. Tayla communicates early and often about upcoming deadlines and offers, helpful, clear suggestions when we are prepping shows for public radio fundraising campaigns. Thanks for everything you do behind the scenes to help us out, Tayla.

If you know someone who would find today's episode to be interesting or useful, please share it with them. Recommendations from our audience are one of the best ways to connect new listeners to the ideas we explore on Hidden Brain. I'm Shankar Vedantam. See you soon.