Measuring Success

In the closing days of Vice President Walter Mondale’s run for president in 1984, the crowds swelled, they cheered loudly and fervently, and the emotion rolling off them was enough that Mondale, bemused, asked one local politician “What are you feeding these people?” As election day neared, it seemed as if support was swinging Mondale’s way.

Of course, Mondale lost in a landslide to Ronald Reagan. The moral to the story is that personal experience cannot always succeed as an indicator of success. Particularly when you’re dealing with national public issues, it is important to supplement experience with careful, objective measurement.

Experience

My experience is that I always win the room. Churches in Pennsylvania, youth groups in Pretoria, the Scottish National Parliament, a university just outside Salt Lake City — I always win the room. That is always my expectation.

When I talk, and particularly if the group is large and I can speak for long enough to unravel a complicated subject (usually forty-five minutes) people come away convinced that eliminating nuclear weapons is not only possible but necessary. Not everyone, of course, but 60 to 70 percent. Over the last ten talks that I’ve given, for example, eight of them were remarkable successes: audiences went from skeptical of elimination to believing it was possible and essential in forty-five minutes. This includes audiences that ought to have been difficult, like the New York Military Affairs Symposium and a class at the Naval War College. I have confidence in the arguments I make for eliminating nuclear weapons because I’ve seen them work over and over again.

(I’ve even had surprising success in hostile situations, like the talk I gave at the A-10 Directorate — the Air Force’s office for nuclear weapons Policy, Planning, and Strategy at the Pentagon.)

Measurement

But what Walter Mondale’s loss taught me is that intuition can sometimes mislead. Experience has to be supplemented with focus groups, surveys, and objective measurement. As a small organization, RealistRevolt has not had access to national polling or extensive focus groups. But we have measured the impact of public presentations where possible. Below is one example of the kind of data we have gathered and the surprising results that can be gleaned from it.

Evaluation

Hiroshima Presentation

Princeton University, December 10, 2015

On Thursday, December 10, 2015 I gave a 35 minute presentation focused on whether dropping nuclear weapons on Hiroshima forced Japan to surrender at the end of World War II to Dr. Harold Feiveson’s Freshman Seminar on Nuclear Weapons at Princeton University. The audience consisted of 17 freshmen and one professor.

There were several obstacles. Freshmen are typically the most difficult college-level students to persuade. They are, after all, new arrivals in a strange institution, anxious to fit in, and trying hard to learn and conform to the norms of the institution. They tend to be the most conservative college audiences.

Feiveson also did several things to undercut the impact of the presentation, for example, not allowing time for questions (which is when the most important persuading occurs), saying before the presentation that he didn’t “completely agree” with what I was going to say, and summing up after the presentation by saying, “That was certainly an unorthodox point of view.”

Despite these obstacles, the presentation would have to be characterized as a remarkable success. The students’ views on Hiroshima shifted substantially, moving from a standard nuclear believers’ position (“nuclear weapons forced Japan to surrender”) toward the position a nuclear eliminator would hold (“nuclear weapons didn’t force Japan to surrender.”)

Students were given a sheet of paper at the beginning of the presentation with 14 statements on it. They were asked to quickly mark down their responses, where 1 equalled “agree very much”, 2 equalled “agree somewhat”, 3 equalled “neither agree nor disagree”, 4 equalled “disagree somewhat”, and 5 equalled “disagree very much.” After the presentation they were asked to give responses to the same 14 statements again. They were also asked to respond to four additional statements having to do with the clarity, factuality, and interest of the presentation. All students completed responses.

ANALYSIS

On the most important question and the main subject of the presentation, the students substantially changed their views. Their average belief before the presentation (1.82) was close to a classic nuclear believers’ position. (Perfect belief that nuclear weapons forced Japan’s surrender would have been a 1). After the presentation their belief had shifted by 51.47%. The class had gone from a very strong nuclear believers’ position to a strong nuclear eliminators’ position. This dramatic shift plainly shows the persuasive power of the presentation.

The presentation apparently conveyed considerable information to the students. They described themselves as much better informed about the incident after the presentation. Average answers went from (2.82) to (3.59), a shift of 19.12%.

On the other question on the survey that related strongly to the content of the presentation, question 6 about the belief in the ability of nuclear weapons to win wars, the results also showed considerable movement. Students began with a somewhat nuclear believer attitude (2.47) that nuclear weapons were somewhat able to win wars. After the presentation they had shifted to the nuclear eliminator’s side of the midpoint to 3.18—a 17.65% shift.

On all three of the questions related to the subject of the presentation, students showed shifts toward a nuclear eliminator’s views, varying from significant to substantial.

Interestingly, the students were ambivalent about the ability of nuclear deterrence to keep the peace. Before the lecture they were only slightly convinced, with an average score of 2.88. After the lecture, even though the presentation only included a brief section pointing out that our knowledge of deterrence is largely subjective, their views had moved 14.71% toward a nuclear eliminator’s position. They were still near the mid-point, but were now leaning toward skepticism.

The response of the students to Question 11 was remarkable. Students expressed the most disagreement with this statement than on any other question. Students apparently have a very negative opinion about the safety of nuclear weapons. And their views on the subject were surprisingly rigid. Only one person in the class changed their evaluation of nuclear weapons as a result of the presentation.

Similarly, students’ views on the necessity of nuclear weapons were both very negative and hardly effected by the presentation at all. Students began in disagreement with the statement, with an average response of 3.94, and after the presentation disagreed only very slightly more, having shifted 1.47% to 4.00.