Thursday, October 13, 2016

New Handbook of Experimental Economics

The new (vol 2 of the) Handbook of Experimental Economics, Eds John Kagel and Alvin Roth, came out!

The chapters are:

  • Chapter 1 Macroeconomics: A Survey of Laboratory Research, John Duffy
  • Chapter 2 Using Experimental Methods to Understand Why and How We Give to Charity, Lise Vesterlund
  • Chapter 3 Neuroeconomics, Colin F. Camerer, Jonathan D. Cohen, Ernst Fehr, Paul W. Glimcher, and David Laibson
  • Chapter 4 Other-Regarding Preferences: A Selective Survey of Experimental Results, David J. Cooper and John H. Kagel
  • Chapter 5 Experiments in Market Design, Alvin E. Roth
  • Chapter 6 Experiments in Political Economy, Thomas R. Palfrey
  • Chapter 7 Experimental Economics across Subject Populations, Guillaume R. Fréchette
  • Chapter 8 Gender, Muriel Niederle
  • Chapter 9 Auctions: A Survey of Experimental Research, John H. Kagel and Dan Levin
  • Chapter 10 Learning and the Economics of Small Decisions, Ido Erev and Ernan Haruvy

Tuesday, October 11, 2016


Andrew Gelman writes in Slate on the Replication Crisis: "Why Does the Replication Crisis Seem Worse in Psychology?"

The article begins with
"Last week, the replication crisis in psychology was pushed back into the news when Susan Fiske, a former president of the Association for Psychological Science, wrote a column in which she criticized “online vigilantes” on blogs, Twitter, and Facebook who have taken prominent work in social psychology to task. Fiske likened these “destructo-critics” to “methodological terrorists.” "

He then asks:
Why does psychology continue to dominate the news when it comes to discussion of the replication crisis?

Why not economics, which is more controversial and gets more space in the news media? Or medicine, which has higher stakes and a regular flow of well-publicized scandals?

He has 5 explanations (abbreviated here)
1. Sophistication: "..psychology is an inherently difficult field, studying constructs such as personality, intelligence, and motivation, which are undeniably important but which by their nature are “latent constructs” that cannot be measured directly."

2. Overconfidence deriving from research designs: When we talk about the replication crisis in psychology, we’re mostly talking about lab experiments and surveys. Either way, you get clean identification of comparisons, hence there’s an assumption that simple textbook methods can’t go wrong. 

3. Openness. ..In psychology, it’s relatively easy to get your hands on the data or at least to find mistakes in published work.

4. Involvement of some of prominent academics. But, in psychology, the replication crisis has engulfed Fiske, Roy BaumeisterJohn BarghCarol Dweck … these are leaders in their field. So there’s a legitimate feeling that the replication crisis strikes at the heart of psychology, or at least social psychology; it’s hard to dismiss it as a series of isolated incidents.

5. Everyone loves psychology: It’s often of general interest (hence all the press coverage, TED Talks, and so on) and accessible, both in its subject matter and its methods. 

What do you get when you put it together?
The strengths and weaknesses of the field of research psychology seemed to have combined to (a) encourage the publication and dissemination of lots of low-quality, unreplicable research, while (b) creating the conditions for this problem to be recognized, exposed, and discussed openly."

I actually always thought that there might be another reason (though admittedly I have not fully tested that theory with psychologists). But it seems in psychology, there is a huge bonus for having your own bias or model, and people mostly don't work on other people's experiments. In economics, on the other hand, it is perfectly fine to work on extensions of others' experiments and still have a great career. Luckily for me, for example, my paper with Lise Vesterlund was replicated many times, where most of these replications, and certainly almost all that are published, are in papers that have extensions where the baseline experiment was a big part of my experiment with Lise Vesterlund.

I feel that replications are common in experimental economics, and perhaps more so than in psychology, and so this may reduce the eagerness to publish results that feel not robust, and definitely may reduce the long successful careers some psychologists enjoyed before it was found out that their papers were fraudulent.

One way to think of this, is that there might be a real need to find out how many papers in economics are "replicated" where replications should be thought of in a very broad term. Then it could be interesting to compare this to psychology papers. And an important twist might be that a replication by the same author should perhaps not get the same weight as a replication by others.

Monday, May 23, 2016

Elections and women

Today, happy news, that the next Austrian president is a former colleague of mine: Alexander (Sascha) Van der Bellen... 50,35 %. 2,254,484 Stimmen fuer VdB und 2,223,458 fuer Hofer. He came is an an independent, but was the former head of the green party. For the economists, he has an Econometrica.

He won all the state capitals, and here is another familiar picture: People who voted for him: Women, Educated, 

Slate writes: "The Austrian Version of Trump Lost His Election, but Only Just Barely"

Monday, May 16, 2016

Grit: A review on scientific progress...

Daniel Engber at Slate writes "Is “Grit” Really the Key to Success?"

"Up in the Hudson Valley, among the plebes at Beast Barracks, Duckworth found the secret of success. Students who identified with statements such as “Setbacks don’t discourage me” and “I never give up” were more likely than their peers to make it through West Point’s seven-week test of fortitude. (That’s how grit is measured: Students assign themselves a score of 1 to 5 for each of 10 related character statements; the 10 scores are then averaged together.) This was Duckworth’s first case study, and it delivered a clear and forceful message: If you want to make it through basic training, you gotta have grit."

"Even the task of graduating from West Point itself doesn’t really compare to the trials of Beast. When Duckworth looked at students’ grades and “military performance scores” during their first year at school, she found that grit offered little guidance on how they’d handle the rest of the United States Military Academy curriculum. The whole candidate score—that old-fashioned, talent-based assessment—did much better. Considering that three-quarters of the students who fail to finish at West Point flunk during the post-Beast curriculum, those first seven gritty weeks appear to represent a special case, and one of marginal importance.

To show that the challenges of Beast stand in for those of life, Duckworth looked for grit in other settings, places where one might not expect to find endurance as a major factor in success. For one study, she surveyed 149 undergrads at the University of Pennsylvania, finding that the students’ grit and SAT scores—the latter used as a proxy for their natural aptitude—were each and independently related to their school performance as measured by their grades. Even for these brainy Ivy Leaguers, grit seemed to be just as important as intelligence. Indeed, Duckworth writes that this was one of the key findings that led her to the “fundamental insight that would guide [her] future work.”"

He then writes

"When everyone excels on one dimension—height, SAT scores—other factors will appear to play an outsize role. In the case of Duckworth’s brainy Ivy Leaguers, this makes their SATs seem less important for predicting how they’ll do in school and exaggerates the relative importance of their grit. If she’d mixed the same people in with a more balanced sample of their peers, let’s say those with average SAT scores closer to 1,000, then the link between their aptitude and grades would have appeared more pronounced—and that would in turn have made the correlation with their grit seem less impressive."

I actually don't fully agree with that, as we could still think of Grit being important when, say, admitting smart students. 

"That’s long been a problem for personality psychologists, who often struggle with competing terms for common, underlying inclinations. The field had become a tangled mess by the 1950s and the 1960s, says Brent Roberts, a professor at the University of Illinois (whom Duckworth also cites). For any given outcome in a person’s life—whether he might turn out to be a drunk, let’s say, or a genius or a crook—researchers would devise a brand-new measure, calibrated to predict it. “It had a brutal elegance,” Roberts says, “and I often pine for those days, to be honest with you.”

But this rampant sowing of new ideas made it hard even for the specialists to find their way within the field. They didn’t always know how their measures related to their colleagues’ or if they might be duplicating one another’s work. By the 1980s and the 1990s, lumpers in psychology had embraced a grand unified theory of personality, which collapsed all the nuances that came before into a set of supertraits—the Big Five. Under this new system, grit and all its near and distant cousins—willpower, superego strength, industriousness, and so on—would fall under an umbrella factor known as “conscientiousness.” (The remaining four of the Big Five supertraits: extraversion, agreeableness, neuroticism, and openness to experience.) Like grit, conscientiousness could be measured with a survey: a set of statements, maybe several hundred, for a person to read and then assign himself a score. (There are other ways to measure personality: A psychologist might ask people, for example, whether they engage in specific behaviors such as making lists or showing up early for meetings.)

“[The Big Five] brought clarity to a true buzz of confusion,” Roberts says, and it allowed researchers to make bigger claims about the broad significance of character. A measure of someone’s conscientiousness, for example, could help predict her longevity and physical health, as well as her marital stability. It could also tell you how likely she would be to find success in high school, college, and the workplace. But if the adoption of the Big Five proved useful in the lab, it made the science of personality harder to explain to outsiders. “When I say, conscientiousness,” says Roberts, “people go, ‘Huh?’ ”

That’s why Duckworth worked so hard to give her measure a catchy name. “I came up with it over other terms like pluck, tenacity, persistence, perseverance,” she said during one interview. “It has the connotations that I wanted. It sounds good.” It’s true: Conscientiousness comes off as something weak—a nerdy way of playing by the rules; grit suggests a vigorous, old-fashioned form of virtue. Grit’s the antidote for an overpolished age, a return to rough-hewn authenticity. “It’s brilliant in terms of marketing,” says Roberts. “People understand it immediately.”

"A brand-new meta-analysis of the literature on grit—conducted by researchers Marcus Credé, Michael Tynan, and Peter Harms using 88 samples and 67,000 subjects—provides some clues.* There isn’t much space between Duckworth’s measure and conscientiousness, the study argues. If you test a group of people for both traits, administering standard surveys to measure grit and conscientiousness, the results will end up very tightly linked; in some studies their relationship approaches 1-to-1. In Roberts’ view, grit corresponds very closely to a facet, or subtrait, of conscientiousness that has for many years been called industriousness."

Tuesday, May 10, 2016

Science and Replication

A topic becomes really important when it reaches TV shows. This is just what happened with replication in science: cHeck out John Oliver's Science segment:

Check out my paper with Lucas Coffman on that topic, and my earlier post

Monday, May 9, 2016


Nicholas Kristof in the NYtimes writes about "A Confession of Liberal Intolerance"

"A study published in The American Journal of Political Science underscored how powerful political bias can be. In an experiment, Democrats and Republicans were asked to choose a scholarship winner from among (fictitious) finalists, with the experiment tweaked so that applicants sometimes included the president of the Democratic or Republican club, while varying the credentials and race of each. Four-fifths of Democrats and Republicans alike chose a student of their own party to win a scholarship, and discrimination against people of the other party was much greater than discrimination based on race."

The study is "Fear and Loathing across Party Lines: New Evidence on Group Polarization," by Shanto Iyengar  from Stanford University and Sean J. Westwood from Princeton University
American Journal of Political Science, Vol. 59, No. 3, July 2015, Pp. 690–707

The abstract reads:
When defined in terms of social identity and affect toward copartisans and opposing partisans, the polarization of the American electorate has dramatically increased. We document the scope and consequences of affective polarization of partisans using implicit, explicit, and behavioral indicators. Our evidence demonstrates that hostile feelings for the opposing party are ingrained or automatic in voters’ minds, and that affective polarization based on party is just as strong as polarization based on race. We further show that party cues exert powerful effects on nonpolitical judgments and behaviors. Partisans discriminate against opposing partisans, doing so to a degree that exceeds discrimination based on race. We note that the willingness of partisans to display open animus for opposing partisans can be attributed to the absence of norms
governing the expression of negative sentiment and that increased partisan affect provides an incentive for elites to engage in confrontation rather than cooperation.

Monday, February 22, 2016

There is a new paper coming out: "Discrimination in the laboratory: A meta-analysis of economics experiments," by Tom Lane in the European Economic Review.

The abstract reads:

"Economists are increasingly using experiments to study and measure discrimination between groups. In a meta-analysis containing 441 results from 77 studies, we find groups significantly discriminate against each other in roughly a third of cases. Discrimination varies depending upon the type of group identity being studied: it is stronger when identity is artificially induced in the laboratory than when the subject pool is divided by ethnicity or nationality, and higher still when participants are split into socially or geographically distinct groups. In gender discrimination experiments, there is significant favouritism towards the opposite gender. There is evidence for both taste-based and statistical discrimination; tastes drive the general pattern of discrimination against out-groups, but statistical beliefs are found to affect discrimination in specific instances. Relative to all other decision-making contexts, discrimination is much stronger when participants are asked to allocate payoffs between passive in-group and out-group members. Students and non-students appear to discriminate equally. We discuss possible interpretations and implications of our findings."

Thanks Christine Exley who pointed this paper out to me.