Update: Instead of saying “Probably Not” in the title, I probably should have said “We have no idea.”
Being a Seattle parent with kids in private schools, I’ve been assailed for years by pronouncements and lectures by and about the Seattle-based Gottman Institute (tagline: “Researching and Restoring Relationships”). Their most widely known claim is their ability to predict, after watching a married couple for fifteen minutes, whether they’ll get divorced.
The basic Gottman theory — that facial expressions of contempt during couples’ interactions are predictive of divorce — seems very plausible, intuitively. But there are many intuitively plausible surmises that are just wrong.
And no matter how intuitively plausible it is, the claim always seemed fishy to me. But I never did the research to find out if their predictions were really accurate. Happily, somebody has finally done it for me. Here’s Andrew Gelman, god of all things statistical, blogging about a Slate article excerpted from Laurie Abraham’s Husbands and Wives Club.
Short story, their “predictions” are built on quicksand. Here’s how they do it.
Gather data on a bunch of couples — say, six variables for each couple. Determine which of those couples get divorced. Then run a program that finds an equation correlating the (presumably) predictive variables to the results. (These quite remarkable programs — the realm of ultimate-wonk physicists only a decade or two ago — are now available for free download, or as $49 Excel plug-ins. To quote my buddy Olav, “Isn’t it great living in the future?”)
Here’s what’s wrong: these programs will always find an equation that correlates the variables to the results. (With a greater or lesser “fit” to the data.) Does that mean the equation is predictive? Only if it makes an accurate prediction when applied to a different set of data.
That is what Gottman has not done, at least in his published papers. Every one of them has a new equation that — surprise — “predicts” the divorces in the group with surprising accuracy — the same group that was used to generate the equation.
Now this is true: if the program finds a good data-fitting equation (which Gottman seems to have done — multiple times), there’s a greater chance that the equation will actually be predictive. But there’s only one way to know: use it to predict. If the prediction fails, the predictive ability of the equation is falsified.
Gottman has not (to my knowledge) attempted any falsifiable predictions, so we have no idea if his predictions are true or false.
The Gottman Institute presumably has all the data to hand, and could test past predictions against future results. I’m wondering: will they now do so?
Google doesn’t turn up any hits for “Abraham” on the gottman.com site, so I’m thinking they haven’t responded. One can rather understand why. Abraham says in a comment to the Slate post (and, she says, in a footnote in the book — it’s not search-insideable on Amazon) that she repeatedly requested an interview in May 2009 but Gottman wouldn’t see her until October — too late for her book. I do wish she’d tried again before the excerpt was published, giving him ample benefit of the doubt.
But absent that, I’m quite curiously waiting to see what we hear from The Gottman Institute.