Believing We’re Right is Not Being Right

So Two People Walk Into a Bar…

You and I are sitting at a bar, discussing our mutual love of fantasy football over beers. After we’re a few drinks in and my words have just barely started bumping into each other as I talk, I lean in towards you and conspiratorily whisper, “I’m going to let you in on a little secret.”

“I have,” I continue after taking another swig, “something of a secret power. You probably won’t believe me when I tell you this, but I’m actually able to communicate with coins. You wouldn’t think that a coin would have much to say, but you would be wrong. In fact, it turns out that coins have a surprisingly deep psychology not wholly dissimilar from our own.”

“We believe that coin flips are independent events, but some coins, it turns out, are actually clutch performers. Other coins, by contrast, are terrible chokers. If a coin flips heads twice in a row, suddenly the pressure mounts for it to flip heads a third time and complete the trifecta. Now, coins are still largely at the mercy of the humans who flip them, but it turns out that the “clutch” coins are capable of influencing events enough that they’re more likely to come up heads that third time. Likewise, “choker” coins are famous for blowing it when their big moment comes and flipping tails, instead.”

“Like I said, I don’t expect you to believe me, but I’m so convinced of my own power that I’m willing to make a wager with you. You can pull a coin out of your pocket, and I’ll talk to it to figure out what kind of coin it is, and I’ll tell you. Then you can flip that coin four times, and if I’m right about it- if I correctly predicted how it would come up after a series of two straight heads- then you pay me $10.”

“In fact, I’m so confident in my secret power, that I’ll even give you the odds. If it turns out I’m wrong and I incorrectly predict the outcome, then I’ll go ahead and pay you $12. And if during those four flips we wind up with a push- a coin that either had as many “Tails” as “heads” flips following consecutive heads, or a coin that never flipped consecutive heads in the first place- then no harm and no foul. In the event of a push, no money changes hands.”

“In fact, despite giving you the odds, I’m so confident in my ability that I’ll gladly play this game for every single coin you’ve got on you. If you want, you can even run to the bank and withdraw more coins. I’ll play it for every coin you bring me. I’m that confident in my ability to talk to coins and interpret their psychology.”

Now, you are smart and numerically literate, I’m sure. And you think my so-called “ability” sounds like complete nonsense; after all, I’m rarely friends with fools. So perhaps you agree to play, thinking you’ll win a few easy bucks. You have 100 coins on you, so we play this game for each of these 100 coins, and to your surprise, you find yourself down nearly $40 at the end. So you run out to the bank, trade in a $10 bill for a thousand pennies, and you come back to the bar and we play again. At the end of this, you are shocked to find yourself down about $400. What the hell is going on?

I’m going to let you in on a little secret. You think I’m making it up when I say I can talk to coins, and you are absolutely correct. You also think that the odds that a coin comes up “heads” after two consecutive heads are exactly the same as the odds that it comes up “tails” after two consecutive heads, and you are once again 100% right.

What you don’t know is that the entire structure of the game has been built to favor me. Let me explain how.

When you boil it down, what we’re doing is flipping a coin four times and counting the number of HHT and HHH patterns. Instead of trying to divine a coin’s unique psychology, I’m just going to predict that every coin you show me is a “choker”. That means that HHT will count as a win for me, while HHH will count as a win for you.

This still probably shouldn’t seem that big of a deal to you. After all, HHH is equally likely as HHT over a string of three flips. And again, you’re right. It’s not about the frequency of those HHH results, it’s about the DISTRIBUTION of those HHH results.

When we flip a coin four times, there are 16 possible outcome strings, (HHHH, HHHT, HHTH, HTHH, …, TTTT). Each of those outcome strings are equally likely. In order for you to win, either flips 1-2-3 have to be HHH, or flips 2-3-4 have to be HHH. The remaining flip can be either heads or tails. This can be generalized to (H or T), H, H, H -or- H, H, H, (H or T). Or, to spell it out, that means the winners for you are HHHH, THHH, HHHT, or HHHH.

Spelling it out this way should make something immediately clear. The first “winner” and the last “winner” on that list are exactly the same sequence. In addition, while HHHT counts as a win for you because of the first three flips, it also counts as a winner for me because of the last three flips. So that one is actually a push.

This leaves you winning $12 from me every time the coin comes up HHHH or THHH. On the other hand, winning scenarios for me include H, H, T, (H or T) and (H or T), H, H, T. In other words, I win $10 from you every time the coin comes up HHTH, HHTT, or THHT, (remember that HHHT is a push).

Since each sequence is equally likely, we should expect each sequence to result 1/16th of the time. That means you have a 2/16 chance of winning $12, and I have a 3/16 chance of winning $10. Over 16 trials, I should expect to win $30 and lose $24, leaving me up by $6 total for an average of 37.5 cents per trial. In fact, I could give even more generous odds and still expect to wind up in the black; the break-even for me in the long run is wagering $15 to win $10.

The key to the trick, remember is that each of us has a winning sequence that occurs three times, but two of your wins happen in the same sequence. HHHH wins twice for you, but only pays once. If we weighted the payouts to account for how often each sequence occurred, (i.e. if we paid you twice when you got the sequence that won twice), then this bias completely disappears.

Rephrasing Things a Bit

Okay, this is an interesting way to abuse probabilities and perhaps win a couple bucks in a bar bet, but what does it have to do with sports? And that’s where this story gets complicated. Let’s say that instead of “coins” in this game, we’re talking about… say… basketball players. And instead of “flipping heads”, we are talking about “sinking shots”.

Suddenly we now have set up the “hot hand” theory- the idea that players get “in the zone” and start playing at a higher level, becoming more likely to make their shots. Think back to NBA Jam, where making enough shots in a row left a player literally “on fire”, with their capabilities seeing a commensurate boost.

The “hot hand theory” has taken quite the beating in academic circles over the last thirty years. It started back in 1985 when a paper was published by Gilovich, Vallone, and Tversky that analyzed the home games of the Philadelphia 76ers over a season. They concluded that players were actually less likely to make the next shot after a string of makes.

Over time, their research has been expanded upon several times, with each attempt confirming that the “hot hand” didn’t exist. The idea that belief in the “hot hand” was merely an example of people falling prey to selective memory and the post hoc fallacy became entrenched, to the point where the “hot hand fallacy” even has its own Wikipedia page.

But, in an unsurprising twist that I telegraphed from a mile away, it turns out that the researchers have analyzed the data in such a way that it falls prey to the same bias that our coinflip game demonstrated. Earlier this month, Josh Miller and Adam Sanjurjo came out with a paper where they expounded upon this bias and re-analyzed the data from that famous Gilovich, Vallone, and Tversky study. And wouldn’t you know it, when properly controlling for the bias, the very data that GVT used to prove that the hot hand was a myth actually demonstrates that the hot hand is a real phenomenon.

Okay, this is all fascinating, but I’m not an NBA writer. Why am I bringing this up? What does it have to do with fantasy football? There are three takeaways that I want to make from this.

A Call to Humility

First, a reminder that humility is pretty much always the proper course of action. Until two weeks ago, I would have told you that I was completely convinced concepts like the “hot hand” and “momentum” were myths, examples of humans reading patterns in randomness, and held no predictive power whatsoever. And I would have been wrong.

Being wrong isn’t necessarily a problem. It happens. The true danger comes when we are confident in our wrongness. Regardless of how compelling the data might seem, this is a great reminder that we cannot expect something which is unexpected. We can, however, expect the unexpected in a general sense. We must remember that “no one can find flaws in this” does not mean “there are no flaws in this”.

Prior to 1985, pretty much everyone would have accepted it as prima facie true that sometimes players got “in the zone” and had a “hot hand”. Research to the contrary essentially said that everyone was wrong and silly and had been fooled by cognitive bias. To paraphrase a famous quote, (which almost certainly did not, in fact, come from either Abraham Lincoln or P.T. Barnum), “you can fool all of the people some of the time”.

But you’d think it’d be hard for everyone to be wrong, and claims to that effect should be approached with a heightened degree of skepticism. Instead, this claim was latched on to with glee, partly as a way for the “educated” to show just how silly and superstitious the masses were. And now it looks like the masses might well have been right all along. This is a great reminder that we should approach sweeping claims with a great degree of humility and hesitation.

Now, of course, this first takeaway is an easy one for me to make. I’ve written before about how humility is good policy. In other words, this entire aside is a great way for me to beat my chest and say “Hey guys, look at that, it turns out I was right all along! Aren’t I so great? Am I great, or am I the greatest?!”

A Not-So-Easy Takeaway

If I ended this post there, I’d be pretty happy with the way things stood. Unfortunately for me, I’m not the type to end a post there.
In truth, not everything is sunshine and roses for me here. You see, one of the positions for which I am best known is the idea that the NFL draft is a relatively efficient market. (In practice, whether the market is efficient or merely rigged is up for debate, but from the perspective of someone trying to project player performance, either situation prescribes the same policy decisions.) In other words, I believe that nobody outperforms the general market over a long timeline.
I have a very good reason to believe this. I have seen multiple studies from credible sources. Each of them compared a team’s draft performance against itself. Basically, they looked at teams that were successful in the draft, and then they looked at subsequent attempts to see if they were likewise successful. In short, they were looking at sequences of hits and misses to see if teams exhibited any signs of streaki… oh dear. Oh, oh, oh, oh dear.
Now, let me be clear: I based my belief that the NFL draft is an efficient market on several studies. Those studies purported to measure the streakiness of draft performance. Some methods of measuring streakiness underestimate the actual observed streakiness. If the studies I based my belief on committed those same errors, then there’s a fantastic chance that my core belief is wrong.
I do not yet know whether these studies committed the same error. I’ve spent much of the past two weeks just trying to make sure I understood the bias in the first place. The next step is to go back and dig into those studies’ methodology and see if it’s prone to this specific error. It might not be. There are absolutely ways to analyze data that do not commit this error. (Think back, for instance, to my example earlier about weighting our coin game so that a streak that wins twice pays twice.)
So, I’m not saying that one of my cornerstone fantasy beliefs is wrong. I’m saying that one of my cornerstone fantasy beliefs very well might be wrong. And. That. Sucks. 
I have devoted a large amount of time and effort to building this position, exploring its ramifications, and applying it to my fantasy philosophy at large. If it turns out I was wrong, then I have to rethink everything from the ground up. That’s hard. That’s a lot of work. That invalidates a huge amount of the effort I’ve already expended. It also means much of what I said over the past two years was foolish, wrong, and misguided. And, all else being equal, I’d somewhat rather not discover that much of what I said over the past two years was foolish, wrong, and misguided.
Again, I don’t know how this will turn out. I’m early in the process of working my way through this new twist. But just the possibility is something that is deeply unsettling.

One Final Takeaway

This leads me to my final takeaway. At some point, we have to decide for ourselves whether or not we want to be right. Now, this might sound crazy to you- of course we want to be right! But I can tell you from the intense personal conflict I have dealt with over this that this is not the case.
In fact, sometimes what we want is merely to believe we are right. When it comes to a choice between being right and believing we are right, one path is easy and comfortable, and it’s not “being right”. One path inspires self-confidence, and it’s not “being right”. One path is appealing to outside observers, likely to win us respect and recognition. Sadly, it’s not “being right”.
We would rather take advice from people who know what they’re doing, and nothing inspires quite so much confidence that we know what we’re doing as believing we’re right. The quest to actually be right involves frankly acknowledging when we are wrong. It is the path of nuance and self-doubt when most are looking for simple prescriptions offered with certainty.
I feel compelled to point out that nobody approached me with this new data. Nobody has used it to argue against my position. I am not writing this post because I happened to lose an argument with someone.
I know that the fantasy space is crowded with smart, talented writers who are all vying for your time and attention, and “I’m always looking to find new ways that I am wrong” is not a very strong sell. When some people are only too happy to tell you that they have it all figured out, I’m telling you that I’m still busy figuring out how wrong I am.
But at the end of the day, an important part of getting better is being honest with ourselves when we’ve gone astray.