It’s Time to A/B Test Your Financial Life

People often don’t know their true preferences. Trying them out could save you a lot of money.
Shlomo Benartzi

On my last work trip, I learned a valuable lesson about myself while melting chocolate in a saucepan. What I learned changed my dining habits—as well as the way I think about my finances.

Let me explain. For many years, I visited fancy restaurants when traveling abroad for work. I was convinced this was how I liked to eat when traveling. 

On this trip, though, I was meeting a friend who preferred that we cook at home. So we compromised: The first night, we would eat at a restaurant I picked, and the second night, we would cook at his place.

Much to my surprise, I vastly preferred the home-cooked meal. By the time we were baking profiteroles for dessert, I realized that I had totally misread my preferences. 

If this mistake was limited to my dining choices, it wouldn’t be such a big deal. However, research suggests that the same basic mistake—not knowing what we really want—can have huge implications for our financial lives. We might think we prefer a big house over a smaller condo, or that we want an aggressive investment portfolio. But if we don’t test these assumptions, we might never know that we’re wrong—with potentially major financial consequences.

So what’s the solution? How can we figure out our financial preferences before it’s too late? I propose that we borrow an essential tool used by the most successful technology companies, such as Amazon, Google and Expedia: the A/B test. 

It’s a simple idea. To improve their sites and apps, these companies generate two versions of a site (A and B). The versions are identical except for one or two key differences. Perhaps version A has a different color scheme from version B, or has the buy button located in a different part of the screen. 

Then, the company randomly assigns users to one of the two sites. The performance of each site is carefully tracked, allowing the firm to learn which version is more effective.

In research I’ve conducted with Hal Hershfield at UCLA and Steve Shu at City University of London, we used A/B tests to experiment with different versions of the same basic financial offer. Users of a saving app were randomly assigned to one of two groups. The first group was asked if they would like to save $5 a day, while the second group was asked if they would like to save $150 a month. (The amounts are essentially equivalent.) Although we had predicted that the $5 a day question would perform slightly better—it made saving seem less painful and intimidating—I was shocked by how much better it did: Those users given the $5 question were four times as likely to sign up.

I believe we should use A/B tests to study our own preferences, especially when it comes to major financial decisions. There are three keys to a successful self-experiment. First, the alternatives have to use randomization. This can be easily done by flipping a coin to see which condition should go first. Then, we should aim for scale, repeating the test as many times as possible. Finally, we have to track the results. In my own experiments, home cooking has yet to lose.

In the hope of inspiring your own A/B tests, I’ve outlined five categories in which this simple process can be used to reveal your financial preferences.

1. Buying basic vs. luxury goods

Many people think luxury equals quality; the more you spend on something, the more you’ll like it.

An A/B test that investigates those assumptions may offer some surprising insights into our true preferences. For instance, I thought I preferred fancy dining, but an accidental experiment taught me that I prefer those homemade desserts.

Another common assumption involves wine. Research by Hilke Plassman, John O’ Doherty, Baba Shiv and Antonio Range showed that when given tastes of wines and told how much each bottle cost, people preferred the more expensive wine. But when the wines were sampled blind—a classic A/B test—the subjects gave a higher average score to the $5 wine than the $90 bottle.

We can apply this to any number of products. We might learn that we prefer inexpensive T-shirts over fancier brands, or that a luxury sedan isn’t as much fun to drive as a more basic coupe with a manual transmission. 

At the very least, testing our more expensive preferences can provide reassurance that they’re really worth the extra money.

2. Monitoring your portfolio more vs. less frequently

New technologies have made it easier than ever to use the tools of science to study ourselves—whether it’s how many steps we walk or our blood-sugar levels or our sleep habits. That can be a good thing, but it also has its drawbacks: We can get obsessed with reaching a certain number, and the resulting anxiety can do us more harm than good.

The same problem afflicts many investors. Thanks to our smartphones, we can now get constant updates on our investment performance. My own research, done with Richard Thaler, has shown that more frequent updates could make people obsess over short-term losses, when they should almost certainly be focused on their long-term investment goals. Over time, this can lead to an excessively conservative portfolio and lower returns.

This suggests that investors should conduct an A/B test on how they’re affected by these new forms of financial feedback. The A condition could involve checking your investments multiple times a day on a mobile app. How do you feel? Does your blood pressure rise on days when the market drops? The B condition could feature quarterly paper reports only. Does less feedback make you less stressed? Or are you missing relevant information?

While some people can benefit from continuous digital feedback—such data led me to change my breakfast for the better when I bought a blood-sugar monitor—others find it too stressful. This means that we need to fine-tune the amount of information and updates we get, conducting frequent tests to make sure that our technology is helping us, not hurting.

3. Retiring now vs. later

One of the most important financial decisions people make concerns the timing of their retirement. Unfortunately, many people are so tempted by the prospect of enjoying their retirement that they retire too soon.

For instance, I know several people who harbor the fantasy of playing golf seven days a week after they stop working. But it’s possible that the same game that’s extremely enjoyable when played once a week becomes tedious when played every day. After all, decades of research shows that people consistently underestimate the power of adaptation, or the tendency to get used to our circumstances. This is why lottery winners are rarely as happy as they would have predicted; they get used to the money.

Similar miscalculations might help explain why, according to a 2017 study by Rand Corp., 39% of workers over 65 who had retired changed their mind and returned to the workforce. While some of these workers almost certainly needed additional income, the Rand authors suggest that many are working because they want to. Maybe a leisurely life of golf and cards wasn’t as much fun as they expected.

That’s why I think it’s important to test-drive our dreams. Rather than guess what we’ll want to do in retirement, we should plan an extended vacation that allows us to simulate the retirement we think we want. We might learn that we’re in no rush to stop working, and can thus enjoy a more luxurious retirement once it begins, or that we don’t want to retire at all.

4. Spending more vs. less in retirement

People often decide how much money they’ll need in retirement by projecting forward their current lifestyle. Such forecasting leads many to conclude that they’ll need to maintain their current standard of living; anything less feels like a loss. Financial advisers, meanwhile, typically recommend an income replacement of 70% in retirement. 

To figure out which projection is correct, people should experiment with higher and lower amounts of spending during their working years. In this experiment, the A condition would involve cutting 40% of your discretionary spending for a month or two, while the B condition would involve cutting just 10%. How did the experiences compare? 

You might learn, for instance, that you’re just as happy spending much less money and can thus retire earlier. Or maybe you miss the extra spending, and thus should save at a higher rate during your working years. Either way, a simple A/B test could be the difference between an enjoyable retirement and one that is profoundly disappointing.

5. Spending on the usual vs. new experiences

We earn money so that we can spend it, investing in those things and experiences that make us happy. But many of these spending decisions reflect our strong bias for the status quo, as people tend to keep choosing those products and experiences they’re already familiar with, even if there’s a better option available.

That’s why it’s important to continually try out novel experiences, like baking profiteroles at home.

Here’s another example. I’ve long had the habit of working in cafes. I’ve assumed I’m more productive when I work in spaces with few temptations besides caffeine. However, on a recent work trip to a place without free Wi-Fi and just a standing counter, I was forced to adjust. As a result, I experimented with different work setups, even after I returned home, alternating between my usual cafe (the A condition) and my house (the B condition.) I was surprised to discover that I was more productive working at home. Not only did I save money making my own coffee, but I found I was better able to grind through my email and think about research. 

The same lesson applies to countless other domains. I know many people whose default vacation choice involves a nice hotel on a pretty beach. In the spirit of A/B testing, however, I believe these people should try out some vacation alternatives, such as hiking in the mountains or visiting a foreign city.

Perhaps they’ll realize that their favorite spot really is the beach. Or maybe they’ll discover that they’ve been leaving happiness on the table, and that they most enjoy experiencing new places on vacation. 

***

Americans are devoted to the pursuit of happiness. Unfortunately, research shows that many of us don’t actually know what makes us happy, so we end up pursuing the wrong things. 

The good news is that self-experimentation can help us figure it out, allowing us to pursue the right things before it’s too late. Personalization is currently a buzzword, as companies like Amazon and Netflix promise to improve our choices by offering highly personalized recommendations based on our past preferences. However, given the errors that plague many of these preferences, it’s clear we can’t just rely on algorithms. If our past choices have been flawed, then these future recommendations will be no better. 

And that’s why we need to A/B test our life, designing our own experiments and comparing alternatives so that we can ensure our decisions reflect what we really want.

The Wall Street Journal