PATTERN RECOGNITION, PART 2: THE PRACTICAL GUIDE

TAKE THESE CRITICAL STEPS TO ACHIEVE YOUR GOALS.

Last week, I talked about the importance of Pattern Recognition for predicting outcomes of your decisions. Today, we walk through the practical steps of how to master this process of using data to get what you want.

Although Pattern Recognition is complex, we can break it down into digestible steps:

Step 1: Divide and Conquer
Managing a vast amount of data

When you have a lot of data at hand, it needs to be simplified. Do this by “bucketing” the data into categories. No matter how complex the issue or amount of data involved, bucketing is always possible. Force yourself to use this technique—it works.

In my book, Never Be Wrong Again, I use the example of heart disease to illustrate this process. While there are a multitude of factors that can relate to heart disease, the data can be distilled into five major buckets: genetics, diet, lifestyle, occupation, and “other.”

When all data is sorted into these five categories, we can analyze volumes of information more simply by analyzing the groups, instead of each individual bit of data.

Step 2: Find Correlation
Connecting the dots

With all information bucketed into categories, we now look for commonalities between data points. Look for a high correlation (at least 75%) between certain factors or events and your desired goal.

Why at least 75%?

If there is only a 50% correlation, that factor is not present half the time that the hoped for event happens—and that is too high a risk.

When you find a connection between data points, ask yourself:

1.What are the assumptions underlying these data points?

2.When was the data derived? High correlation decades ago may not be relevant today.

Example: A recent article claimed that when IBM’s earnings beat estimates, 80% of the time the S&P (the broader market) traded higher.

Should we invest in the market just based on this information? No. We have to know when that data was derived and over what period of time. Was the data based on how IBM was performing from 1975-1985? Are those years reflective of the current IBM? Also, what was going on the other 20% of the time this didn’t work?

To find the assumptions underlying the data points – ask, “WHY?”

Why is there correlation? If you don’t know why, the data is not useful. Just like the example about the IBM stock, until you know the reason correlation exists, don’t base a decision on it.

Don’t just take correlation at face value. Understand the facts around the data.

Step 3: Look for Causation
Spotting smoking guns

Causation is the “holy grail” of decision making. We search for it, but rarely find it. Causation indicates that a certain factor always, or almost always, causes an event to happen.

If I flip a switch, it causes the light to go on. If something doesn’t cause something to happen, it is not causative. It may be may be relevant, but that is very different. In the quest to find a causative factor, be careful not to jump to the wrong conclusion about its importance. Don’t gloss over the word “cause.”

When there is no direct causative factor...

Step 4: Determine Whether Relevance Exists

Figuring out what matters

It is a common practice to focus only on correlation and causation, ignoring the all-too-important relevance. What if there is a strong connection between two items but nothing that rises to the level of direct causation? Then you have found relevance.

Just because something doesn’t cause an event to happen, doesn’t mean it isn’t important.

Something is relevant if it greatly increases your chance to get the result you want—even if it does not guarantee the outcome.

Pattern Recognition in Practice
Example: Sushi and Same-Sex Marriage

A recent study showed an almost 100% correlation between people who eat sushi and those who support same-sex marriage. The statistic makes for interesting cocktail party banter, but what does it mean? We certainly have correlation, but do we have relevance or causation?

Let’s focus first on causation. If a person who is against same-sex marriage eats a piece of raw tuna, will he then support same-sex marriage? Clearly, the answer is no.

With causation out the window, can the correlating data be relevant at all? Possibly. Perhaps these two data points, which seem to have nothing in common, indicate that if you are more open minded about food, you might also be open minded about other things that are new or less traditional. There have been a number of studies on how to achieve these very types of results; getting people to agree to small changes with the goal of having them accept a larger change.

Recap

With the goal of spotting patterns that will shed insight on predictable outcomes, collect data, sort it into like categories, and then see if there is meaningful correlation between two things. Ask yourself “why” this connection exists. Is it mere coincidence or is there more to the story?

Next, determine whether the highly correlative factor will cause you to achieve your target, or at least be of some relevance to your decision.

Be objective in assessing all of the information at your disposal. After you have “connected the dots,” step back to see if it paints a picture. Identifying patterns will allow you to predict your outcome and plan a clear path to achieving your goal.

For more in depth look at Pattern Recognition check out my book Never Be Wrong Again; Four Steps to Making Better Decisions in Work and Life.