I don't think that I'm alone in feeling like TripAdvisor rankings have became relatively useless. Sure, TripAdvisor is the largest database of travel-related information and reviews on hotels, restaurants, and activities across the globe, but in my opinion there are a couple huge problems plaguing it. First, it's well-known that TripAdvisor is filled with completely fake reviews. These fabricated reviews work both ways - positive ones to boost ranks, and negative reviews to sabotage the ranks of competitors. A tell-tale sign of this are extreme reviews posted by new members with just 1 or 2 total reviews.
Second, even if we focus on the genuine user reviews, I would posit that 25%-50% of them are heavily biased. I use the word "heavily" because every single person, including myself, has biases that find their way into reviews that they write. But I'm talking about reviews so biased that they simply provide little to no indication of the potential experience that a typical customer can expect.
It's a shame that we can't take these ratings at face-value. After all, we all like ranking things and compiling lists. But nowadays, I've started to go about my online research in a different way. I typically form a list of the top 5-10 candidates by cross-referencing rankings on multiple sites, and ultimately perform my own qualitative analysis of reviews which involves filtering out 1-star reviews, taking 5-star reviews with a grain of salt, and weeding through the rest to really get a sense for the "real" customer experience.
That's a lot more work than simply going by the rankings shown on the screen in front of you. But I still remember an instance back in 2010 when my wife and I went on TripAdvisor to search for a restaurant in Athens, Greece for dinner. Out of over 1,500 restaurants in TripAdvisor's database, we decided to eat at the #2 ranked restaurant in ALL of Athens. When we turned up, we found a complete hole-in-the-wall, which is perfectly fine if the food is great. Except it wasn't, and instead we found ourselves in a foreign country eating at a tourist trap with overpriced, crappy food. To this day, it just boggles our mind how a place like that came to be so highly ranked.
I don't mean to pick on TripAdvisor, since the same can be said about Yelp and other review aggregator sites. Today's reality is that you simply cannot trust the ranks that these sites give. That's where the idea for for this post came from. I wanted to do a bit of digging into the human psyche to see how cognitive biases manifest themselves in reviews, with the goal of being able to better identify and ultimately ignore the heavily biased portions of user reviews, which sometimes may be the entire thing.
Here's a long list of cognitive biases from which I pulled out the ones that I've most commonly found in reviews. Note that I am not a psychologist, nor do I pretend to play one on TV, so let me know if I have completely mis-interpreted any of these. As you can see below, biases can influence reviews both positively and negatively.
#1. Anchoring effect: The tendency to rely too heavily, or "anchor," on one trait or piece of information when making decisions (usually the first piece of information that we acquire on that subject)
What's that age old saying? Oh yeah, "you only get once chance to make a first impression". How many reviews have you read where the person was completely and utterly fixated on something that happened, often related to a first impression. Even if a service recovery was adequate and successful, they simply can't let it go, no matter how insignificant the incident may be in the overall scheme of things.
Example of what to look for: "our entire stay was ruined right off the bat as it took over over 5 minutes for someone to greet us and help with our bags", "there was one instance where a staff member did not greet me properly by my full name, so in general the staff at this hotel were completely unprofessional"
#2. Bandwagon effect: The tendency to do (or believe) things because many other people do (or believe) the same
Society can be harsh on people with independent and unique perspectives - the path of least resistance is to just follow the herd. This can work both ways based on popular sentiment at the time - overwhelmingly positive feedback can lead to continued grade-inflation, and negative buzz can cause others to pile-on. More than anything I want trustworthy and independent reviews of that customer's particular experience, not simply a carbon copy of what everyone else has been saying.
Example of what to look for: "consistent with what others have mentioned", "similar experience to previous reviews", "agree with what others have posted"
#3. Halo effect: The tendency for a person's positive or negative traits to "spill over" from one personality area to another in others' perceptions of them
This is harder to spot in an isolated review, but really surfaces when some of the viral reviews lead to a conversation on forums such as FlyerTalk. How many times have you seen a review titled "STARWOOD HOTELS RUINED MY HONEYMOON NEVER STAY AT STARWOOD EVER AGAIN". When the onion gets peeled back, often times the reviewer will admit that the overwhelming sense of negativity was a chain reaction starting with some form of stress as a catalyst, often times related to a bumpy situation with their significant other. Once that chain reaction starts, it's all downhill from there.
Example of what to look for: "I was already stressed from dealing with", "my significant other wasn't happy with the situation and started crying", "we were already running late due to a delayed flight"
#4. Recency bias: focus on "what's happened lately" when evaluation or judging something
I'm really guilty of this one. How many times have you heard about an experience being the "best ever" or "top 3" in the reviewer's life? I mean, really, the best ever? If you gave someone completely identical experiences, one which occurred yesterday and one a year before that, the vast majority of people would think more favorably on the more recent one. That's why we tend to rank recent experiences higher than others, even if they weren't necessarily better.
Example of what to look for: "top 5 meals I've ever had", "best hotel I have ever stayed at"
#5. Money illusion: The tendency to concentrate on the nominal value (face value) of money rather than its value in terms of purchasing power
I've noticed this a lot on TripAdvisor - in almost any city in the world, you won't find the most expensive hotels ranked at the top even in cases where they are widely regarded as the "best" by industry experts and other experienced travelers. Why is that? My hunch is that for expensive properties beyond a certain $ threshold (both from a cash or points perspective), no matter how amazing the experience was and no matter how much fun that person had, the value will never live up to the price tag.
Example of what to look for: "this was the best hotel I've ever stayed at, but I still can't believe how much it cost", "for the price I paid I could have gotten twice as many nights at a cheaper hotel, so I just don't feel like it was a good value"
#6. Post-purchase rationalization: The tendency to persuade oneself through rational argument that a purchase was a good value
Related to #4 but in the opposite direction, almost forcing yourself to feel good about a purchase that you made and justifying that it was a good value.
Example of what to look for: "I'm not sure about the value here, but I'll give it the benefit of the doubt"
#7. Status-quo bias: The tendency to like things to stay relatively the same (see also loss aversion, endowment effect, and system justification)
We see this a lot with hotel reviews, since we all have our go-to loyalty program, and within each program even our go-to hotels. When we take a chance on something different and break away from the status-quo, we always find ourselves comparing to what we're used to.
Example of what to look for: "compared to where I usually stay", "I was reluctant to try a non-Starwood hotel", "the amenities looked just as nice but were completely different from what I'm used to, not sure how I feel about that"
#8. Gambler's fallacy: The tendency to think that future probabilities are altered by past events, when in reality they are unchanged
The actuary in me would say that every situation has a series of possible outcomes, with each outcome having a certain probability of occurring. So if you think about it from that perspective, each time you stay at a hotel, you have a X% probability that it will be a "good" stay vs. a "bad" stay. Better hotels are able to get that percentage closer to 100%, but in reality any hotel stay or meal has a greater than 0% chance of going badly. I have seen reviews where people expect every stay to be identical to prior experiences, which can even lead to bad reviews when they aren't given as good of an upgrade as a prior time.
Example of what to look for: "I wasn't upgraded to as nice a room as my last trip", "my burger wasn't cooked as well this time", "the person staying next door was much louder than last time"
It would make our lives so much easier if rankings on sites like TripAdvisor were credible, but the reality is that they are not. That's why, at least for me, travel-related research has become much more qualitative. That means scouring through actual user reviews, which invariably contain biases that we all have in different shapes and forms, and trying to really figure out what the "typical" customer can expect. It's no easy task, but until there's a better option, I'm willing to put in the time and effort to avoid being duped by false rankings.
I'm interested in hearing how you view rankings and filter through the good, the bad, and the fake reviews posted on sites like TripAdvisor.
Please follow up with the "quantitative analysis" mentioned - inquiring minds wish to know!
ReplyDelete