Reblogged:I'll Take the Dentist With the Lower Rating, Please

[Gus Van Horn blog]

Drill more deeply than the numerical rating when choosing a dentist. (Image via Pixabay.)

Having just moved and in need of a routine cleaning, I recently had to choose a new dentist. Driving time is no small concern where I live, so I started by looking for practices nearby. I knew about two: One I had driven past a few times and another a neighbor had used. An internet search revealed that these are indeed the only ones that are very close. But the good news is that both are highly rated. All I had to do was pick the one with the better rating, right? Wrong. And this was despite my neighbor's good experience and the fact that the higher-rated practice had an order of magnitude more reviews than the one I chose. (The neighbor mentioned this dentist in passing, and at a time I wasn't thinking about choosing a dentist, so I didn't probe.)

Often, a large number of reviews can add credibility to a rating, but it's only one piece of information. The fact that a product or company is popular or rated highly is never as important as why, so I did what I always do when having to gauge unfamiliar choices: I read enough reviews, positive and negative to get a feel for how credible I found the rating. With the larger practice, the good reviews seemed a little too glowing, and some of the negative reviews credibly stated that the practice likes to sell unnecessary treatments. In contrast, the smaller practice had both more credible positive reviews and the kinds of negative reviews one would normally expect, ranging from non-alarming minor problems any business might sometimes have -- to reviews that really amount to, "I am a difficult person with unrealistic expectations, and this business was unlucky I came by."

So, yes, I'm going with the smaller, lower-rated practice, and it will need to earn future visits, although I am fairly confident it will.

This all reminds me of a couple of things I recently encountered pertinent to the same kind of problem, one of them being a recent Suzanne Lucas column at Inc., where she discusses "How to Spot Fake Glassdoor Reviews." Among other things, it is interesting to note that many companies do try to manipulate online reviews:

The Wall Street Journal reports that Guaranteed Rate CEO Victor Ciardelli, "instructed his team to enlist employees likely to post positive reviews." The result was a flood of positive reviews at the same time.

Because many people write a review after they leave a company, you'd expect a surge in reviews after a large layoff, but you would also expect them to lean to the negative side. (Although, it's perfectly possible to be laid off from a great place to work.) But a flood of positive reviews is a pretty good signal that there's a problem. [link omitted]

More important than learning how to spot this particular kind of problem or even knowing that it happens, is the rest of the article, whose main message I'd summarize as, "Using reviews takes much more than looking for a couple of numbers. Fortunately, there is more information there than you might think."

Another way of learning about whether you might want to hire someone or use a product is to use that fact that there are lots of proxies for reviews out there. As an example, consider the common problem of deciding whether to adopt a new technology. The following came from a very interesting Hacker News thread on the subject:

• Is there a clear reason the new tech exists? What differentiates it from its competitors? This alone rules out like 90% of new front-end web frameworks / widgets / plugins.
• Bonus points for tools whose authors have made the effort to explicitly compare it with competitor tools, particularly ones that acknowledge points where the competitor might have the advantage. ("Our new tech is better than existing old tech in every possible way" gets the side-eye from me; "Our new tech is better than existing old tech for these particular purposes, but old tech may still be more appropriate for these other purposes" goes a tremendous way towards confirming that the new tech has a real reason to exist.
• Is there documentation? Is it any good? This is a really low bar, but far too many new tools have no documentation at all ("just check out the source code") or have minimal, incomplete, or tautological docs ("bar.foo(): executes the foo method of bar"). A message board or IRC channel is nice, but not a substitute.
• How big is the API surface? Does it need to be that big? I tend to avoid tools where there are six different ways to do the same thing -- looking at you, Angular -- it suggests the developers are unfocused or in disagreement, and makes it harder to find support or documentation on any particular issue. Same thing if the API has undergone major breaking changes or paradigm shifts between versions (looking at you, Angular...)
• What does the tag look like on stackoverflow? This serves as a good indicator of whether the tech is too new or obscure to bother with, what the common pain points are, the average skill/knowledge level of its users, and whether help will be available if I get stuck when using it.
• Is there a relatively simple way to try it out? I'm much more likely to experiment with something where I can clone a repo and get going with simple but nontrivial example code; if I have to reconfigure half the settings on my machine just to get a hello world, I'm not going to bother. [format edits, bold added]

There are many other good suggestions in that thread that can be applied elsewhere, especially also self-knowledge (What problem do I have to solve, anyway? Do I even need new tech to solve it?) and using terms like "disadvantages" with the search term for the thing in question.

Evaluating the unfamiliar is something too many people try to do quickly, but in the wrong way. With a little bit of thought, and perhaps by approaching it a little like a puzzle, one can do so efficiently and with a degree of confidence that rises the more one can integrate the new knowledge with what one already knows.

-- CAV

Link to Original