Thursday, March 27, 2014

Snapsort Doesn't Think For You

For the uninitiated, Snapsort is a website where users can compare specifications of two cameras, and the site will give all the comparisons with layman's annotations, and a verdict between the two cameras to decide which option is superior. The site has become unspeakably popular in the last few years, owing mainly to usage by people on bargain hunting or consumer tech sites like RFD or Slickdeals by people who know little about photography, and act as if the site's evaluation is the final word on the matter (example 1, example 2...I could go on). Thankfully dedicated photography forums aren't as rife with people trying to oversimplify what is in fact a difficult decision for people, since users there almost always have actual insight to give. But through Snapsort's oversimplification, all the site (and those who post its comparison links without commentary) manages to accomplish is an emphasis on the quantitative, when the qualitative is more important.


Obviously camera A's images look 50% better than camera B's.
It's for these same reasons that I don't put scores in my reviews - people end up not reading any important details and skip purely to the arbitrary numbers at the end. My case and point there is DxO, which as I've mentioned before is the posterboy for this type of over-simplification in the photography world. The biggest flaw with assigning a score to cameras as the sole basis decision making is that humans decision making is far more complex than that. When making a choice, we tend to weigh certain criteria more heavily than others, according to what we consider important. For example, someone who intends to use their camera for sports shooting will obviously care far more about framerate and autofocus than resolution or dynamic range. Likewise, travellers will prefer something small, light, and quiet for their needs. Additionally the site also has no way to account for the immeasurable properties of a camera, like ergonomics and ease of use. In short, Snapsort tries to make a subjective decision objective.

And that's the problem, the site pretty much has to pick a winner because that's what the user expects it to do. Even when two cameras are virtually identical, the site will choose for you. Case and point is a comparison between a Sony NEX-7 and a Hasselblad Lunar (link). For those unaware, the Hasselblad Lunar is just a rebadged NEX-7 made with 'luxury materials' and a price tag to match. Yet Snapsort, aside from having the wrong picture for the Lunar, picks the NEX-7 as the winner by a wide margin. Scroll down to the differences section and you'll see the text 'Snapsort is not aware of any important important differences between the Hasselblad Lunar and the Sony NEX-7'. Of course every site has its bugs, but here score gaps like this aren't uncommon when comparing similar cameras.

Worse still is how comparisons between brands is done. All the site takes into account is which camera is 'better', regardless of which lenses and accessories are available. As I mentioned earlier, the better camera choice for someone is highly context-sensitive. As is the case with all interchangeable lens cameras, making your decision on the body alone is foolish, as one really needs to consider what each brand's system can offer. Normally this wouldn't be a problem, but the heaviest users (or at least posters) of Snapsort are first time buyers who are getting entry-level cameras who won't know any better. It's always unfortunate when somebody buys a body only find out a few months later that the system they bought into doesn't have a particular lens or flash they need available, or if the system lacks any clear upgrade path.


Nowhere is this better illustrated than in the screenshot above, where Snapsort declared the winner to be the EOS M (review) by quite a significant margin over the comparable Panasonic GF5. While the M may be the better camera in many respects, it sadly belongs to a system that offers only two bodies and 3 lenses after 2 years on the market (no, adapted EF lenses don't count) whereas the GF5 gives its user access to a huge and complete lens selection, including a numerous fast pancake primes and a full range of zooms. Plus it was Sansmirror's entry level camera of the year in 2012. The cameras also have similar feature sets, yet there's a 24 point difference between them, indicating that Snapsort places perhaps a little too much emphasis on sensor performance.

Can Snapsort be a useful tool for comparing two cameras? Absolutely. The site is well designed and easy to use, and the plain English explanations of what each spec means will be a godsend for budding photographers. But it isn't a replacement for independent thought; it doesn't do your work for you, and it can't decide what you value as a feature. Those trying to decide on their first big camera purchase will be much better off gaining a deeper understanding of what they're getting into than trying to save a few hours by getting a site to do their research for them. Welcome to the world of consumer electronics, where there's no black and white regarding which product is 'better'. People love simple answers, but sometimes they just don't exist and you have to work to find them.

If you need to do a comparison of two cameras, DPReview's plain vanilla spec comparison page is an excellent tool, as is camerasize.com. They have the added benefit of allowing comparisons between more than 2 cameras, and show a wider range of specs. In the case of DPReview, most of the cameras have been given very detailed reviews as well. Snapsort, on the other hand, is photography's equivalent of Buzzfeed, wherein it acts as user-linked clickbait without any substance or justification for its rankings and ratings.

So please, next time you see someone drop a Snapsort link in a forum thread, do the OP a favour and add some useful info to the mix to compliment it and rationalize the decision.

No comments:

Post a Comment