Lens Tests - A pragmatic guide to using them.

Lens tests can be found all over the Internet, as well as in photography magazines. Can you trust them? How much weight should you give them when deciding on a purchase?

A simple fact to remember is that a lens test is as much a test of the tester, as it is of the lens.

Lens tests take several forms. Typical ones are:

• Online reviews by the lens owners, typically in a 'free for all' forum
• Online reviews by a single, often professional, photographer
• Online opinions (as opposed to full reviews) in photography discussion forums
• Detailed reports, complete with MTF resolution information, either online or in photography magazines.

I refer to quite a few of these resources, especially when considering a major purchase. However, there are a few things to always bear in mind.

Online reviews by the lens owners, typically in a 'free for all' forum

In these forums, lens owners are free to say whatever they like about their lenses. And they often do! The danger with these opinions is that human beings tend to rationalise - if the lens gives bad results, then they blame the lens. Of course, the lens may be bad (or they have a bad copy) but there are other factors, including the reviewers own ability and interpretation of the results. Sometimes comments verge on the ridiculous - I've read reviews where the lens is blamed for sensor noise (he used the lens at ISO 1600 and the image was noisy, so the lens is blamed!), sensor dust (he saw a few particles of dust in the lens and blames the sensor dust spots appearing in the image on it), etc.

Often, reviewers say obvious, but unobjective, things. They buy a budget lens and then complain it's 'Not an L lens' (L being Canon's top quality lens designation). Or they buy an F4 lens and then complain it's not an F2.8 lens. Well, if you want an L lens, buy one. If you want an F2.8 lens, buy one. Don't knock the one you bought just because it didn't magically transform itself into something else.

At the other end of the scale, a reviewer, having spent many (possibly a thousand or more) $$$ on a lens, then feels the need to justify the purchase. So, he may be inclined to over-praise the lens or ignore its shortcomings.

One particular area where caution is needed is when the user says the lens is 'soft', especially at the telephoto end. Softness can be caused by all sorts of things - poor focus and poor lens technique being the main two. True, many zoom lenses have less resolution at the longest focal lengths, but many amateurs have no idea how to hand-hold a 300MM+ lens.

I would takes these reviews with a pinch of salt - over time, when enough reviews are written on a particular lens, the average of the comments may be an accurate reflection of it.

'Laboratory' reviews

At the other end of the spectrum are reviews given by photo magazines and online sites that claim to have tested the lens under laboratory conditions. They produce MTF data, which is (or should be) a very accurate reflection of the lens' optical resolving power.

Generally speaking, these reviews are more objective than users' opinions. However, it still requires that the test be set up correctly, so human error can still be a factor. Another thing to bear in mind is that, unfortunately, there can be significant differences between the copy of the lens they are testing and the one you end up buying. A number of factors are involved - there may be quality control issues causing different batches of lenses to be of different qualities, lenses may be damaged in transit, etc. I often compare the results of two or more of these reviews, and when significant differences exist between them I consider that at least one of the reviews is flawed - either the reviewer made an error or he had a bad sample of the lens.

Professional Photographer's Reviews

Quite a few pro photographers publish 'field tests' of their lenses. These can be quite helpful as a pro is possibly going to be more objective in his opinions and more skilled at using the lens. And there is a world of difference between using a lens in the real world and testing it in a lab. Factors such as handling, ease of (or otherwise) of using the various switches on the lens, ease of hand-holding, speed of focus and other factors are much more important in the field than in the lab. Also, labs make a lot of noise about MTF (definition) figures. In the real world, the different resolving abilities of lenses may not be that important. If one lens is technically 10% sharper than another, would you actually see the difference (without using a microscope)?

So a field test may be the best test of all.

Links

Fred Miranda - lens and camera reviews posted by users. Generally good in that there are many reviews but always take with a pinch of salt...

Photozone - a fairly comprehensive set of lens tests. Be aware that the lens tests are carried out on a camera using an APS-C sized chip. If your SLR has one of these, then the tests are relevant. If you use a full frame digital slr or a film slr then these tests are worthless, at least as far as vignetting, distortion, chromatic aberration and edge resolution are concerned. At times their tests differ from my experience and from other lab results, but that may be due to the factors mentioned above with variations between lens samples.

Bob Atkins - some good reviews by a decent photographer.

The Luminous Landscape - excellent reviews and field tests of all things photographic.

Naturescapes.net - Excellent discussion forums, with objective opinions of cameras and lenses. Excellent galleries, although small, web posted images are not a true reflection of a len's quality, as even a softish image can be made to look sharp when it is small.