At the third day en plein air with the NEX-6 I ran a quick-and-dirty comparative test with the Nikon D5100 and the AF-S 35mm ƒ/1.8G DX, roughly the same focal as the Sigma 30mm ƒ/2.8 DN A. The purpose of this test is also to “calibrate” the reviews I've found on the web, as there's a problem to solve (by the way, you might want to read this excellent post about lens reviews). Basically a review can be:
- Subjective, that is a personal perspective of the photographer; this unfortunately leads to a lot of ambiguity due to the fact either different photographers run the test, or the same person does, but he might have different evaluation metrics considering that we're comparing two worlds (DLRS and mirrorless). While there are some reputable reviewers out there, there are also lots of person who seem to misuse terms such as “superb” or “stellar”. For reputable people, there's also a possible language barrier thing: for instance, “decent” or “average” render as slightly negative in Italian and it's not probably what the reviewer from a different culture means.
- Objective by means of looking at photo qualities inspected in their details; this could be useful if the two lenses I'm interested in are compared in the same run, but unfortunately I couldn't find anything like that for the lenses I'd like to investigate.
- Strictly objective by means of scientifically measuring the optical qualities of the lens (such as the job done at DxOMark or Photozone.de); this has the advantage of results being expressed in terms of numbers, but unfortunately they cannot be directly compared because often the sensors used are different.
That's the reason for which I've run a side-by-side test by myself. The expectation after reading the other reviews I've read was that the Sigma is slightly ahead of the Nikkor: it was confirmed by the test, even though the difference looks even bigger than I thought.
Conditions of the test: tripod, self-timer, aperture-priority mode, ISO 100; furthermore, each photo was shot three times and the three shots were manually inspected to eventually reject a motion blurred one (it was never the case). Lenses were auto-focused at the beginning of the shot, then switched to manual to prevent the lens from re-focusing; no protection filter was used. There were some slight luminosity changes as the sky was clear, but with a few small clouds that were running and I didn't notice at the beginning; they however did cast their shadows on the background. The nearest subjects (the cranes) were roughly 600m from me.
The samples were imported in Adobe Lightroom 5, no post-processing was applied, and they were captured on the screen while in “Compare” mode.
The first set contains 1:1 crops from the center:
There's not much to comment here: at ƒ/4 the Sigma is clearly sharper, at ƒ/5.6 and ƒ/8 the two lenses are difficult to distinguish (perhaps the Nikkor leads), at ƒ/11 the Sigma seems to lead again; the best performance for both seems to be at ƒ/4 (this is consistent to measurements made at Photozone.de). All in all, the centre performance is roughly equivalent.
The second set are 1:1 crops from the right border:
Here the difference is relevant: the Sigma is always better, at ƒ/4 in a dramatic way, while the Nikkor reduces the gap at ƒ/11, anyway staying behind. The big difference is in chromatic aberration: it's clearly visible in the Nikkor (even though Adobe Lightroom 5 correction can make it disappear), while it's virtually invisible in the Sigma. While I knew the performance of the Nikkor, which I've been using for three years, I was quite surprised by the Sigma. In a first moment I thought I was doing something wrong (such as erroneously activating Lightroom corrections), but converting the Sony photos with dcraw I could confirm that the images came out of the camera as you can see. Further enlarging the images (up to 3:1 or 4:1) some green and magenta appears, even though I'm not sure whether it's chromatic aberration or demosaicing artefacts. Another possibility is that the NEX-6 performs some in-camera correction even for the raw file (I think it could be possible, still producing a Bayer-array arranged image), even though there's no mention of it in all the documentation and reviews I've found. As a final puzzling point, the tests at Photozone.de say that at ƒ/4 and ƒ/5.6 the Nikkor sharpness should be much more uniform between the centre and the border than the Sigma: this is not consistent with what I'm seeing. It's to be said that Photozone.de tested the previous version of the Sigma: perhaps the latter revision has been improved; furthermore they tested it on the NEX-7 which sports a 24MP sensor, instead of 16MP. To be verified.
As a final note, this side-by-side comparison confirms that the NEX-6, or the Adobe Lightroom 5 rendition, produces images with a reddish cast — something that I have to fix.
Unless I did a blatant mistake in the procedure, the test is won by the Sigma. This not only means that Sony sounds reliable and — at least for its best lenses — fears no comparison with the equivalent gear of the more established Nikon brand; but it also validates, in my perspective, the existing reviews of the system. The next step is to try the wide angle Sony E 4/10-18mm OSS, which I will also use to evaluate the stabilisation feature.
Updated on November 9, 2013: Actually SLRGear says that for chromatic aberration there is a significant improvement over the previous version, which probably explains the difference of my sample images with respect to PhotoZone.de:
We saw fairly significant chromatic aberration with the previous model, and despite a seemingly identical optical construction, the new Art-series model shows much better control of CA. While the older model averaged around a solid 6/100th of a percent of frame height, the new model cuts the average by more than half, averaging well below 3/100th of a percent of frame height. In fact, the maximum values of CA in the new lens are less than the average values from the old lens throughout all apertures.