Signal to Noise Ratio: Why some camera comparisons are wrong
An age-old measure of the performance of many types of recording is “signal to noise ratio”. In essence, it looks at how much noise there is compared to useful signal in any given medium, whether it’s an analogue transmission or even a digital recording. There’s one thing that I’ve noticed a lot recently, and that is when people compare cameras they often look at 1:1 crops to compare the results. In particular, they use this to compare noise performance between cameras. However, when comparing cameras of different resolutions, this may not actually give you an accurate comparison. The reason for this is that when doing this you’re only looking at one part of the equation, the noise, without considering the other part: the signal.
This is kind of a super nerdy piece, so you’ve been warned.
Consider a sensor of 24mp vs a sensor of 46mp. (I’m just taking this size as an example, as they’re the two most common sensor resolutions at this point) The 46mp sensor has almost twice the resolution of the 24mp version. If you look at 1:1 crops of both of these, there may be more noise apparent in the 46mp version. While there may be more noise at 1:1, overall, there may appear to be less noise. How is this possible you may ask? The reason for this is that there is considerably more “signal”, in this case, resolution. So if the 46mp version has, say 20% more noise, but nearly twice the “signal” the signal to noise ratio would actually be lower than the 24mp version.
In order to accurately, visually compare both sensors, in an ideal world you would downsample the larger version so that it matches the 24mp size. Or downsample both to some other resolution. I know of one site that does this regularly and has constantly caused controversy with its readers because of this, even though their methodology is reasonably sound in my opinion. Some people argue that this is unfair, because downsampling inherently denoises an image, but it only does this because the signal to noise ratio is inherently lower in higher resolution images, to begin with (depending on the sensor of course). Yet still, this causes a lot of people to get upset, because they don’t understand the reasoning behind it and are convinced that 1:1 crops are the most accurate measure of noise. But they’re not.
ISO performance is another issue, and what I’m going to say here will undoubtedly be a bit controversial too. Direct comparisons of ISO values aren’t necessarily correct either, and for similar reasons. Many cameras ISO values are slightly different in terms of the amount of light they record at set ISO values. Some sensors are more different than others, and while most are within a third of a stop of each other, some cameras are known for being a full stop different to others. Some will argue tooth and nail that its fair to just compare images of the same ISO value, regardless of the other exposure settings, because ISO is ISO. In my opinion, though, this isn’t a fair comparison, because the amount of light needed to reach a certain ISO value is different depending on the sensor.
Let me explain. Say you are shooting with a 50mm lens, you want to shoot at f/1.4 to maximise bokeh, and you want to keep your shutter speed at 1/50th of a second in order to minimise camera blur. On brand A you might need to set your ISO to 400 (let's say it’s a low light situation) but on brand B you may need to set it to 800. In this case Brand B requires twice as much light as brand A to get the exposure at the same settings. So in this case, if you were to compare the two sensors at 800 iso, for example, the amount of light involved will be different, and the exposure will be different, so it’s not a fair comparison. You should be comparing the noise performance for a fixed signal, so surely the actual signal should be the same? In my mind at least, the point of comparing ISO values is to compare how cameras perform under certain lighting conditions, so surely then the actual amount of light involved should be the factor, not some number the manufacturer has assigned?
In my opinion, sensors should have the ISO bias taken into account when comparing cameras, yet almost no one seems to do this. I saw one video review where the reviewer did take the difference into account, and they were attacked like crazy in the comments section. It may seem like this doesn’t matter in the real world, but it does, as demonstrated by the example above.
This isn’t meant as a slight against any particular brand. This affects all cameras and all comparisons. In my opinion, being properly objective benefits everyone. Someone needs to come up with a standard methodology for all this that’s scientifically sound, and transparent so people can’t complain. I guess this is what DXO Mark is supposed to be, but I guess that needs to be a little easier to understand and a little more transparent (and visual) for some people to understand.
Help Support the Blog
This blog (and my YouTube Channel) are my full time jobs, and are supported entirely by sales of my eBooks, presets and by support from readers like you. If you want to help keep this going then you can do so by…
Buy me a coffee / Tip Jar**. If you want to support the blog, you can buy me a coffee via PayPal with a one off donation to my PayPal tip jar.
You can support me on Patreon. If you like what I do here and find the information useful, then you can help by subscribing to me on Patreon
You can also show support by buying something from my from my Digital Download Store where I have Lightroom Presets, and e-books available for download. If you're a Fuji X-Trans shooter and Lightroom user, check out my guide to post processing X-Trans files in Lightroom. I also have a guides for processing X-Trans files in Capture One and Iridient Developer. For Sony Alpha shooters I have a new guide with tips on how to get the best from processing your A6000 Images in Lightroom.
If you like this post then you can see more of my work on Instagram, Twitter and Facebook. I also have a YouTube channel that you might like. You should also check out my other Photography Project: The Streets of Dublin. If you want to get regular updates, and notices of occasional special offers, and discounts from my store, then please sign up for the Newsletter.
