Measurement and Sample Variation
Prepared 2018-10-26
by Bill Claff
Last revised 2018-11-01
Introduction
I have been publishing a variety of sensor measurements since 2007 and have now accumulated information on about 250 cameras.
From time to time people inquire about the accuracy of the measurements and about sample variation.
With the help of LensProToGo (a sister company to LensRentals) I was able to gather data to get some clarity on this issue.
Nine Nikon D500 bodies were made available.
I collected eight sets of raw files from one body and one set of raw data from
each of the other eight bodies.
From the first set I hoped to quantify measurement variation;
how repeatable my testing is.
And from the second set to quantify sample variation; how much variation there
is between cameras.
Read Noise
The first test is for Read Noise which is measured from raw
files and reported in Digital Numbers (DN).
The published Read Noise in DNs for the Nikon D500 at PhotonsToPhotos looks
like this:
You can see other cameras and do comparisons at by visiting the interactive chart.
Read noise is not directly relevant to photographers but is a
strong influence on a number of factors that are of interest.
We can get some useful information directly from the chart.
We see confirmation that ISO 100 is the base ISO setting; the highest ISO
setting with the lowest read noise.
We also see a drop in read noise at ISO 400 due to the fact that this is a dual
conversion gain sensor.
For more technical background on dual conversion gain see this Aptina DR-Pix Technology White Paper.
The straightness of the lines, particularly at the intermediate ISO settings,
is also meaningful to someone experienced at reading such charts.
So, what did testing with multiple bodies reveal? Here is the detailed data for read noise measurement variation:
|
ISO |
Read Noise (DN) |
|
Read Noise (DN) |
|||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
50 |
1.297 |
1.322 |
1.324 |
1.337 |
1.345 |
1.337 |
1.335 |
1.346 |
|
1.328 |
1.297 |
1.345 |
|
62 |
1.316 |
1.312 |
1.333 |
1.340 |
1.334 |
1.347 |
1.335 |
1.341 |
|
1.331 |
1.312 |
1.347 |
|
79 |
1.321 |
1.310 |
1.317 |
1.340 |
1.327 |
1.354 |
1.334 |
1.332 |
|
1.329 |
1.310 |
1.354 |
|
100 |
1.306 |
1.309 |
1.317 |
1.340 |
1.322 |
1.340 |
1.331 |
1.348 |
|
1.324 |
1.306 |
1.340 |
|
125 |
1.556 |
1.551 |
1.556 |
1.583 |
1.591 |
1.583 |
1.560 |
1.575 |
|
1.568 |
1.551 |
1.591 |
|
158 |
1.870 |
1.892 |
1.914 |
1.911 |
1.911 |
1.903 |
1.904 |
1.927 |
|
1.901 |
1.870 |
1.914 |
|
200 |
2.230 |
2.244 |
2.250 |
2.272 |
2.276 |
2.243 |
2.272 |
2.283 |
|
2.255 |
2.230 |
2.276 |
|
251 |
2.735 |
2.797 |
2.815 |
2.722 |
2.742 |
2.718 |
2.724 |
2.789 |
|
2.751 |
2.718 |
2.815 |
|
317 |
3.349 |
3.358 |
3.414 |
3.429 |
3.373 |
3.417 |
3.397 |
3.435 |
|
3.391 |
3.349 |
3.429 |
|
400 |
1.977 |
1.996 |
1.946 |
1.977 |
2.028 |
1.958 |
2.009 |
1.992 |
|
1.984 |
1.946 |
2.028 |
|
503 |
2.418 |
2.465 |
2.448 |
2.495 |
2.405 |
2.434 |
2.459 |
2.452 |
|
2.446 |
2.405 |
2.495 |
|
634 |
2.957 |
3.059 |
2.989 |
3.009 |
3.078 |
3.055 |
2.967 |
3.018 |
|
3.016 |
2.957 |
3.078 |
|
800 |
3.671 |
3.675 |
3.694 |
3.692 |
3.668 |
3.681 |
3.717 |
3.743 |
|
3.685 |
3.668 |
3.717 |
|
1007 |
4.427 |
4.540 |
4.502 |
4.499 |
4.508 |
4.407 |
4.543 |
4.583 |
|
4.490 |
4.407 |
4.543 |
|
1269 |
5.537 |
5.552 |
5.480 |
5.507 |
5.610 |
5.460 |
5.429 |
5.536 |
|
5.511 |
5.429 |
5.610 |
|
1600 |
6.716 |
6.687 |
6.986 |
6.717 |
6.870 |
6.788 |
6.990 |
7.042 |
|
6.822 |
6.687 |
6.990 |
|
2015 |
8.121 |
8.154 |
8.700 |
8.322 |
8.385 |
8.204 |
8.390 |
8.224 |
|
8.325 |
8.121 |
8.700 |
|
2539 |
10.415 |
10.058 |
10.343 |
10.277 |
10.442 |
10.440 |
10.174 |
10.072 |
|
10.307 |
10.058 |
10.442 |
|
3200 |
12.452 |
12.248 |
12.258 |
12.233 |
12.571 |
12.625 |
12.434 |
12.428 |
|
12.403 |
12.233 |
12.625 |
|
4031 |
15.385 |
15.867 |
15.333 |
15.747 |
15.509 |
15.623 |
15.577 |
15.875 |
|
15.577 |
15.333 |
15.867 |
|
5079 |
19.356 |
19.823 |
19.189 |
18.867 |
19.210 |
19.406 |
19.400 |
19.156 |
|
19.322 |
18.867 |
19.823 |
|
6400 |
23.878 |
24.094 |
23.383 |
23.570 |
23.832 |
24.358 |
24.284 |
23.168 |
|
23.914 |
23.383 |
24.358 |
|
8063 |
29.671 |
28.847 |
28.873 |
29.913 |
28.883 |
28.776 |
29.778 |
29.743 |
|
29.249 |
28.776 |
29.913 |
|
10159 |
35.698 |
35.302 |
36.209 |
36.404 |
36.992 |
37.055 |
36.666 |
36.104 |
|
36.332 |
35.302 |
37.055 |
|
12800 |
44.060 |
46.586 |
46.167 |
46.343 |
45.201 |
45.814 |
46.543 |
45.882 |
|
45.816 |
44.060 |
46.586 |
|
16126 |
56.872 |
56.916 |
57.938 |
56.123 |
56.848 |
57.412 |
57.822 |
56.387 |
|
57.133 |
56.123 |
57.938 |
|
20318 |
71.240 |
71.046 |
70.325 |
70.466 |
71.345 |
70.546 |
72.709 |
72.918 |
|
71.097 |
70.325 |
72.709 |
|
25600 |
83.562 |
84.150 |
82.828 |
83.847 |
83.853 |
84.648 |
84.845 |
83.693 |
|
83.962 |
82.828 |
84.845 |
|
32253 |
104.931 |
105.759 |
104.339 |
104.974 |
106.706 |
106.707 |
105.314 |
105.722 |
|
105.533 |
104.339 |
106.707 |
|
40637 |
131.626 |
131.142 |
131.902 |
132.454 |
132.036 |
131.743 |
131.056 |
132.382 |
|
131.709 |
131.056 |
132.454 |
|
51200 |
159.441 |
162.557 |
164.273 |
164.124 |
164.866 |
164.951 |
164.768 |
164.751 |
|
163.568 |
159.441 |
164.951 |
|
64507 |
197.914 |
203.097 |
203.761 |
202.546 |
204.949 |
200.274 |
204.216 |
203.646 |
|
202.394 |
197.914 |
204.949 |
|
81274 |
245.719 |
247.435 |
242.537 |
252.568 |
247.554 |
248.263 |
246.259 |
248.837 |
|
247.191 |
242.537 |
252.568 |
|
102400 |
299.422 |
300.345 |
298.828 |
296.090 |
303.553 |
296.891 |
296.938 |
299.303 |
|
298.867 |
296.090 |
303.553 |
|
204800 |
497.311 |
512.157 |
498.913 |
511.336 |
503.063 |
512.372 |
513.006 |
510.106 |
|
506.880 |
497.311 |
513.006 |
|
409600 |
902.947 |
880.923 |
877.589 |
862.757 |
901.813 |
891.230 |
885.480 |
894.146 |
|
886.105 |
862.757 |
902.947 |
|
819200 |
1626.139 |
1568.881 |
1582.377 |
1616.346 |
1587.605 |
1609.370 |
1647.454 |
1645.210 |
|
1605.453 |
1568.881 |
1647.454 |
|
1638400 |
3013.404 |
2980.689 |
3067.988 |
3016.601 |
3065.618 |
3105.912 |
3116.458 |
3099.209 |
|
3052.381 |
2980.689 |
3116.458 |
And here is the detailed data for read noise sample variation:
|
One Dataset for Each of Eight
Nikon D500 Bodies |
||||||||||||
|
ISO |
Read Noise (DN) |
|
Read Noise (DN) |
|||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
50 |
1.327 |
1.333 |
1.347 |
1.321 |
1.341 |
1.324 |
1.286 |
1.330 |
|
1.326 |
1.286 |
1.347 |
|
62 |
1.316 |
1.318 |
1.345 |
1.331 |
1.344 |
1.326 |
1.293 |
1.341 |
|
1.325 |
1.293 |
1.345 |
|
79 |
1.307 |
1.306 |
1.328 |
1.358 |
1.333 |
1.327 |
1.314 |
1.339 |
|
1.325 |
1.306 |
1.358 |
|
100 |
1.315 |
1.311 |
1.336 |
1.331 |
1.340 |
1.319 |
1.295 |
1.320 |
|
1.321 |
1.295 |
1.340 |
|
125 |
1.608 |
1.546 |
1.597 |
1.577 |
1.588 |
1.577 |
1.554 |
1.604 |
|
1.578 |
1.546 |
1.608 |
|
158 |
1.935 |
1.911 |
1.940 |
1.916 |
1.957 |
1.935 |
1.904 |
1.956 |
|
1.928 |
1.904 |
1.957 |
|
200 |
2.363 |
2.312 |
2.340 |
2.375 |
2.367 |
2.345 |
2.242 |
2.300 |
|
2.335 |
2.242 |
2.375 |
|
251 |
2.890 |
2.778 |
2.881 |
2.940 |
2.874 |
2.880 |
2.788 |
2.794 |
|
2.862 |
2.778 |
2.940 |
|
317 |
3.509 |
3.479 |
3.520 |
3.528 |
3.477 |
3.433 |
3.426 |
3.457 |
|
3.482 |
3.426 |
3.528 |
|
400 |
2.000 |
2.003 |
2.081 |
2.087 |
2.015 |
1.947 |
1.944 |
1.990 |
|
2.011 |
1.944 |
2.087 |
|
503 |
2.429 |
2.447 |
2.579 |
2.504 |
2.519 |
2.446 |
2.391 |
2.440 |
|
2.474 |
2.391 |
2.579 |
|
634 |
2.994 |
2.978 |
3.170 |
3.081 |
3.037 |
2.996 |
2.896 |
2.993 |
|
3.022 |
2.896 |
3.170 |
|
800 |
3.693 |
3.675 |
3.790 |
3.881 |
3.820 |
3.686 |
3.646 |
3.655 |
|
3.741 |
3.646 |
3.881 |
|
1007 |
4.503 |
4.442 |
4.683 |
4.708 |
4.719 |
4.570 |
4.489 |
4.473 |
|
4.588 |
4.442 |
4.719 |
|
1269 |
5.635 |
5.547 |
5.719 |
5.786 |
5.738 |
5.612 |
5.531 |
5.428 |
|
5.652 |
5.531 |
5.786 |
|
1600 |
6.711 |
7.044 |
7.108 |
7.132 |
7.042 |
6.800 |
6.790 |
6.919 |
|
6.947 |
6.711 |
7.132 |
|
2015 |
8.485 |
8.371 |
8.828 |
8.675 |
8.606 |
8.545 |
8.256 |
8.097 |
|
8.538 |
8.256 |
8.828 |
|
2539 |
10.157 |
10.309 |
10.984 |
10.841 |
10.357 |
10.593 |
10.459 |
10.184 |
|
10.529 |
10.157 |
10.984 |
|
3200 |
12.612 |
12.372 |
12.981 |
13.231 |
13.065 |
12.749 |
12.857 |
12.370 |
|
12.838 |
12.372 |
13.231 |
|
4031 |
15.660 |
15.555 |
16.171 |
16.309 |
16.007 |
15.954 |
15.384 |
15.326 |
|
15.863 |
15.384 |
16.309 |
|
5079 |
19.000 |
19.245 |
20.333 |
20.481 |
19.708 |
19.065 |
18.824 |
19.503 |
|
19.522 |
18.824 |
20.481 |
|
6400 |
23.684 |
23.755 |
25.377 |
24.565 |
24.017 |
24.346 |
23.901 |
23.004 |
|
24.235 |
23.684 |
25.377 |
|
8063 |
29.634 |
28.609 |
30.814 |
30.116 |
30.157 |
29.605 |
29.479 |
28.493 |
|
29.774 |
28.609 |
30.814 |
|
10159 |
36.616 |
37.028 |
38.476 |
38.625 |
37.543 |
37.077 |
36.062 |
35.876 |
|
37.347 |
36.062 |
38.625 |
|
12800 |
45.355 |
46.131 |
48.254 |
48.288 |
45.646 |
45.902 |
46.622 |
44.902 |
|
46.600 |
45.355 |
48.288 |
|
16126 |
56.018 |
56.512 |
59.941 |
59.584 |
58.219 |
58.497 |
57.224 |
56.919 |
|
57.999 |
56.018 |
59.941 |
|
20318 |
70.868 |
70.941 |
74.446 |
74.785 |
71.449 |
71.701 |
70.729 |
69.882 |
|
72.131 |
70.729 |
74.785 |
|
25600 |
84.054 |
83.852 |
89.036 |
87.274 |
87.234 |
83.329 |
83.897 |
83.636 |
|
85.525 |
83.329 |
89.036 |
|
32253 |
105.872 |
104.711 |
111.835 |
108.732 |
108.680 |
106.477 |
104.732 |
104.258 |
|
107.291 |
104.711 |
111.835 |
|
40637 |
131.789 |
130.104 |
138.397 |
135.913 |
136.766 |
132.679 |
131.505 |
130.408 |
|
133.879 |
130.104 |
138.397 |
|
51200 |
164.410 |
164.558 |
175.309 |
171.017 |
167.966 |
160.951 |
165.752 |
162.511 |
|
167.137 |
160.951 |
175.309 |
|
64507 |
203.878 |
203.282 |
213.465 |
209.666 |
208.799 |
203.155 |
200.608 |
198.725 |
|
206.122 |
200.608 |
213.465 |
|
81274 |
250.041 |
246.392 |
260.326 |
256.010 |
256.257 |
247.714 |
247.367 |
245.227 |
|
252.015 |
246.392 |
260.326 |
|
102400 |
298.603 |
296.809 |
315.078 |
308.213 |
302.694 |
302.211 |
294.201 |
293.915 |
|
302.544 |
294.201 |
315.078 |
|
204800 |
500.129 |
514.544 |
532.070 |
527.505 |
515.184 |
504.355 |
511.762 |
497.097 |
|
515.079 |
500.129 |
532.070 |
|
409600 |
910.687 |
889.512 |
934.463 |
901.945 |
907.114 |
882.334 |
880.932 |
883.772 |
|
900.998 |
880.932 |
934.463 |
|
819200 |
1589.257 |
1599.712 |
1700.273 |
1637.190 |
1609.225 |
1624.230 |
1625.268 |
1618.001 |
|
1626.451 |
1589.257 |
1700.273 |
|
1638400 |
3028.489 |
2987.952 |
3248.938 |
3215.677 |
3070.537 |
2999.510 |
3075.645 |
3044.515 |
|
3089.535 |
2987.952 |
3248.938 |
You're welcome to copy and paste data out of these tables for
your own analysis.
Here's what is looks like in chart form:
You can play with the interactive chart for yourself here.
It's hard to see, but error bars have been added to show the
measurement and sample variation ranges.
We can zoom in on the chart by drawing a rectangle with the mouse:
Now the error bars showing the measurement and sample
variation can be seen.
Note that the horizontal gridlines are 1/3 stop apart and that generally the
error bars are well under 1/6 stop in height.
So it appears that read noise measurements are quite accurate
and not particularly subject to sample variation.
As is often the case there are some exceptions that are beyond the scope of
this article.
Another measure, Input-referred Read Noise is directly related to Read Noise in
DNs and has identical variation.
Photographic Dynamic Range (PDR)
The second test is for Photographic Dynamic Range (PDR).
This is a practical measure which incorporates all noise sources and targets a
Signal to Noise Ratio (SNR) that is visually acceptable under standard viewing
conditions.
PDR is the most popular measure published at PhotonsToPhotos.
The published PDR for the Nikon D500 looks like this:
You can see other cameras and do comparisons at by visiting the interactive chart.
For this particular sensor design we expect a pretty straight
line.
Note the slight boost in PDR at ISO 400 where the High Conversion Gain (HCG) of
this dual conversion gain sensor kicks in.
Here is the detailed data for PDR measurement variation:
|
Eight Datasets Using One Nikon
D500 Body |
||||||||||||
|
ISO |
PDR (stops) |
|
PDR (stops) |
|||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
50 |
10.67 |
10.67 |
10.65 |
10.7 |
10.66 |
10.68 |
10.7 |
10.68 |
|
10.67 |
10.65 |
10.70 |
|
62 |
10.69 |
10.68 |
10.69 |
10.7 |
10.65 |
10.69 |
10.66 |
10.65 |
|
10.67 |
10.65 |
10.69 |
|
79 |
10.65 |
10.6 |
10.66 |
10.7 |
10.65 |
10.68 |
10.68 |
10.67 |
|
10.66 |
10.60 |
10.68 |
|
100 |
10.68 |
10.68 |
10.69 |
10.7 |
10.65 |
10.67 |
10.67 |
10.66 |
|
10.67 |
10.65 |
10.69 |
|
125 |
10.4 |
10.37 |
10.37 |
10.4 |
10.35 |
10.37 |
10.36 |
10.37 |
|
10.37 |
10.35 |
10.40 |
|
158 |
10.05 |
10.05 |
10.07 |
10.1 |
10.06 |
10.07 |
10.07 |
10.08 |
|
10.07 |
10.05 |
10.08 |
|
200 |
9.8 |
9.78 |
9.79 |
9.8 |
9.77 |
9.78 |
9.79 |
9.79 |
|
9.79 |
9.77 |
9.80 |
|
251 |
9.49 |
9.5 |
9.5 |
9.5 |
9.51 |
9.47 |
9.49 |
9.5 |
|
9.50 |
9.47 |
9.51 |
|
317 |
9.19 |
9.14 |
9.19 |
9.17 |
9.16 |
9.18 |
9.17 |
9.16 |
|
9.17 |
9.14 |
9.19 |
|
400 |
9.19 |
9.17 |
9.2 |
9.2 |
9.17 |
9.17 |
9.18 |
9.17 |
|
9.18 |
9.17 |
9.20 |
|
503 |
8.88 |
8.86 |
8.87 |
8.87 |
8.86 |
8.87 |
8.88 |
8.89 |
|
8.87 |
8.86 |
8.89 |
|
634 |
8.54 |
8.53 |
8.5 |
8.52 |
8.53 |
8.54 |
8.5 |
8.51 |
|
8.52 |
8.50 |
8.54 |
|
800 |
8.21 |
8.2 |
8.17 |
8.15 |
8.16 |
8.19 |
8.2 |
8.18 |
|
8.18 |
8.15 |
8.21 |
|
1007 |
7.83 |
7.81 |
7.85 |
7.83 |
7.83 |
7.87 |
7.84 |
7.85 |
|
7.84 |
7.81 |
7.87 |
|
1269 |
7.52 |
7.52 |
7.56 |
7.5 |
7.46 |
7.5 |
7.53 |
7.5 |
|
7.51 |
7.46 |
7.56 |
|
1600 |
7.21 |
7.18 |
7.21 |
7.18 |
7.22 |
7.22 |
7.21 |
7.21 |
|
7.21 |
7.18 |
7.22 |
|
2015 |
6.88 |
6.85 |
6.89 |
6.87 |
6.86 |
6.89 |
6.91 |
6.9 |
|
6.88 |
6.85 |
6.91 |
|
2539 |
6.57 |
6.55 |
6.57 |
6.53 |
6.53 |
6.57 |
6.54 |
6.58 |
|
6.56 |
6.53 |
6.58 |
|
3200 |
6.24 |
6.2 |
6.23 |
6.19 |
6.24 |
6.23 |
6.23 |
6.24 |
|
6.23 |
6.19 |
6.24 |
|
4031 |
5.92 |
5.87 |
5.89 |
5.87 |
5.87 |
5.9 |
5.89 |
5.91 |
|
5.89 |
5.87 |
5.92 |
|
5079 |
5.57 |
5.51 |
5.59 |
5.55 |
5.52 |
5.54 |
5.53 |
5.57 |
|
5.55 |
5.51 |
5.59 |
|
6400 |
5.2 |
5.21 |
5.24 |
5.23 |
5.22 |
5.24 |
5.25 |
5.24 |
|
5.23 |
5.20 |
5.25 |
|
8063 |
4.9 |
4.87 |
4.91 |
4.89 |
4.86 |
4.9 |
4.86 |
4.88 |
|
4.88 |
4.86 |
4.91 |
|
10159 |
4.55 |
4.53 |
4.58 |
4.57 |
4.57 |
4.58 |
4.55 |
4.56 |
|
4.56 |
4.53 |
4.58 |
|
12800 |
4.22 |
4.19 |
4.22 |
4.25 |
4.21 |
4.25 |
4.23 |
4.22 |
|
4.22 |
4.19 |
4.25 |
|
16126 |
3.9 |
3.87 |
3.92 |
3.91 |
3.87 |
3.91 |
3.9 |
3.88 |
|
3.90 |
3.87 |
3.92 |
|
20318 |
3.56 |
3.54 |
3.58 |
3.52 |
3.55 |
3.58 |
3.58 |
3.54 |
|
3.56 |
3.52 |
3.58 |
|
25600 |
3.26 |
3.21 |
3.24 |
3.25 |
3.22 |
3.26 |
3.25 |
3.25 |
|
3.24 |
3.21 |
3.26 |
|
32253 |
2.89 |
2.88 |
2.92 |
2.89 |
2.86 |
2.92 |
2.91 |
2.9 |
|
2.90 |
2.86 |
2.92 |
|
40637 |
2.58 |
2.52 |
2.58 |
2.56 |
2.57 |
2.55 |
2.58 |
2.57 |
|
2.56 |
2.52 |
2.58 |
|
51200 |
2.24 |
2.22 |
2.27 |
2.2 |
2.22 |
2.25 |
2.24 |
2.23 |
|
2.23 |
2.20 |
2.27 |
|
64507 |
1.92 |
1.9 |
1.89 |
1.92 |
1.92 |
1.92 |
1.91 |
1.89 |
|
1.91 |
1.89 |
1.92 |
|
81274 |
1.57 |
1.58 |
1.56 |
1.54 |
1.53 |
1.58 |
1.57 |
1.57 |
|
1.56 |
1.53 |
1.58 |
|
102400 |
1.25 |
1.23 |
1.26 |
1.22 |
1.23 |
1.25 |
1.23 |
1.21 |
|
1.24 |
1.21 |
1.26 |
|
204800 |
0.45 |
0.45 |
0.45 |
0.46 |
0.45 |
0.45 |
0.46 |
0.44 |
|
0.45 |
0.44 |
0.46 |
|
409600 |
0.27 |
0.26 |
0.26 |
0.27 |
0.26 |
0.25 |
0.28 |
0.27 |
|
0.27 |
0.25 |
0.28 |
|
819200 |
0.18 |
0.17 |
0.16 |
0.19 |
0.19 |
0.18 |
0.19 |
0.19 |
|
0.18 |
0.16 |
0.19 |
|
1638400 |
0.13 |
0.13 |
0.13 |
0.13 |
0.13 |
0.13 |
0.13 |
0.15 |
|
0.13 |
0.13 |
0.15 |
And here is the detailed data for PDR sample variation:
|
ISO |
PDR (stops) |
|
PDR (stops) |
|||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
50 |
10.69 |
10.68 |
10.65 |
10.7 |
10.62 |
10.64 |
10.71 |
10.68 |
|
10.67 |
10.62 |
10.71 |
|
62 |
10.64 |
10.7 |
10.68 |
10.64 |
10.63 |
10.66 |
10.68 |
10.7 |
|
10.67 |
10.63 |
10.70 |
|
79 |
10.7 |
10.69 |
10.7 |
10.7 |
10.65 |
10.67 |
10.71 |
10.7 |
|
10.69 |
10.65 |
10.71 |
|
100 |
10.67 |
10.71 |
10.67 |
10.68 |
10.65 |
10.66 |
10.69 |
10.7 |
|
10.68 |
10.65 |
10.71 |
|
125 |
10.38 |
10.36 |
10.37 |
10.38 |
10.35 |
10.37 |
10.37 |
10.37 |
|
10.37 |
10.35 |
10.38 |
|
158 |
10.09 |
10.06 |
10.06 |
10.02 |
10.05 |
10.08 |
10.08 |
10.05 |
|
10.06 |
10.02 |
10.09 |
|
200 |
9.79 |
9.78 |
9.77 |
9.8 |
9.77 |
9.81 |
9.8 |
9.82 |
|
9.79 |
9.77 |
9.82 |
|
251 |
9.47 |
9.49 |
9.48 |
9.48 |
9.47 |
9.5 |
9.49 |
9.48 |
|
9.48 |
9.47 |
9.50 |
|
317 |
9.17 |
9.16 |
9.15 |
9.16 |
9.15 |
9.19 |
9.14 |
9.14 |
|
9.16 |
9.14 |
9.19 |
|
400 |
9.17 |
9.14 |
9.19 |
9.16 |
9.13 |
9.16 |
9.18 |
9.19 |
|
9.17 |
9.13 |
9.19 |
|
503 |
8.87 |
8.84 |
8.83 |
8.85 |
8.85 |
8.83 |
8.85 |
8.87 |
|
8.85 |
8.83 |
8.87 |
|
634 |
8.54 |
8.5 |
8.54 |
8.5 |
8.5 |
8.54 |
8.49 |
8.53 |
|
8.52 |
8.49 |
8.54 |
|
800 |
8.18 |
8.16 |
8.17 |
8.18 |
8.18 |
8.17 |
8.17 |
8.17 |
|
8.17 |
8.16 |
8.18 |
|
1007 |
7.85 |
7.83 |
7.85 |
7.85 |
7.82 |
7.82 |
7.84 |
7.87 |
|
7.84 |
7.82 |
7.87 |
|
1269 |
7.53 |
7.52 |
7.49 |
7.51 |
7.51 |
7.51 |
7.49 |
7.49 |
|
7.51 |
7.49 |
7.53 |
|
1600 |
7.2 |
7.2 |
7.19 |
7.18 |
7.19 |
7.17 |
7.18 |
7.16 |
|
7.18 |
7.16 |
7.20 |
|
2015 |
6.9 |
6.84 |
6.85 |
6.86 |
6.87 |
6.87 |
6.85 |
6.84 |
|
6.86 |
6.84 |
6.90 |
|
2539 |
6.55 |
6.52 |
6.56 |
6.55 |
6.55 |
6.53 |
6.57 |
6.53 |
|
6.55 |
6.52 |
6.57 |
|
3200 |
6.22 |
6.19 |
6.2 |
6.22 |
6.22 |
6.22 |
6.19 |
6.22 |
|
6.21 |
6.19 |
6.22 |
|
4031 |
5.91 |
5.87 |
5.86 |
5.86 |
5.87 |
5.89 |
5.85 |
5.88 |
|
5.87 |
5.85 |
5.91 |
|
5079 |
5.59 |
5.5 |
5.53 |
5.57 |
5.53 |
5.53 |
5.52 |
5.58 |
|
5.54 |
5.50 |
5.59 |
|
6400 |
5.23 |
5.17 |
5.2 |
5.24 |
5.21 |
5.21 |
5.18 |
5.21 |
|
5.21 |
5.17 |
5.24 |
|
8063 |
4.89 |
4.88 |
4.86 |
4.84 |
4.88 |
4.87 |
4.9 |
4.92 |
|
4.88 |
4.84 |
4.92 |
|
10159 |
4.56 |
4.54 |
4.55 |
4.55 |
4.54 |
4.56 |
4.58 |
4.53 |
|
4.55 |
4.53 |
4.58 |
|
12800 |
4.24 |
4.2 |
4.2 |
4.22 |
4.19 |
4.21 |
4.2 |
4.23 |
|
4.21 |
4.19 |
4.24 |
|
16126 |
3.91 |
3.87 |
3.87 |
3.83 |
3.91 |
3.86 |
3.86 |
3.89 |
|
3.88 |
3.83 |
3.91 |
|
20318 |
3.57 |
3.54 |
3.57 |
3.56 |
3.55 |
3.54 |
3.58 |
3.59 |
|
3.56 |
3.54 |
3.59 |
|
25600 |
3.24 |
3.2 |
3.22 |
3.22 |
3.22 |
3.25 |
3.23 |
3.27 |
|
3.23 |
3.20 |
3.27 |
|
32253 |
2.88 |
2.87 |
2.91 |
2.89 |
2.88 |
2.89 |
2.89 |
2.92 |
|
2.89 |
2.87 |
2.92 |
|
40637 |
2.58 |
2.53 |
2.57 |
2.54 |
2.56 |
2.56 |
2.57 |
2.58 |
|
2.56 |
2.53 |
2.58 |
|
51200 |
2.24 |
2.21 |
2.22 |
2.21 |
2.22 |
2.21 |
2.21 |
2.23 |
|
2.22 |
2.21 |
2.24 |
|
64507 |
1.93 |
1.89 |
1.86 |
1.91 |
1.87 |
1.88 |
1.89 |
1.93 |
|
1.90 |
1.86 |
1.93 |
|
81274 |
1.55 |
1.53 |
1.55 |
1.54 |
1.55 |
1.57 |
1.54 |
1.54 |
|
1.55 |
1.53 |
1.57 |
|
102400 |
1.24 |
1.2 |
1.24 |
1.25 |
1.23 |
1.25 |
1.24 |
1.26 |
|
1.24 |
1.20 |
1.26 |
|
204800 |
0.46 |
0.43 |
0.45 |
0.45 |
0.45 |
0.45 |
0.45 |
0.46 |
|
0.45 |
0.43 |
0.46 |
|
409600 |
0.27 |
0.27 |
0.26 |
0.27 |
0.26 |
0.27 |
0.27 |
0.26 |
|
0.27 |
0.26 |
0.27 |
|
819200 |
0.18 |
0.18 |
0.18 |
0.19 |
0.18 |
0.19 |
0.18 |
0.18 |
|
0.18 |
0.18 |
0.19 |
|
1638400 |
0.14 |
0.15 |
0.13 |
0.13 |
0.14 |
0.12 |
0.14 |
0.13 |
|
0.14 |
0.12 |
0.15 |
You're welcome to copy and paste data out of these tables for
your own analysis.
Here's what is looks like in chart form:
You can play with the interactive chart for yourself here.
As with the read noise it's hard to see the error bars looking
at this full view. Let's zoom in for a closer look:
Now we can see the error bars showing the measurement and
sample variation.
They are quite small, typically no more than 1/20th of a stop in height.
So PDR measurements are quite accurate and not particularly
subject to sample variation.
Another measure, Photographic Dynamic Range Shadow Improvement is directly
related to PDR and has identical variation.
Fixed Pattern Noise Results
The third and final standard test is for Fixed Pattern Noise
(FPN).
When the FPN test is performed several other related measures are also
gathered.
The FPN results are visualized using heatmaps (this is the
name for this type of display and has nothing to do with measuring heat).
The primary heatmaps for the Nikon D500 look like this:
The false colors have no particular meaning and are present to
help visualize any patterns that might be present.
The stacked images represent FPN whereas the single images could show temporal
pattern noise that is not fixed.
Although the illuminated FPN looks a little "blotchy" these are quite
clean.
Here's an example of a camera that clearly has temporal (but
not fixed) pattern noise in the shadows:
You can see the Sensor Heatmaps for other cameras and compare pairs of cameras here.
In addition to the visualizations then FPN analysis determines
several numeric values presented in a sort-able table that looks like this:
The row and column metrics are measures of pattern noise where
zero would be perfect.
The relatively high Black Row Metric values for the Nikon D5200 and Nikon D7100
quantify the temporal pattern noise that is evident in the visualizations.
DSNU is Dark Signal Non-Uniformity and PRNU is Photo-Response
Non-Uniformity; these are classic components of Fixed Pattern Noise (FPN).
A detailed explanation of all of the columns in the table is beyond the scope
of this article; but we can look at the measurement and sample variation.
Here's the detailed data for measurement variation:
|
|
Eight Datasets Using One Nikon
D500 Body at ISO 100 |
|||||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
Black
Level Range |
0.138 |
0.127 |
0.149 |
0.128 |
0.139 |
0.136 |
0.141 |
0.137 |
|
0.137 |
0.127 |
0.149 |
|
Black
Column Metric |
0.013 |
0.013 |
0.012 |
0.012 |
0.011 |
0.011 |
0.011 |
0.012 |
|
0.012 |
0.011 |
0.013 |
|
Black Row
Metric |
0.021 |
0.023 |
0.022 |
0.021 |
0.023 |
0.022 |
0.023 |
0.023 |
|
0.022 |
0.021 |
0.023 |
|
Illuminated
Column Metric |
0.013 |
0.013 |
0.012 |
0.012 |
0.012 |
0.011 |
0.011 |
0.012 |
|
0.012 |
0.011 |
0.013 |
|
Illuminated
Row Metric |
0.008 |
0.009 |
0.008 |
0.008 |
0.008 |
0.008 |
0.008 |
0.008 |
|
0.008 |
0.008 |
0.009 |
|
DSNU |
8.7% |
4.5% |
7.2% |
9.5% |
5.6% |
5.6% |
9.4% |
6.1% |
|
7.1% |
4.5% |
9.5% |
|
PRNU |
0.31% |
0.30% |
0.31% |
0.29% |
0.31% |
0.30% |
0.32% |
0.30% |
|
0.31% |
0.29% |
0.32% |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Eight Datasets Using One Nikon
D500 Body at ISO 400 |
|||||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
Black
Level Range |
0.112 |
0.126 |
0.111 |
0.103 |
0.113 |
0.104 |
0.108 |
0.105 |
|
0.110 |
0.103 |
0.126 |
|
Black
Column Metric |
0.010 |
0.011 |
0.010 |
0.010 |
0.010 |
0.011 |
0.011 |
0.010 |
|
0.010 |
0.010 |
0.011 |
|
Black Row
Metric |
0.005 |
0.007 |
0.006 |
0.007 |
0.007 |
0.007 |
0.006 |
0.008 |
|
0.007 |
0.005 |
0.008 |
|
Illuminated
Column Metric |
0.008 |
0.008 |
0.009 |
0.009 |
0.009 |
0.008 |
0.007 |
0.009 |
|
0.008 |
0.007 |
0.009 |
|
Illuminated
Row Metric |
0.006 |
0.005 |
0.005 |
0.005 |
0.005 |
0.005 |
0.004 |
0.005 |
|
0.005 |
0.004 |
0.006 |
|
DSNU |
6.0% |
5.8% |
7.7% |
9.3% |
7.8% |
6.4% |
8.1% |
9.0% |
|
7.5% |
5.8% |
9.3% |
|
PRNU |
0.31% |
0.30% |
0.30% |
0.29% |
0.33% |
0.32% |
0.28% |
0.31% |
|
0.30% |
0.28% |
0.33% |
And the detailed data for sample variation:
|
|
One Dataset for Each of Eight
Nikon D500 Bodies at ISO 100 |
|||||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
Black
Level Range |
0.177 |
0.124 |
0.208 |
0.125 |
0.161 |
0.138 |
0.147 |
0.158 |
|
0.155 |
0.124 |
0.208 |
|
Black
Column Metric |
0.011 |
0.012 |
0.012 |
0.014 |
0.012 |
0.010 |
0.011 |
0.011 |
|
0.012 |
0.010 |
0.014 |
|
Black Row
Metric |
0.025 |
0.014 |
0.047 |
0.016 |
0.026 |
0.019 |
0.018 |
0.026 |
|
0.024 |
0.014 |
0.047 |
|
Illuminated
Column Metric |
0.013 |
0.014 |
0.014 |
0.012 |
0.015 |
0.012 |
0.012 |
0.012 |
|
0.013 |
0.012 |
0.015 |
|
Illuminated
Row Metric |
0.008 |
0.011 |
0.010 |
0.009 |
0.011 |
0.015 |
0.006 |
0.011 |
|
0.010 |
0.006 |
0.015 |
|
DSNU |
1.8% |
3.9% |
5.4% |
3.9% |
6.5% |
2.8% |
7.0% |
2.6% |
|
4.2% |
1.8% |
7.0% |
|
PRNU |
0.27% |
0.27% |
0.26% |
0.27% |
0.27% |
0.27% |
0.26% |
0.27% |
|
0.27% |
0.26% |
0.27% |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
One Dataset for Each of Eight
Nikon D500 Bodies at ISO 400 |
|||||||||||
|
|
#1 |
#2 |
#3 |
#4 |
#5 |
#6 |
#7 |
#8 |
|
Average |
Minimum |
Maximum |
|
Black
Level Range |
0.131 |
0.093 |
0.165 |
0.094 |
0.107 |
0.108 |
0.118 |
0.059 |
|
0.109 |
0.059 |
0.165 |
|
Black
Column Metric |
0.008 |
0.008 |
0.010 |
0.011 |
0.010 |
0.009 |
0.010 |
0.009 |
|
0.009 |
0.008 |
0.011 |
|
Black Row
Metric |
0.006 |
0.005 |
0.011 |
0.005 |
0.005 |
0.005 |
0.006 |
0.005 |
|
0.006 |
0.005 |
0.011 |
|
Illuminated
Column Metric |
0.009 |
0.010 |
0.008 |
0.007 |
0.010 |
0.009 |
0.008 |
0.009 |
|
0.009 |
0.007 |
0.010 |
|
Illuminated
Row Metric |
0.005 |
0.006 |
0.006 |
0.006 |
0.006 |
0.007 |
0.004 |
0.005 |
|
0.006 |
0.004 |
0.007 |
|
DSNU |
5.3% |
6.7% |
5.0% |
3.6% |
1.9% |
2.8% |
4.2% |
2.3% |
|
4.0% |
1.9% |
6.7% |
|
PRNU |
0.26% |
0.26% |
0.27% |
0.27% |
0.27% |
0.27% |
0.26% |
0.27% |
|
0.27% |
0.26% |
0.27% |
Note that the Black Row Metric for body#3 appears to be an outlier; perhaps this body is slightly out of spec.
Here are the results summarized in a column chart with error
bars:
You can play with the interactive chart for yourself here.
Looking at the measurement (dark weight) error bars there
appears to be fairly high measurement accuracy with the exception of DSNU.
(I'll have to investigate if I can measure this more accurately.)
Looking at the sample (light weight) error bars it's not
surprising to see sample variation is larger than measurement variation.
The outlier for Black Row Metric makes this error bar really stand out.
From measurement variation we expect DSNU to have a large range; and it does.
Of the other results it appears the Black Level Delta might be more subject to
sample variation than the other measures (excluding DSNU).
This may make sense as Black Level Delta depends on the stability of a voltage
bias in the sensor circuitry.
Overall, these are pretty clean results.
Conclusions
It was a worthwhile exercise to perform 16 sets of tests to determine measurement and sample variation.
The results are not particularly surprising to me.
Although most of the circuitry in a digital camera is analog; these integrated
circuits are solid state electronics that are manufactured in a highly
repeatable way.
Power sources and the adjustment of voltage levels are the factors that are
most susceptible to variation and this variation shows primarily in the very
darkest signals.
Naturally there is always potential for sample variation and in this testing we saw one camera that had one reading that was consistently an outlier.
In my experience at PhotonsToPhotos measurement and sample
variation is seldom an issue.
Honestly, the most serious issue I have is that sometimes raw files collected
on my behalf aren't properly taken to my protocol.