Measurement and Sample Variation

Prepared 2018-10-26 by Bill Claff
Last revised 2018-11-01

Introduction

I have been publishing a variety of sensor measurements since 2007 and have now accumulated information on about 250 cameras.

From time to time people inquire about the accuracy of the measurements and about sample variation.

With the help of LensProToGo (a sister company to LensRentals) I was able to gather data to get some clarity on this issue.

Nine Nikon D500 bodies were made available.
I collected eight sets of raw files from one body and one set of raw data from each of the other eight bodies.

From the first set I hoped to quantify measurement variation; how repeatable my testing is.
And from the second set to quantify sample variation; how much variation there is between cameras.

Read Noise

The first test is for Read Noise which is measured from raw files and reported in Digital Numbers (DN).
The published Read Noise in DNs for the Nikon D500 at PhotonsToPhotos looks like this:

You can see other cameras and do comparisons at by visiting the interactive chart.

Read noise is not directly relevant to photographers but is a strong influence on a number of factors that are of interest.
We can get some useful information directly from the chart.
We see confirmation that ISO 100 is the base ISO setting; the highest ISO setting with the lowest read noise.
We also see a drop in read noise at ISO 400 due to the fact that this is a dual conversion gain sensor.
For more technical background on dual conversion gain see this Aptina DR-Pix Technology White Paper.
The straightness of the lines, particularly at the intermediate ISO settings, is also meaningful to someone experienced at reading such charts.

So, what did testing with multiple bodies reveal? Here is the detailed data for read noise measurement variation:

Eight Datasets Using One Nikon D500 Body

ISO
Setting

Read Noise (DN)

 

Read Noise (DN)

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

50

1.297

1.322

1.324

1.337

1.345

1.337

1.335

1.346

 

1.328

1.297

1.345

62

1.316

1.312

1.333

1.340

1.334

1.347

1.335

1.341

 

1.331

1.312

1.347

79

1.321

1.310

1.317

1.340

1.327

1.354

1.334

1.332

 

1.329

1.310

1.354

100

1.306

1.309

1.317

1.340

1.322

1.340

1.331

1.348

 

1.324

1.306

1.340

125

1.556

1.551

1.556

1.583

1.591

1.583

1.560

1.575

 

1.568

1.551

1.591

158

1.870

1.892

1.914

1.911

1.911

1.903

1.904

1.927

 

1.901

1.870

1.914

200

2.230

2.244

2.250

2.272

2.276

2.243

2.272

2.283

 

2.255

2.230

2.276

251

2.735

2.797

2.815

2.722

2.742

2.718

2.724

2.789

 

2.751

2.718

2.815

317

3.349

3.358

3.414

3.429

3.373

3.417

3.397

3.435

 

3.391

3.349

3.429

400

1.977

1.996

1.946

1.977

2.028

1.958

2.009

1.992

 

1.984

1.946

2.028

503

2.418

2.465

2.448

2.495

2.405

2.434

2.459

2.452

 

2.446

2.405

2.495

634

2.957

3.059

2.989

3.009

3.078

3.055

2.967

3.018

 

3.016

2.957

3.078

800

3.671

3.675

3.694

3.692

3.668

3.681

3.717

3.743

 

3.685

3.668

3.717

1007

4.427

4.540

4.502

4.499

4.508

4.407

4.543

4.583

 

4.490

4.407

4.543

1269

5.537

5.552

5.480

5.507

5.610

5.460

5.429

5.536

 

5.511

5.429

5.610

1600

6.716

6.687

6.986

6.717

6.870

6.788

6.990

7.042

 

6.822

6.687

6.990

2015

8.121

8.154

8.700

8.322

8.385

8.204

8.390

8.224

 

8.325

8.121

8.700

2539

10.415

10.058

10.343

10.277

10.442

10.440

10.174

10.072

 

10.307

10.058

10.442

3200

12.452

12.248

12.258

12.233

12.571

12.625

12.434

12.428

 

12.403

12.233

12.625

4031

15.385

15.867

15.333

15.747

15.509

15.623

15.577

15.875

 

15.577

15.333

15.867

5079

19.356

19.823

19.189

18.867

19.210

19.406

19.400

19.156

 

19.322

18.867

19.823

6400

23.878

24.094

23.383

23.570

23.832

24.358

24.284

23.168

 

23.914

23.383

24.358

8063

29.671

28.847

28.873

29.913

28.883

28.776

29.778

29.743

 

29.249

28.776

29.913

10159

35.698

35.302

36.209

36.404

36.992

37.055

36.666

36.104

 

36.332

35.302

37.055

12800

44.060

46.586

46.167

46.343

45.201

45.814

46.543

45.882

 

45.816

44.060

46.586

16126

56.872

56.916

57.938

56.123

56.848

57.412

57.822

56.387

 

57.133

56.123

57.938

20318

71.240

71.046

70.325

70.466

71.345

70.546

72.709

72.918

 

71.097

70.325

72.709

25600

83.562

84.150

82.828

83.847

83.853

84.648

84.845

83.693

 

83.962

82.828

84.845

32253

104.931

105.759

104.339

104.974

106.706

106.707

105.314

105.722

 

105.533

104.339

106.707

40637

131.626

131.142

131.902

132.454

132.036

131.743

131.056

132.382

 

131.709

131.056

132.454

51200

159.441

162.557

164.273

164.124

164.866

164.951

164.768

164.751

 

163.568

159.441

164.951

64507

197.914

203.097

203.761

202.546

204.949

200.274

204.216

203.646

 

202.394

197.914

204.949

81274

245.719

247.435

242.537

252.568

247.554

248.263

246.259

248.837

 

247.191

242.537

252.568

102400

299.422

300.345

298.828

296.090

303.553

296.891

296.938

299.303

 

298.867

296.090

303.553

204800

497.311

512.157

498.913

511.336

503.063

512.372

513.006

510.106

 

506.880

497.311

513.006

409600

902.947

880.923

877.589

862.757

901.813

891.230

885.480

894.146

 

886.105

862.757

902.947

819200

1626.139

1568.881

1582.377

1616.346

1587.605

1609.370

1647.454

1645.210

 

1605.453

1568.881

1647.454

1638400

3013.404

2980.689

3067.988

3016.601

3065.618

3105.912

3116.458

3099.209

 

3052.381

2980.689

3116.458

And here is the detailed data for read noise sample variation:

One Dataset for Each of Eight Nikon D500 Bodies

ISO
Setting

Read Noise (DN)

 

Read Noise (DN)

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

50

1.327

1.333

1.347

1.321

1.341

1.324

1.286

1.330

 

1.326

1.286

1.347

62

1.316

1.318

1.345

1.331

1.344

1.326

1.293

1.341

 

1.325

1.293

1.345

79

1.307

1.306

1.328

1.358

1.333

1.327

1.314

1.339

 

1.325

1.306

1.358

100

1.315

1.311

1.336

1.331

1.340

1.319

1.295

1.320

 

1.321

1.295

1.340

125

1.608

1.546

1.597

1.577

1.588

1.577

1.554

1.604

 

1.578

1.546

1.608

158

1.935

1.911

1.940

1.916

1.957

1.935

1.904

1.956

 

1.928

1.904

1.957

200

2.363

2.312

2.340

2.375

2.367

2.345

2.242

2.300

 

2.335

2.242

2.375

251

2.890

2.778

2.881

2.940

2.874

2.880

2.788

2.794

 

2.862

2.778

2.940

317

3.509

3.479

3.520

3.528

3.477

3.433

3.426

3.457

 

3.482

3.426

3.528

400

2.000

2.003

2.081

2.087

2.015

1.947

1.944

1.990

 

2.011

1.944

2.087

503

2.429

2.447

2.579

2.504

2.519

2.446

2.391

2.440

 

2.474

2.391

2.579

634

2.994

2.978

3.170

3.081

3.037

2.996

2.896

2.993

 

3.022

2.896

3.170

800

3.693

3.675

3.790

3.881

3.820

3.686

3.646

3.655

 

3.741

3.646

3.881

1007

4.503

4.442

4.683

4.708

4.719

4.570

4.489

4.473

 

4.588

4.442

4.719

1269

5.635

5.547

5.719

5.786

5.738

5.612

5.531

5.428

 

5.652

5.531

5.786

1600

6.711

7.044

7.108

7.132

7.042

6.800

6.790

6.919

 

6.947

6.711

7.132

2015

8.485

8.371

8.828

8.675

8.606

8.545

8.256

8.097

 

8.538

8.256

8.828

2539

10.157

10.309

10.984

10.841

10.357

10.593

10.459

10.184

 

10.529

10.157

10.984

3200

12.612

12.372

12.981

13.231

13.065

12.749

12.857

12.370

 

12.838

12.372

13.231

4031

15.660

15.555

16.171

16.309

16.007

15.954

15.384

15.326

 

15.863

15.384

16.309

5079

19.000

19.245

20.333

20.481

19.708

19.065

18.824

19.503

 

19.522

18.824

20.481

6400

23.684

23.755

25.377

24.565

24.017

24.346

23.901

23.004

 

24.235

23.684

25.377

8063

29.634

28.609

30.814

30.116

30.157

29.605

29.479

28.493

 

29.774

28.609

30.814

10159

36.616

37.028

38.476

38.625

37.543

37.077

36.062

35.876

 

37.347

36.062

38.625

12800

45.355

46.131

48.254

48.288

45.646

45.902

46.622

44.902

 

46.600

45.355

48.288

16126

56.018

56.512

59.941

59.584

58.219

58.497

57.224

56.919

 

57.999

56.018

59.941

20318

70.868

70.941

74.446

74.785

71.449

71.701

70.729

69.882

 

72.131

70.729

74.785

25600

84.054

83.852

89.036

87.274

87.234

83.329

83.897

83.636

 

85.525

83.329

89.036

32253

105.872

104.711

111.835

108.732

108.680

106.477

104.732

104.258

 

107.291

104.711

111.835

40637

131.789

130.104

138.397

135.913

136.766

132.679

131.505

130.408

 

133.879

130.104

138.397

51200

164.410

164.558

175.309

171.017

167.966

160.951

165.752

162.511

 

167.137

160.951

175.309

64507

203.878

203.282

213.465

209.666

208.799

203.155

200.608

198.725

 

206.122

200.608

213.465

81274

250.041

246.392

260.326

256.010

256.257

247.714

247.367

245.227

 

252.015

246.392

260.326

102400

298.603

296.809

315.078

308.213

302.694

302.211

294.201

293.915

 

302.544

294.201

315.078

204800

500.129

514.544

532.070

527.505

515.184

504.355

511.762

497.097

 

515.079

500.129

532.070

409600

910.687

889.512

934.463

901.945

907.114

882.334

880.932

883.772

 

900.998

880.932

934.463

819200

1589.257

1599.712

1700.273

1637.190

1609.225

1624.230

1625.268

1618.001

 

1626.451

1589.257

1700.273

1638400

3028.489

2987.952

3248.938

3215.677

3070.537

2999.510

3075.645

3044.515

 

3089.535

2987.952

3248.938

You're welcome to copy and paste data out of these tables for your own analysis.
Here's what is looks like in chart form:

You can play with the interactive chart for yourself here.

It's hard to see, but error bars have been added to show the measurement and sample variation ranges.
We can zoom in on the chart by drawing a rectangle with the mouse:

Now the error bars showing the measurement and sample variation can be seen.
Note that the horizontal gridlines are 1/3 stop apart and that generally the error bars are well under 1/6 stop in height.

So it appears that read noise measurements are quite accurate and not particularly subject to sample variation.
As is often the case there are some exceptions that are beyond the scope of this article.
Another measure, Input-referred Read Noise is directly related to Read Noise in DNs and has identical variation.

Photographic Dynamic Range (PDR)

The second test is for Photographic Dynamic Range (PDR).
This is a practical measure which incorporates all noise sources and targets a Signal to Noise Ratio (SNR) that is visually acceptable under standard viewing conditions.
PDR is the most popular measure published at PhotonsToPhotos.

The published PDR for the Nikon D500 looks like this:

You can see other cameras and do comparisons at by visiting the interactive chart.

For this particular sensor design we expect a pretty straight line.
Note the slight boost in PDR at ISO 400 where the High Conversion Gain (HCG) of this dual conversion gain sensor kicks in.

Here is the detailed data for PDR measurement variation:

Eight Datasets Using One Nikon D500 Body

ISO
Setting

PDR (stops)

 

PDR (stops)

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

50

10.67

10.67

10.65

10.7

10.66

10.68

10.7

10.68

 

10.67

10.65

10.70

62

10.69

10.68

10.69

10.7

10.65

10.69

10.66

10.65

 

10.67

10.65

10.69

79

10.65

10.6

10.66

10.7

10.65

10.68

10.68

10.67

 

10.66

10.60

10.68

100

10.68

10.68

10.69

10.7

10.65

10.67

10.67

10.66

 

10.67

10.65

10.69

125

10.4

10.37

10.37

10.4

10.35

10.37

10.36

10.37

 

10.37

10.35

10.40

158

10.05

10.05

10.07

10.1

10.06

10.07

10.07

10.08

 

10.07

10.05

10.08

200

9.8

9.78

9.79

9.8

9.77

9.78

9.79

9.79

 

9.79

9.77

9.80

251

9.49

9.5

9.5

9.5

9.51

9.47

9.49

9.5

 

9.50

9.47

9.51

317

9.19

9.14

9.19

9.17

9.16

9.18

9.17

9.16

 

9.17

9.14

9.19

400

9.19

9.17

9.2

9.2

9.17

9.17

9.18

9.17

 

9.18

9.17

9.20

503

8.88

8.86

8.87

8.87

8.86

8.87

8.88

8.89

 

8.87

8.86

8.89

634

8.54

8.53

8.5

8.52

8.53

8.54

8.5

8.51

 

8.52

8.50

8.54

800

8.21

8.2

8.17

8.15

8.16

8.19

8.2

8.18

 

8.18

8.15

8.21

1007

7.83

7.81

7.85

7.83

7.83

7.87

7.84

7.85

 

7.84

7.81

7.87

1269

7.52

7.52

7.56

7.5

7.46

7.5

7.53

7.5

 

7.51

7.46

7.56

1600

7.21

7.18

7.21

7.18

7.22

7.22

7.21

7.21

 

7.21

7.18

7.22

2015

6.88

6.85

6.89

6.87

6.86

6.89

6.91

6.9

 

6.88

6.85

6.91

2539

6.57

6.55

6.57

6.53

6.53

6.57

6.54

6.58

 

6.56

6.53

6.58

3200

6.24

6.2

6.23

6.19

6.24

6.23

6.23

6.24

 

6.23

6.19

6.24

4031

5.92

5.87

5.89

5.87

5.87

5.9

5.89

5.91

 

5.89

5.87

5.92

5079

5.57

5.51

5.59

5.55

5.52

5.54

5.53

5.57

 

5.55

5.51

5.59

6400

5.2

5.21

5.24

5.23

5.22

5.24

5.25

5.24

 

5.23

5.20

5.25

8063

4.9

4.87

4.91

4.89

4.86

4.9

4.86

4.88

 

4.88

4.86

4.91

10159

4.55

4.53

4.58

4.57

4.57

4.58

4.55

4.56

 

4.56

4.53

4.58

12800

4.22

4.19

4.22

4.25

4.21

4.25

4.23

4.22

 

4.22

4.19

4.25

16126

3.9

3.87

3.92

3.91

3.87

3.91

3.9

3.88

 

3.90

3.87

3.92

20318

3.56

3.54

3.58

3.52

3.55

3.58

3.58

3.54

 

3.56

3.52

3.58

25600

3.26

3.21

3.24

3.25

3.22

3.26

3.25

3.25

 

3.24

3.21

3.26

32253

2.89

2.88

2.92

2.89

2.86

2.92

2.91

2.9

 

2.90

2.86

2.92

40637

2.58

2.52

2.58

2.56

2.57

2.55

2.58

2.57

 

2.56

2.52

2.58

51200

2.24

2.22

2.27

2.2

2.22

2.25

2.24

2.23

 

2.23

2.20

2.27

64507

1.92

1.9

1.89

1.92

1.92

1.92

1.91

1.89

 

1.91

1.89

1.92

81274

1.57

1.58

1.56

1.54

1.53

1.58

1.57

1.57

 

1.56

1.53

1.58

102400

1.25

1.23

1.26

1.22

1.23

1.25

1.23

1.21

 

1.24

1.21

1.26

204800

0.45

0.45

0.45

0.46

0.45

0.45

0.46

0.44

 

0.45

0.44

0.46

409600

0.27

0.26

0.26

0.27

0.26

0.25

0.28

0.27

 

0.27

0.25

0.28

819200

0.18

0.17

0.16

0.19

0.19

0.18

0.19

0.19

 

0.18

0.16

0.19

1638400

0.13

0.13

0.13

0.13

0.13

0.13

0.13

0.15

 

0.13

0.13

0.15

And here is the detailed data for PDR sample variation:

One Dataset for Each of Eight Nikon D500 Bodies

ISO
Setting

PDR (stops)

 

PDR (stops)

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

50

10.69

10.68

10.65

10.7

10.62

10.64

10.71

10.68

 

10.67

10.62

10.71

62

10.64

10.7

10.68

10.64

10.63

10.66

10.68

10.7

 

10.67

10.63

10.70

79

10.7

10.69

10.7

10.7

10.65

10.67

10.71

10.7

 

10.69

10.65

10.71

100

10.67

10.71

10.67

10.68

10.65

10.66

10.69

10.7

 

10.68

10.65

10.71

125

10.38

10.36

10.37

10.38

10.35

10.37

10.37

10.37

 

10.37

10.35

10.38

158

10.09

10.06

10.06

10.02

10.05

10.08

10.08

10.05

 

10.06

10.02

10.09

200

9.79

9.78

9.77

9.8

9.77

9.81

9.8

9.82

 

9.79

9.77

9.82

251

9.47

9.49

9.48

9.48

9.47

9.5

9.49

9.48

 

9.48

9.47

9.50

317

9.17

9.16

9.15

9.16

9.15

9.19

9.14

9.14

 

9.16

9.14

9.19

400

9.17

9.14

9.19

9.16

9.13

9.16

9.18

9.19

 

9.17

9.13

9.19

503

8.87

8.84

8.83

8.85

8.85

8.83

8.85

8.87

 

8.85

8.83

8.87

634

8.54

8.5

8.54

8.5

8.5

8.54

8.49

8.53

 

8.52

8.49

8.54

800

8.18

8.16

8.17

8.18

8.18

8.17

8.17

8.17

 

8.17

8.16

8.18

1007

7.85

7.83

7.85

7.85

7.82

7.82

7.84

7.87

 

7.84

7.82

7.87

1269

7.53

7.52

7.49

7.51

7.51

7.51

7.49

7.49

 

7.51

7.49

7.53

1600

7.2

7.2

7.19

7.18

7.19

7.17

7.18

7.16

 

7.18

7.16

7.20

2015

6.9

6.84

6.85

6.86

6.87

6.87

6.85

6.84

 

6.86

6.84

6.90

2539

6.55

6.52

6.56

6.55

6.55

6.53

6.57

6.53

 

6.55

6.52

6.57

3200

6.22

6.19

6.2

6.22

6.22

6.22

6.19

6.22

 

6.21

6.19

6.22

4031

5.91

5.87

5.86

5.86

5.87

5.89

5.85

5.88

 

5.87

5.85

5.91

5079

5.59

5.5

5.53

5.57

5.53

5.53

5.52

5.58

 

5.54

5.50

5.59

6400

5.23

5.17

5.2

5.24

5.21

5.21

5.18

5.21

 

5.21

5.17

5.24

8063

4.89

4.88

4.86

4.84

4.88

4.87

4.9

4.92

 

4.88

4.84

4.92

10159

4.56

4.54

4.55

4.55

4.54

4.56

4.58

4.53

 

4.55

4.53

4.58

12800

4.24

4.2

4.2

4.22

4.19

4.21

4.2

4.23

 

4.21

4.19

4.24

16126

3.91

3.87

3.87

3.83

3.91

3.86

3.86

3.89

 

3.88

3.83

3.91

20318

3.57

3.54

3.57

3.56

3.55

3.54

3.58

3.59

 

3.56

3.54

3.59

25600

3.24

3.2

3.22

3.22

3.22

3.25

3.23

3.27

 

3.23

3.20

3.27

32253

2.88

2.87

2.91

2.89

2.88

2.89

2.89

2.92

 

2.89

2.87

2.92

40637

2.58

2.53

2.57

2.54

2.56

2.56

2.57

2.58

 

2.56

2.53

2.58

51200

2.24

2.21

2.22

2.21

2.22

2.21

2.21

2.23

 

2.22

2.21

2.24

64507

1.93

1.89

1.86

1.91

1.87

1.88

1.89

1.93

 

1.90

1.86

1.93

81274

1.55

1.53

1.55

1.54

1.55

1.57

1.54

1.54

 

1.55

1.53

1.57

102400

1.24

1.2

1.24

1.25

1.23

1.25

1.24

1.26

 

1.24

1.20

1.26

204800

0.46

0.43

0.45

0.45

0.45

0.45

0.45

0.46

 

0.45

0.43

0.46

409600

0.27

0.27

0.26

0.27

0.26

0.27

0.27

0.26

 

0.27

0.26

0.27

819200

0.18

0.18

0.18

0.19

0.18

0.19

0.18

0.18

 

0.18

0.18

0.19

1638400

0.14

0.15

0.13

0.13

0.14

0.12

0.14

0.13

 

0.14

0.12

0.15

You're welcome to copy and paste data out of these tables for your own analysis.
Here's what is looks like in chart form:

You can play with the interactive chart for yourself here.

As with the read noise it's hard to see the error bars looking at this full view. Let's zoom in for a closer look:

Now we can see the error bars showing the measurement and sample variation.
They are quite small, typically no more than 1/20th of a stop in height.

So PDR measurements are quite accurate and not particularly subject to sample variation.
Another measure, Photographic Dynamic Range Shadow Improvement is directly related to PDR and has identical variation.

Fixed Pattern Noise Results

The third and final standard test is for Fixed Pattern Noise (FPN).
When the FPN test is performed several other related measures are also gathered.

The FPN results are visualized using heatmaps (this is the name for this type of display and has nothing to do with measuring heat).
The primary heatmaps for the Nikon D500 look like this:

The false colors have no particular meaning and are present to help visualize any patterns that might be present.
The stacked images represent FPN whereas the single images could show temporal pattern noise that is not fixed.
Although the illuminated FPN looks a little "blotchy" these are quite clean.

Here's an example of a camera that clearly has temporal (but not fixed) pattern noise in the shadows:

You can see the Sensor Heatmaps for other cameras and compare pairs of cameras here.

In addition to the visualizations then FPN analysis determines several numeric values presented in a sort-able table that looks like this:

The row and column metrics are measures of pattern noise where zero would be perfect.
The relatively high Black Row Metric values for the Nikon D5200 and Nikon D7100 quantify the temporal pattern noise that is evident in the visualizations.

DSNU is Dark Signal Non-Uniformity and PRNU is Photo-Response Non-Uniformity; these are classic components of Fixed Pattern Noise (FPN).
A detailed explanation of all of the columns in the table is beyond the scope of this article; but we can look at the measurement and sample variation.

Here's the detailed data for measurement variation:

 

Eight Datasets Using One Nikon D500 Body at ISO 100

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

Black Level Range

0.138

0.127

0.149

0.128

0.139

0.136

0.141

0.137

 

0.137

0.127

0.149

Black Column Metric

0.013

0.013

0.012

0.012

0.011

0.011

0.011

0.012

 

0.012

0.011

0.013

Black Row Metric

0.021

0.023

0.022

0.021

0.023

0.022

0.023

0.023

 

0.022

0.021

0.023

Illuminated Column Metric

0.013

0.013

0.012

0.012

0.012

0.011

0.011

0.012

 

0.012

0.011

0.013

Illuminated Row Metric

0.008

0.009

0.008

0.008

0.008

0.008

0.008

0.008

 

0.008

0.008

0.009

DSNU

8.7%

4.5%

7.2%

9.5%

5.6%

5.6%

9.4%

6.1%

 

7.1%

4.5%

9.5%

PRNU

0.31%

0.30%

0.31%

0.29%

0.31%

0.30%

0.32%

0.30%

 

0.31%

0.29%

0.32%

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Eight Datasets Using One Nikon D500 Body at ISO 400

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

Black Level Range

0.112

0.126

0.111

0.103

0.113

0.104

0.108

0.105

 

0.110

0.103

0.126

Black Column Metric

0.010

0.011

0.010

0.010

0.010

0.011

0.011

0.010

 

0.010

0.010

0.011

Black Row Metric

0.005

0.007

0.006

0.007

0.007

0.007

0.006

0.008

 

0.007

0.005

0.008

Illuminated Column Metric

0.008

0.008

0.009

0.009

0.009

0.008

0.007

0.009

 

0.008

0.007

0.009

Illuminated Row Metric

0.006

0.005

0.005

0.005

0.005

0.005

0.004

0.005

 

0.005

0.004

0.006

DSNU

6.0%

5.8%

7.7%

9.3%

7.8%

6.4%

8.1%

9.0%

 

7.5%

5.8%

9.3%

PRNU

0.31%

0.30%

0.30%

0.29%

0.33%

0.32%

0.28%

0.31%

 

0.30%

0.28%

0.33%

And the detailed data for sample variation:

 

One Dataset for Each of Eight Nikon D500 Bodies at ISO 100

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

Black Level Range

0.177

0.124

0.208

0.125

0.161

0.138

0.147

0.158

 

0.155

0.124

0.208

Black Column Metric

0.011

0.012

0.012

0.014

0.012

0.010

0.011

0.011

 

0.012

0.010

0.014

Black Row Metric

0.025

0.014

0.047

0.016

0.026

0.019

0.018

0.026

 

0.024

0.014

0.047

Illuminated Column Metric

0.013

0.014

0.014

0.012

0.015

0.012

0.012

0.012

 

0.013

0.012

0.015

Illuminated Row Metric

0.008

0.011

0.010

0.009

0.011

0.015

0.006

0.011

 

0.010

0.006

0.015

DSNU

1.8%

3.9%

5.4%

3.9%

6.5%

2.8%

7.0%

2.6%

 

4.2%

1.8%

7.0%

PRNU

0.27%

0.27%

0.26%

0.27%

0.27%

0.27%

0.26%

0.27%

 

0.27%

0.26%

0.27%

 

 

 

 

 

 

 

 

 

 

 

 

 

 

One Dataset for Each of Eight Nikon D500 Bodies at ISO 400

 

#1

#2

#3

#4

#5

#6

#7

#8

 

Average

Minimum

Maximum

Black Level Range

0.131

0.093

0.165

0.094

0.107

0.108

0.118

0.059

 

0.109

0.059

0.165

Black Column Metric

0.008

0.008

0.010

0.011

0.010

0.009

0.010

0.009

 

0.009

0.008

0.011

Black Row Metric

0.006

0.005

0.011

0.005

0.005

0.005

0.006

0.005

 

0.006

0.005

0.011

Illuminated Column Metric

0.009

0.010

0.008

0.007

0.010

0.009

0.008

0.009

 

0.009

0.007

0.010

Illuminated Row Metric

0.005

0.006

0.006

0.006

0.006

0.007

0.004

0.005

 

0.006

0.004

0.007

DSNU

5.3%

6.7%

5.0%

3.6%

1.9%

2.8%

4.2%

2.3%

 

4.0%

1.9%

6.7%

PRNU

0.26%

0.26%

0.27%

0.27%

0.27%

0.27%

0.26%

0.27%

 

0.27%

0.26%

0.27%

Note that the Black Row Metric for body#3 appears to be an outlier; perhaps this body is slightly out of spec.

Here are the results summarized in a column chart with error bars:

You can play with the interactive chart for yourself here.

Looking at the measurement (dark weight) error bars there appears to be fairly high measurement accuracy with the exception of DSNU.
(I'll have to investigate if I can measure this more accurately.)

Looking at the sample (light weight) error bars it's not surprising to see sample variation is larger than measurement variation.
The outlier for Black Row Metric makes this error bar really stand out.
From measurement variation we expect DSNU to have a large range; and it does.
Of the other results it appears the Black Level Delta might be more subject to sample variation than the other measures (excluding DSNU).
This may make sense as Black Level Delta depends on the stability of a voltage bias in the sensor circuitry.

Overall, these are pretty clean results.

Conclusions

It was a worthwhile exercise to perform 16 sets of tests to determine measurement and sample variation.

The results are not particularly surprising to me.
Although most of the circuitry in a digital camera is analog; these integrated circuits are solid state electronics that are manufactured in a highly repeatable way.
Power sources and the adjustment of voltage levels are the factors that are most susceptible to variation and this variation shows primarily in the very darkest signals.

Naturally there is always potential for sample variation and in this testing we saw one camera that had one reading that was consistently an outlier.

In my experience at PhotonsToPhotos measurement and sample variation is seldom an issue.
Honestly, the most serious issue I have is that sometimes raw files collected on my behalf aren't properly taken to my protocol.