Quantitative Morphology in Kidney Research
February 13-14, 2012 Conference Videos

Digital Pathology: Whole-Slide Imaging - Asking the Critical Questions
Stephen Hewitt, National Cancer Institute

Video Transcript

1
00:00:00,000 --> 00:00:09,100
JEFFREY KOPP: And so, we have one final speaker before we go to the break, which is Dr. Steve Hewitt of NCI, a pathologist who is a

2
00:00:09,100 --> 00:00:17,400
Commander in the Public Health Service and who will be talking to us on digital pathology.

3
00:00:17,400 --> 00:00:22,366
STEVE HEWITT: Thank you, Jeffrey. So, I’m not going to take much of your time. We’re going to change topics real quickly. The reason I’m talking

4
00:00:22,366 --> 00:00:34,099
about this is that I’m providing the imaging database for the NEPTUNE clinical trial and that’s taught us a lot about whole slide imaging, and

5
00:00:34,100 --> 00:00:41,266
obviously the move of using these morphometric analysis to clinical samples is going to probably involve whole slide imaging; it’s a natural

6
00:00:41,266 --> 00:00:49,466
connection between these two technologies. This is my black box warning thing. We had Aliza earlier from the FDA and they talk about what you

7
00:00:49,466 --> 00:00:54,232
need to know before somebody speaks, and what you really need to know here is, number one, I’m a consultant to the FDA on whole slide

8
00:00:54,233 --> 00:01:02,399
imaging, and so I beat this topic up for a living every day, and then the last thing on the list is a special thanks to Catherine Conway, one of my

9
00:01:02,400 --> 00:01:08,366
post-doctoral fellows who’s in the back of the room who generated most of these images. I think most of you are actually familiar with whole

10
00:01:08,366 --> 00:01:13,999
slide imaging and have seen this technology, virtual microscopy. It goes by a number of different names and it’s the capacity to make a

11
00:01:14,000 --> 00:01:22,966
whole slide image of a single microscope slide and you’ll see that there are two commonly used magnifications: our objective microscopy lenses

12
00:01:22,966 --> 00:01:33,166
used in these instances and they generate images basically at these two resolutions of either .45 microns per pixel or .23 microns per

13
00:01:33,166 --> 00:01:40,432
pixel. Some of the instruments are going to be available in Z-stacking. Most of this is done in bright field but this is starting to take over in

14
00:01:40,433 --> 00:01:50,466
fluorescence as well, and what I’ll talk about in a moment is germane to both instrumentation systems. So, whole slide imaging. You have the

15
00:01:50,466 --> 00:01:56,199
issues of what you’re viewing on a computer screen and we’re going to talk about those a little bit. They’re really not as important in this subject

16
00:01:56,200 --> 00:02:01,200
matter because most of the time you actually want to perform some more sophisticated analysis and that’s what I’m going to talk about,

17
00:02:01,200 --> 00:02:09,233
whether it’s image analysis or stereology. Magnification is manipulated by the software and an image pyramid—I’ll come back and talk about

18
00:02:09,233 --> 00:02:17,066
that in a moment as well—and as I said, it’s directly applicable to the subject at hand. So, why are we talking about this? Well, making a whole

19
00:02:17,066 --> 00:02:27,932
slide image is easy. In fact, it’s too stinking easy. It’s so easy you don’t think about what happened in the box. You need to pay attention to the box or

20
00:02:27,933 --> 00:02:36,966
the box is going to come back and bite you. The box is full of all the assumptions that you need to worry about. So, the assumptions that we’re

21
00:02:36,966 --> 00:02:44,132
going to talk about that are really germane today are contrast, color, and spatial reproduction and scale, because these are the three that we’ve

22
00:02:44,133 --> 00:02:52,366
unearthed lately, as we’ve been working on the NEPTUNE trial, as problematic. Are they going to be problematic in every study? Absolutely not, but

23
00:02:52,366 --> 00:03:01,566
if you’re not aware of them, you can’t ask the right questions. So, here’s an image. This is actually a double stain. It’s a p16/Ki-67 stain of

24
00:03:01,566 --> 00:03:13,699
some ovarian cancer and so it’s a very hard image to view. The Ki-67 is actually a red nucleus and you’ll see some in the stroma on the p16 is

25
00:03:13,700 --> 00:03:21,333
staining the cytoplasm, and as you can see on this screen, you can’t interpret anything in the cytoplasm of those positively stained cells. This is

26
00:03:21,333 --> 00:03:31,033
on one instrument. Here’s the same image on a second instrument; they’re vastly different. If you are lucky…in these projectories you probably

27
00:03:31,033 --> 00:03:42,433
can’t see it, but in truth, you actually can see some of the red nuclei in the stained epithelial sections. Here is what you would see with a

28
00:03:42,433 --> 00:03:49,899
Kohler illumination under a microscope and again, with these projectors, it’s not quite evident but I think it’s pretty obvious that this image is much

29
00:03:49,900 --> 00:04:00,000
closer to this image and it’s not at all related to that image, but it’s exactly the same object imaged on three different instruments. So, this is a

30
00:04:00,000 --> 00:04:07,033
function of the illumination systems that are built into these systems in the designs of the condensers. Matching of the numerical aperture

31
00:04:07,033 --> 00:04:14,266
of the instrument is essential to developing an appropriately contrasted image for interpretation. How does this come back and impact you? Well,

32
00:04:14,266 --> 00:04:20,899
my guess if you’re doing a lot of stereology and trying to count podocytes, this might get you if you’re not using a fluorescent stain; I think it’s

33
00:04:20,900 --> 00:04:32,233
pretty obvious to this community here. Here’s another example of two images of the same core off the same slide imaged on two different

34
00:04:32,233 --> 00:04:40,999
instruments viewed in one software from a different manufacturer, and you can see that there’s a substantial difference in color. Now, for

35
00:04:41,000 --> 00:04:47,400
the pathologists in the room, I don’t think there’s any question they could interpret this accurately; either “H and E” would be adequate, but if this

36
00:04:47,400 --> 00:04:57,133
was eosinophilic esophagitis the pathologists would all be running from the room. But more importantly, once you start looking at this in a

37
00:04:57,133 --> 00:05:05,433
more quantitative fashion—here we’re focusing on the white color balance of these lights—you can see that there’s a substantial difference. One

38
00:05:05,433 --> 00:05:16,033
of them…this is a 256 RGB scale for color that’s used by these instruments. One is giving a white balance around 320…or, I’m sorry, 232, and the

39
00:05:16,033 --> 00:05:24,866
other one is closer to 445. You’ll see a small box—you can barely see it on the screen—that’s highlighting the “tissue with the fingerprint

40
00:05:24,866 --> 00:05:31,866
region,” and here’s an analysis of the region of interest, the staining, the hemotoxin and [---] in staining that was demonstrated on those slides,

41
00:05:31,866 --> 00:05:38,632
showing that although it’s the same slide, there’s actually a different histogram of color intensity between these two instruments. Simple

42
00:05:38,633 --> 00:05:49,599
normalization is not going to be adequate to fix these problems if you look at this further in depth. So, here we take it even a step further—I’m

43
00:05:49,600 --> 00:05:56,733
sorry, the choice of subject field wasn’t great, it is kidney, at least—and we have two instruments, the bottom left and the upper right, or

44
00:05:56,733 --> 00:06:03,033
two images, bottom left and upper right, that are actually from instruments from the same manufacturer but different instruments, and the

45
00:06:03,033 --> 00:06:12,799
other, the upper left, is the one that was from the other manufacturer. And again, here is another further demonstration of the diversity of the color

46
00:06:12,800 --> 00:06:20,166
scales and the shifts that you’re seeing, both in the white balance as well as the scaling, with reference to the staining. What’s interesting is

47
00:06:20,166 --> 00:06:27,899
that you see in the red and the blue channels the difference in the white balance, even between instruments from the same manufacturer. How is

48
00:06:27,900 --> 00:06:36,433
this important? Well, if you’re doing sophisticated image analysis and you’re over-detailing your data, you’re going to have a problem because

49
00:06:36,433 --> 00:06:43,066
your image analysis is not going to work. Now, I’m showing this to you in bright field, but I can replicate this in fluorescence as well, and that’s

50
00:06:43,066 --> 00:06:53,032
even more disturbing. Another example, similar type of issues, obviously. This is a silver stain, and again, it’s a NEPTUNE case and here again

51
00:06:53,033 --> 00:07:01,566
we see the diversity of the distribution of color balances from these slides. That was very concerning to us because it’s going to really make

52
00:07:01,566 --> 00:07:08,032
some of this very difficult. Obviously, this one is demonstrating the differences between both the contrast as well as the stain function for light

53
00:07:08,033 --> 00:07:19,066
transmission. Lastly, here’s an example of scanning a calibration slide. This is not a traditional calibration slide, and we’re showing

54
00:07:19,066 --> 00:07:28,699
the differences in results from two different instruments and what we’re seeing is both a function of a difference in resolution between the

55
00:07:28,700 --> 00:07:35,766
instruments, how many pixels there are per micron, as well as actually the accuracy of the calibration, and as you’ll notice, there is a

56
00:07:35,766 --> 00:07:43,466
difference between these two instruments. You’ll also notice—and it’s a little hard to demonstrate here—that the tools that we have here are a little

57
00:07:43,466 --> 00:07:50,866
crude for actually making these measurements. Again, why is this important in stereology? You’re supposed to be calibrators. You’re supposed to

58
00:07:50,866 --> 00:07:57,332
know how large your objects are. If your instruments are not providing the same measurement for a fixed object, you’re going to

59
00:07:57,333 --> 00:08:05,533
have a problem with scale and that’s exactly what’s happened. It also means that under certain export instances you may actually not

60
00:08:05,533 --> 00:08:18,299
have correct magnification. Here’s our favorite. I’m being facetious for a moment. The image on the left is a corrected image. This again is one of

61
00:08:18,300 --> 00:08:24,966
our sophisticated stage calibers and actually there’s a small defect and if you’ll notice I don’t think it would affect anything, but there’s a small

62
00:08:24,966 --> 00:08:32,699
defect here on the scan image. But, if you look at this image here on the right, you’ll see these two dots versus that dot. The stage actually got

63
00:08:32,700 --> 00:08:40,466
stuck, burned, worn out. The machine didn’t catch it at all. None of the software caught it. You would have thought that the machine should

64
00:08:40,466 --> 00:08:48,566
know how big the object is. It’s supposed to be making an image “is” and that if that image is suddenly a half of a stripe larger or a stripe larger

65
00:08:48,566 --> 00:08:55,099
that it would know that, but in fact, it did not account for this at all. So, one could imagine that if this was a systematic error in your data

66
00:08:55,100 --> 00:09:01,400
collection that you would have real problems. We’ve actually been able to replicate this numerous times in this exact location on this

67
00:09:01,400 --> 00:09:09,366
exact instrument, but have seen this problem on other instruments, and in fact, I’ve seen at least one glomerulus that was expanded by one stripe

68
00:09:09,366 --> 00:09:20,299
in width; that’s not good. So in summary, it’s really essential to evaluate the calibration of your scanners and address intra-scanner variability

69
00:09:20,300 --> 00:09:27,766
and inter-scanner variability. What I didn’t have an opportunity to show you, because I haven’t actually generated the full set of data yet, is the

70
00:09:27,766 --> 00:09:35,399
fact that the bulbs in these instruments degrade and the color balances are going to change over time and that they are not calibrated appropriately.

71
00:09:35,400 --> 00:09:41,700
Obviously, we’ve demonstrated the color is poorly addressed. There is no reference color system currently, although there is some in

72
00:09:41,700 --> 00:09:49,833
development. We understand at least one manufacturer will introduce one in March, and then obviously there are going to be issues of

73
00:09:49,833 --> 00:09:55,599
spatial calibration that need to be addressed. One of the biggest challenges that we’re facing right now is that the tools that have been used to do

74
00:09:55,600 --> 00:10:04,733
this in the past, stage micrometers, are actually completely inadequate. They’re too thick because they’re printed and they’re actually too wide, and

75
00:10:04,733 --> 00:10:11,333
so we’re going to need to move to more photolithography-based measurement tools as well as slides that represent a calibrator across

76
00:10:11,333 --> 00:10:17,466
the entire surface area of the microscope and not just a focused region. Otherwise, you wouldn’t catch those gaps in your stages where they stick

77
00:10:17,466 --> 00:10:27,266
or jump. So, what do you need to do? Well, you need to question everything. I think this community is very good at that, but you need to

78
00:10:27,266 --> 00:10:34,866
question your imaging. You need to use single imagers when it’s feasible for your studies. Many of you are probably well-poised for that; you

79
00:10:34,866 --> 00:10:41,532
probably already do that. Even in your older approaches you used fixed microscopes and don’t change anything. For those of us who are

80
00:10:41,533 --> 00:10:51,299
doing big clinical trials, that’s not feasible. We’re using at least four or five instruments for part of the NEPTUNE clinical trial, which means you need

81
00:10:51,300 --> 00:10:57,000
to know that ahead of time; that’s actually why I generated this data. You need to work on your monitor calibrations and I haven’t talked about

82
00:10:57,000 --> 00:11:07,300
that; that’s much more complicated than you think it is. We have seen one slide on one computer screen concurrently in three different browsers

83
00:11:07,300 --> 00:11:14,566
show three different color sets; that should make us all run from the room. Then obviously, when you’re moving in a multi-scanner environment, we

84
00:11:14,566 --> 00:11:21,432
need to develop reference slides for calibration and then test and validate everything, and demand more from your vendors. Don’t just say,

85
00:11:21,433 --> 00:11:34,733
“Oh, it auto-calibrates!” Obviously, it doesn’t. That’s it. Thank you.




Date Last Updated: 10/5/2012

General Inquiries may be addressed to:
Office of Communications and Public Liaison
NIDDK, NIH
Building 31, Rm 9A06
31 Center Drive, MSC 2560
Bethesda, MD 20892-2560
USA
Phone: 301.496.3583