Mastering Patient Reported Outcome Measures with Dr. Theresa Coles

1 00:00:00,000 --> 00:00:05,860 Welcome to the Clinician Researcher podcast, where academic clinicians learn the skills 2 00:00:05,860 --> 00:00:11,260 to build their own research program, whether or not they have a mentor. 3 00:00:11,260 --> 00:00:17,340 As clinicians, we spend a decade or more as trainees learning to take care of patients. 4 00:00:17,340 --> 00:00:22,380 When we finally start our careers, we want to build research programs, but then we find 5 00:00:22,380 --> 00:00:27,780 that our years of clinical training did not adequately prepare us to lead our research 6 00:00:27,780 --> 00:00:29,200 program. 7 00:00:29,200 --> 00:00:35,480 Through no fault of our own, we struggle to find mentors, and when we can't, we quit. 8 00:00:35,480 --> 00:00:40,580 However, clinicians hold the keys to the greatest research breakthroughs. 9 00:00:40,580 --> 00:00:46,200 For this reason, the Clinician Researcher podcast exists to give academic clinicians 10 00:00:46,200 --> 00:00:51,800 the tools to build their own research program, whether or not they have a mentor. 11 00:00:51,800 --> 00:01:01,000 Now introducing your host, Toyosi Onwuemene. 12 00:01:01,000 --> 00:01:03,200 Welcome to the Clinician Researcher podcast. 13 00:01:03,200 --> 00:01:06,880 I'm your host Toyosi Onwuemene, and it is a pleasure to be talking with you today. 14 00:01:06,880 --> 00:01:10,400 I'm especially excited because I have a special guest today, Dr. Teresa Coles. 15 00:01:10,400 --> 00:01:12,400 Teresa, welcome to the show. 16 00:01:12,400 --> 00:01:13,400 Thank you. 17 00:01:13,400 --> 00:01:14,400 I'm excited to be here. 18 00:01:14,400 --> 00:01:19,400 So, Teresa, how would you introduce yourself to the audience, especially with regard to 19 00:01:19,400 --> 00:01:22,440 your role as an academic faculty member? 20 00:01:22,440 --> 00:01:23,440 Sure. 21 00:01:23,440 --> 00:01:28,920 So, I am an assistant professor, and I am very interested in measuring health. 22 00:01:28,920 --> 00:01:33,000 How can we better measure health, particularly quality of life? 23 00:01:33,000 --> 00:01:40,900 So, Teresa, I first met you, I feel like about a year and a half ago, and at the time we 24 00:01:40,900 --> 00:01:46,820 met because I was interested in health measurements, specifically patient-reported outcome measures, 25 00:01:46,820 --> 00:01:52,240 and I would like you to just talk about how different our perspectives were in terms of 26 00:01:52,240 --> 00:01:56,440 what I was thinking of as a clinician with regard to patient-reported outcome measures 27 00:01:56,440 --> 00:01:59,760 and how you think about it as the expert. 28 00:01:59,760 --> 00:02:01,240 Oh, okay. 29 00:02:01,240 --> 00:02:03,480 Well, let's see what I can remember. 30 00:02:03,480 --> 00:02:11,200 So, I think one of the first things we discussed at the very beginning was what outcomes do 31 00:02:11,200 --> 00:02:14,600 we think might be important to patients? 32 00:02:14,600 --> 00:02:20,040 And there's so many different quality of life outcomes that might be of interest to patients, 33 00:02:20,040 --> 00:02:21,720 but actually we don't really know. 34 00:02:21,720 --> 00:02:28,300 So, I think that was one of our very first conversations is which outcomes should we 35 00:02:28,300 --> 00:02:30,400 focus on and why? 36 00:02:30,400 --> 00:02:32,520 We could go about measuring all types. 37 00:02:32,520 --> 00:02:36,920 We can measure like 90 different types of quality of life outcomes, but that would be 38 00:02:36,920 --> 00:02:39,040 a lot to have patients to do. 39 00:02:39,040 --> 00:02:43,080 So, we try to focus in on what are the most important ones for them. 40 00:02:43,080 --> 00:02:44,080 Sure. 41 00:02:44,080 --> 00:02:50,440 So, one of the things that I thought about was using something like the SF-36, for example, 42 00:02:50,440 --> 00:02:52,240 in my population. 43 00:02:52,240 --> 00:02:57,640 And you talked about the need to have validity evidence to be able to use it in my specific 44 00:02:57,640 --> 00:02:58,800 population. 45 00:02:58,800 --> 00:03:00,680 And I was thinking, validity, what? 46 00:03:00,680 --> 00:03:02,760 So, I want you to speak about that. 47 00:03:02,760 --> 00:03:07,960 Speak about the fact that there are all these, I mean, they're validated or they have validity 48 00:03:07,960 --> 00:03:12,320 evidence, but they may not have it in the disease in which I'm interested in. 49 00:03:12,320 --> 00:03:15,920 How should clinicians be thinking about these tools as they're trying to use them in their 50 00:03:15,920 --> 00:03:16,920 research? 51 00:03:16,920 --> 00:03:17,920 Yeah. 52 00:03:17,920 --> 00:03:18,920 Thank you for bringing that up. 53 00:03:18,920 --> 00:03:21,240 So, here's the situation. 54 00:03:21,240 --> 00:03:24,880 There are a lot of questionnaires, patient reported outcome measures, clinical outcomes 55 00:03:24,880 --> 00:03:26,380 assessments out there. 56 00:03:26,380 --> 00:03:34,600 And just because they exist or even if they've been used in various populations successfully, 57 00:03:34,600 --> 00:03:42,360 that doesn't mean that they have enough validity evidence for every single use case. 58 00:03:42,360 --> 00:03:50,000 So, for example, we might have a patient reported outcome measure or questionnaire that is working 59 00:03:50,000 --> 00:03:56,960 really well for folks who are 65 and older with some sort of physical functioning issues. 60 00:03:56,960 --> 00:03:58,020 Okay. 61 00:03:58,020 --> 00:04:04,360 We may not be able to take that same exact questionnaire and apply it to adolescents. 62 00:04:04,360 --> 00:04:11,040 And the reason for that is because these individuals will have different types of, let's say, physical 63 00:04:11,040 --> 00:04:14,400 functioning impairments or issues. 64 00:04:14,400 --> 00:04:19,600 And if we're asking questions that are not relevant to that patient, then we end up with 65 00:04:19,600 --> 00:04:24,500 biases and we will not be actually measuring the right things. 66 00:04:24,500 --> 00:04:30,320 And then we miss an opportunity to intervene and help folks with their quality of life. 67 00:04:30,320 --> 00:04:32,380 That's a really great point. 68 00:04:32,380 --> 00:04:37,480 Now I recognize it because we've worked together and you've taught me a lot. 69 00:04:37,480 --> 00:04:42,160 But what I do see is a lot of people like, for example, the SF-36, maybe one of the more 70 00:04:42,160 --> 00:04:48,040 widely used ones, why would it be a problem to just take it and apply it to your population, 71 00:04:48,040 --> 00:04:50,360 especially because it's just so widely used? 72 00:04:50,360 --> 00:04:51,360 Yeah. 73 00:04:51,360 --> 00:04:53,200 So there's nothing wrong with SF-36. 74 00:04:53,200 --> 00:04:56,000 I like the SF-36. 75 00:04:56,000 --> 00:05:00,880 The strength of the SF-36 is also its limitation. 76 00:05:00,880 --> 00:05:03,840 So the SF-36 is very broad. 77 00:05:03,840 --> 00:05:07,980 It's used to look at general quality of life. 78 00:05:07,980 --> 00:05:11,760 That's great if you want to detect general quality of life. 79 00:05:11,760 --> 00:05:17,880 If you want to look at specific quality of life impacts based on a particular treatment 80 00:05:17,880 --> 00:05:24,680 or particular condition, something like the SF-36 may not be able to detect those changes 81 00:05:24,680 --> 00:05:29,200 or the very specific issues that patients experience. 82 00:05:29,200 --> 00:05:36,320 So one example we use a lot in the class that Kristi Ziegler and I teach is fatigue, for 83 00:05:36,320 --> 00:05:37,320 example. 84 00:05:37,320 --> 00:05:38,880 Somebody might say that they're fatigued. 85 00:05:38,880 --> 00:05:39,880 Okay. 86 00:05:39,880 --> 00:05:42,760 Well, there's a thousand different measures we can use to evaluate fatigue. 87 00:05:42,760 --> 00:05:44,960 But are we looking at physical fatigue? 88 00:05:44,960 --> 00:05:49,520 Are we looking at emotional fatigue, mental fatigue? 89 00:05:49,520 --> 00:05:53,480 Unless we actually know what those experiences are, we can't measure them effectively. 90 00:05:53,480 --> 00:06:00,080 So SF-36 is great at measuring those broad experiences, but it also is so broad it may 91 00:06:00,080 --> 00:06:04,320 not be able to detect the specific experiences we're looking for. 92 00:06:04,320 --> 00:06:07,560 So both its kind of strength and its weakness. 93 00:06:07,560 --> 00:06:08,560 Okay. 94 00:06:08,560 --> 00:06:11,400 Okay, Teresa, it sounds pretty complicated. 95 00:06:11,400 --> 00:06:17,840 So why should a clinician researcher think about working with an expert like you if they're 96 00:06:17,840 --> 00:06:21,820 trying to use patient reported outcome measures in their projects? 97 00:06:21,820 --> 00:06:27,120 So there are some best practices and how do I say this? 98 00:06:27,120 --> 00:06:29,120 You're going to have to cut this out. 99 00:06:29,120 --> 00:06:32,600 Why would you want to work with somebody like myself? 100 00:06:32,600 --> 00:06:35,040 Because you're awesome. 101 00:06:35,040 --> 00:06:41,800 Well, one of the reasons is because folks like myself who are trained in health measurement 102 00:06:41,800 --> 00:06:46,840 know the methodologies, which include both qualitative and quantitative psychometric 103 00:06:46,840 --> 00:06:52,320 methodologies to ensure we're measuring what it is that we actually want to measure. 104 00:06:52,320 --> 00:07:00,120 If we do not approach measurement with this type of rigor, we risk bias in how we measure. 105 00:07:00,120 --> 00:07:07,080 So we may have great sample sizes and we're measuring something, but we don't actually 106 00:07:07,080 --> 00:07:08,680 really know what that is. 107 00:07:08,680 --> 00:07:13,000 We also risk the potential of missing information. 108 00:07:13,000 --> 00:07:17,760 So we may not be able to detect changes based on treatment or changes based on a condition 109 00:07:17,760 --> 00:07:21,320 that we need to intervene on. 110 00:07:21,320 --> 00:07:22,320 Thank you. 111 00:07:22,320 --> 00:07:25,320 Thank you for highlighting what's important, why it's important. 112 00:07:25,320 --> 00:07:28,440 I mean, because honestly, before we started working together, I really just thought, you 113 00:07:28,440 --> 00:07:31,560 know, you take a measure and there's no reason why you couldn't use it. 114 00:07:31,560 --> 00:07:33,680 But really there's a whole science behind it. 115 00:07:33,680 --> 00:07:37,360 You are clearly a measurement expert and you've been doing this for a long time. 116 00:07:37,360 --> 00:07:40,760 And so one of the things you highlight is just the need for collaboration. 117 00:07:40,760 --> 00:07:42,840 So I'm really excited to talk about collaboration. 118 00:07:42,840 --> 00:07:48,480 So I want to say, Teresa, that it's been an amazing opportunity for me to work with you 119 00:07:48,480 --> 00:07:53,320 as a fellow researcher, as we've submitted some projects. 120 00:07:53,320 --> 00:07:58,520 And one of the things I thought it was really just important to highlight is why should 121 00:07:58,520 --> 00:08:03,040 clinicians and PhD trained researchers work together? 122 00:08:03,040 --> 00:08:04,680 Oh my gosh. 123 00:08:04,680 --> 00:08:06,960 The reasons are endless. 124 00:08:06,960 --> 00:08:11,680 So I think I'll start with like a personal example. 125 00:08:11,680 --> 00:08:16,560 For somebody like that, myself that's trained in methodology specifically, we're great at 126 00:08:16,560 --> 00:08:22,840 coming up with methods, but we are not great at knowing what the clinical problems are. 127 00:08:22,840 --> 00:08:29,340 So where are the opportunities to actually use those methods and make a difference? 128 00:08:29,340 --> 00:08:32,280 So we often have what we call hammers, right? 129 00:08:32,280 --> 00:08:37,040 So like we might want to use the same hammers of methodologies across many studies. 130 00:08:37,040 --> 00:08:38,160 That's fine. 131 00:08:38,160 --> 00:08:40,960 But really at the end of the day, that's not the most important thing. 132 00:08:40,960 --> 00:08:46,480 The most important thing is that we're pushing science forward clinically and we're causing 133 00:08:46,480 --> 00:08:51,420 improvement in quality of life and survival for patients. 134 00:08:51,420 --> 00:08:57,760 So in order for us to like meet the needs of that issue, we have to collaborate with 135 00:08:57,760 --> 00:09:00,640 clinicians who know what the issues are. 136 00:09:00,640 --> 00:09:06,240 Not only that, but the clinicians kind of throughout a project, like for example, when 137 00:09:06,240 --> 00:09:10,840 I collaborate with clinicians on a project, they keep it centered on the clinical issues 138 00:09:10,840 --> 00:09:13,360 and the clinical relevance of what it is that we're doing. 139 00:09:13,360 --> 00:09:15,540 So what are some patient experiences? 140 00:09:15,540 --> 00:09:17,440 What are clinical experiences? 141 00:09:17,440 --> 00:09:21,800 The clinicians also have direct contact with patients. 142 00:09:21,800 --> 00:09:24,200 So how might we recruit patients? 143 00:09:24,200 --> 00:09:29,440 They also have knowledge of what do the results actually mean, right? 144 00:09:29,440 --> 00:09:34,660 So we, you know, like from a methodological standpoint, we have a certain perspective 145 00:09:34,660 --> 00:09:39,000 on what these results might mean and we can provide percentages and tell you about IRT 146 00:09:39,000 --> 00:09:40,000 loadings. 147 00:09:40,000 --> 00:09:44,440 Like we can tell you all sorts of things, but at the end of the day, what does it mean? 148 00:09:44,440 --> 00:09:45,920 How do we interpret those results? 149 00:09:45,920 --> 00:09:50,840 And I think that's extremely important to work with clinicians on. 150 00:09:50,840 --> 00:09:51,840 Absolutely. 151 00:09:51,840 --> 00:09:56,620 You know, one thing you said reminded me of something that's come up before is the difference 152 00:09:56,620 --> 00:10:02,840 between developing a measure with developing validity evidence for a measure for clinical 153 00:10:02,840 --> 00:10:05,600 use versus for like a clinical trial. 154 00:10:05,600 --> 00:10:07,560 Can you just speak to that difference? 155 00:10:07,560 --> 00:10:08,560 Sure. 156 00:10:08,560 --> 00:10:16,360 So there's a lot of common methodologies we would use for measures that we might use in 157 00:10:16,360 --> 00:10:18,800 clinical care versus clinical trials. 158 00:10:18,800 --> 00:10:21,440 It's all about the use case. 159 00:10:21,440 --> 00:10:26,400 So in clinical trials, typically we would use, let's say, a patient reported outcome 160 00:10:26,400 --> 00:10:34,480 measure to look at some sort of quality of life outcome over time and answer the question 161 00:10:34,480 --> 00:10:39,360 whether the treatment is making a difference, improving or worsening whatever that outcome 162 00:10:39,360 --> 00:10:41,760 is, or maybe not doing either of those. 163 00:10:41,760 --> 00:10:45,220 Maybe that outcome is just staple. 164 00:10:45,220 --> 00:10:49,320 That's like the primary use case in clinical trials and that could be a primary endpoint, 165 00:10:49,320 --> 00:10:52,500 secondary endpoint, exploratory endpoint. 166 00:10:52,500 --> 00:10:56,200 Another use case in clinical trials is screening. 167 00:10:56,200 --> 00:11:01,520 You can use a patient reported outcome measure to screen individuals for participation in 168 00:11:01,520 --> 00:11:04,520 the trial, so like inclusion and exclusion criteria. 169 00:11:04,520 --> 00:11:09,440 How we set up the PRO measure and provide the validity evidence for those two different 170 00:11:09,440 --> 00:11:13,360 use cases requires two different types of methodologies. 171 00:11:13,360 --> 00:11:15,040 So that's just even within clinical trials. 172 00:11:15,040 --> 00:11:19,320 And then we look at clinical care, there's lots of different ways we can use patient 173 00:11:19,320 --> 00:11:21,240 reported outcome measures. 174 00:11:21,240 --> 00:11:27,360 One is, similar to clinical trials, we can track patients' outcomes over time. 175 00:11:27,360 --> 00:11:33,320 One additional thing we might do in clinical care is not only continually track those outcomes 176 00:11:33,320 --> 00:11:40,400 over time, but identify specific instances for specific patients when they've either 177 00:11:40,400 --> 00:11:46,620 decreased or increased on an outcome in an unacceptable way, where we know some sort 178 00:11:46,620 --> 00:11:50,020 of intervention needs to take place. 179 00:11:50,020 --> 00:11:55,780 That is a really important clinical care application and requires particular methods. 180 00:11:55,780 --> 00:12:00,760 Same thing, so like most of us are familiar with something like the PHQ-9, which is a 181 00:12:00,760 --> 00:12:02,840 depression screener. 182 00:12:02,840 --> 00:12:07,720 That is something that we use in clinical care and the methodologies behind something 183 00:12:07,720 --> 00:12:12,040 like the PHQ-9 are unique to screeners. 184 00:12:12,040 --> 00:12:15,840 So do you want me to talk about the different methodologies or shall I stop there? 185 00:12:15,840 --> 00:12:22,720 Please, I'm learning a lot and I do feel it's important because it's nuanced, so I think 186 00:12:22,720 --> 00:12:24,200 it's helpful to know. 187 00:12:24,200 --> 00:12:25,200 Sure. 188 00:12:25,200 --> 00:12:30,320 So I wish I had my slides so I could show everybody. 189 00:12:30,320 --> 00:12:38,440 So for clinical trials, anytime we're looking at outcomes over time, we are interested in 190 00:12:38,440 --> 00:12:40,620 lots of different types of evidence. 191 00:12:40,620 --> 00:12:44,220 That evidence is based on what is the use case. 192 00:12:44,220 --> 00:12:49,440 So if we're looking at something over time, we first need to ensure that we're measuring 193 00:12:49,440 --> 00:12:50,720 what we think we're measuring. 194 00:12:50,720 --> 00:12:52,980 That's a type of validity. 195 00:12:52,980 --> 00:12:57,960 We need to look at how we might score that measure and what would the scores actually 196 00:12:57,960 --> 00:12:58,960 mean. 197 00:12:58,960 --> 00:13:04,980 That gets into methodologies for understanding what we call the dimensionality of that measure. 198 00:13:04,980 --> 00:13:08,440 So how many scores might there be coming out of that measure? 199 00:13:08,440 --> 00:13:12,720 For example, the SF-36 has multiple different types of scores for different dimensions of 200 00:13:12,720 --> 00:13:14,800 quality of life. 201 00:13:14,800 --> 00:13:19,360 Another thing we need to look at if we're looking at scores over time is reliability. 202 00:13:19,360 --> 00:13:28,540 So can we be sure that if somebody is, let's say, rated a 33 on day one and they have no 203 00:13:28,540 --> 00:13:34,880 change in their quality of life by day seven, are they still going to be a 33 or somewhere 204 00:13:34,880 --> 00:13:36,240 close to that? 205 00:13:36,240 --> 00:13:41,200 That is ensuring that type of reliability is called test-retest reliability. 206 00:13:41,200 --> 00:13:46,360 Another important aspect if we're measuring quality of life over time is being able to 207 00:13:46,360 --> 00:13:48,800 detect change. 208 00:13:48,800 --> 00:13:55,760 So just like with a thermometer, if I get a fever, that thermometer we hope is going 209 00:13:55,760 --> 00:13:57,360 to detect that fever, right? 210 00:13:57,360 --> 00:13:59,240 If it doesn't, we have problems. 211 00:13:59,240 --> 00:14:01,580 Same thing with quality of life measures. 212 00:14:01,580 --> 00:14:07,720 If we see an improvement or a worsening in quality of life, we use methodologies to be 213 00:14:07,720 --> 00:14:13,000 able to ensure that we can detect that change. 214 00:14:13,000 --> 00:14:20,440 Finally, all of this kind of filters into the final issue, which is how do we interpret 215 00:14:20,440 --> 00:14:21,680 those scores? 216 00:14:21,680 --> 00:14:26,680 So at the end of the day, all the methodology that we're using is all based in supporting 217 00:14:26,680 --> 00:14:29,880 the interpretation of those scores from the measure. 218 00:14:29,880 --> 00:14:36,200 We cannot interpret or we can't do a great job of interpreting scores if we don't have 219 00:14:36,200 --> 00:14:40,800 evidence to prove that they're reliable or we don't have evidence to show that we're 220 00:14:40,800 --> 00:14:46,880 able to detect change over time or if we don't have evidence to show what is the dimensionality 221 00:14:46,880 --> 00:14:49,900 of our measure, what is it that we're actually measuring? 222 00:14:49,900 --> 00:14:52,020 So that's just one example in clinical trials. 223 00:14:52,020 --> 00:14:54,420 Those are typical things that we'll look for. 224 00:14:54,420 --> 00:14:55,420 That's really excellent. 225 00:14:55,420 --> 00:14:56,420 Thank you for summarizing that. 226 00:14:56,420 --> 00:15:03,040 That was really, really well summarized and clearly you've been doing this a long time. 227 00:15:03,040 --> 00:15:08,400 So for a clinician like me who says, okay, I want to work with a health measurement expert 228 00:15:08,400 --> 00:15:15,400 like you, what kinds of questions should I be asking and what kinds of things would you 229 00:15:15,400 --> 00:15:19,280 want to know to help decide if this collaboration makes sense? 230 00:15:19,280 --> 00:15:20,960 Oh, great question. 231 00:15:20,960 --> 00:15:28,580 So I think the key piece about working with folks who are health measurements focused 232 00:15:28,580 --> 00:15:34,120 is one, learning what their interests are. 233 00:15:34,120 --> 00:15:38,040 Some of us, for example, specialize in different types of populations. 234 00:15:38,040 --> 00:15:42,980 So I typically work in studies that focus on adults. 235 00:15:42,980 --> 00:15:46,920 Some of my colleagues focus on pediatric patients. 236 00:15:46,920 --> 00:15:52,800 Those are slightly different methodologies that we might use to address these different 237 00:15:52,800 --> 00:15:57,000 populations and how we might ask questions about quality of life for a three-year-old 238 00:15:57,000 --> 00:16:01,240 versus an 18-year-old versus a 45-year-old. 239 00:16:01,240 --> 00:16:08,920 Another piece is some of us have a lot of experience in certain conditions or diseases. 240 00:16:08,920 --> 00:16:15,360 So for example, at UNC, there are a number of people who are really focused in cancer 241 00:16:15,360 --> 00:16:20,640 and have lots of expertise on measuring quality of life specifically in cancer populations. 242 00:16:20,640 --> 00:16:22,360 Some of us are more generalists. 243 00:16:22,360 --> 00:16:26,120 So it's like, what is our background and where are we coming from and then how can we contribute 244 00:16:26,120 --> 00:16:28,000 to the study? 245 00:16:28,000 --> 00:16:33,840 So another piece is I would say in most studies where we're doing prospective data collection, 246 00:16:33,840 --> 00:16:37,080 there's some sort of measurement component. 247 00:16:37,080 --> 00:16:42,080 And a collaboration with a measurement colleague may be really small where that person is just 248 00:16:42,080 --> 00:16:45,120 kind of like providing advice. 249 00:16:45,120 --> 00:16:46,960 And they may not even actually be under study. 250 00:16:46,960 --> 00:16:50,360 They just may be behind the scenes giving you some ideas. 251 00:16:50,360 --> 00:16:55,640 And then all the way up to something like a co-PI, which is what you and I have done 252 00:16:55,640 --> 00:16:56,640 before. 253 00:16:56,640 --> 00:17:03,240 So as a co-PI, those would be very measurement specific studies where we are trying to optimize 254 00:17:03,240 --> 00:17:06,440 measurement of some sort of quality of life outcome. 255 00:17:06,440 --> 00:17:16,240 I also think another question that clinicians might want to bring to the table with potential 256 00:17:16,240 --> 00:17:27,360 health measurement colleagues is what level of involvement do they like to have? 257 00:17:27,360 --> 00:17:33,480 So that helps to kind of like focus in on I think the size of the study that you can 258 00:17:33,480 --> 00:17:38,120 pull off and what types of studies you can move forward. 259 00:17:38,120 --> 00:17:42,960 And a third thing that I would recommend is if that health measurement researcher is not 260 00:17:42,960 --> 00:17:49,120 the right person, then who else might be because there's so many connections among all of us 261 00:17:49,120 --> 00:17:54,280 and we have a good sense, like a telephone book in our heads of who's an expert in X, 262 00:17:54,280 --> 00:17:59,640 Y, or Z, and then we can connect to you with the right health measurement person. 263 00:17:59,640 --> 00:18:00,640 That's good. 264 00:18:00,640 --> 00:18:01,640 That's good. 265 00:18:01,640 --> 00:18:02,640 Thank you for sharing that. 266 00:18:02,640 --> 00:18:05,960 I wonder from your perspective in working with clinicians, what are some concerns that 267 00:18:05,960 --> 00:18:06,960 you have? 268 00:18:06,960 --> 00:18:17,960 I think at the get-go, sometimes clinicians don't have a sense for kind of like the best 269 00:18:17,960 --> 00:18:21,120 practices for health measurement. 270 00:18:21,120 --> 00:18:29,520 So unfortunately, sometimes a lot of our fundamental work to get a study up and running measurement-wise 271 00:18:29,520 --> 00:18:31,220 takes a lot of time. 272 00:18:31,220 --> 00:18:35,540 So there's like formative qualitative work or formative literature reviews that need 273 00:18:35,540 --> 00:18:42,080 to happen and that just sucks out a lot of time where clinicians feel like they could 274 00:18:42,080 --> 00:18:46,240 just move forward really quickly and just throw some patient-reported outcome measures 275 00:18:46,240 --> 00:18:51,400 or COAs into a trial or in clinical care. 276 00:18:51,400 --> 00:18:52,720 People do that all the time. 277 00:18:52,720 --> 00:18:58,160 I think, and sometimes it's fine because that measure has already has validity evidence. 278 00:18:58,160 --> 00:19:04,840 But a lot of times, well, I'm not going to say a lot of times, sometimes it's not okay. 279 00:19:04,840 --> 00:19:08,400 So I don't even remember what the question was anymore. 280 00:19:08,400 --> 00:19:11,200 Just concerns that you have about working with clinicians. 281 00:19:11,200 --> 00:19:13,840 Well, I love working with clinicians. 282 00:19:13,840 --> 00:19:17,840 I absolutely love working with them because they keep me focused on what are the actual 283 00:19:17,840 --> 00:19:18,960 issues. 284 00:19:18,960 --> 00:19:29,200 I can go down my circles and caves of measurement ideas and math and qualitative inquiry, but 285 00:19:29,200 --> 00:19:32,080 at the end of the day, is that really going to make a difference? 286 00:19:32,080 --> 00:19:37,040 So I feel like the clinicians help keep me centered on what are the actual clinical issues 287 00:19:37,040 --> 00:19:41,360 that are going to move us forward in terms of outcomes of research. 288 00:19:41,360 --> 00:19:45,320 So how are we going to improve patient's quality of life? 289 00:19:45,320 --> 00:19:46,320 I like that. 290 00:19:46,320 --> 00:19:50,240 So it's almost like what you're really speaking to is the fact that you're a partner in the 291 00:19:50,240 --> 00:19:51,240 work. 292 00:19:51,240 --> 00:19:55,720 And earlier you started talking about the speed to move forward because there's a sense 293 00:19:55,720 --> 00:20:01,380 that starting the project and administering the outcomes, administering the PRO measures 294 00:20:01,380 --> 00:20:03,280 is the most important thing. 295 00:20:03,280 --> 00:20:09,040 But really there is a need for clinicians to understand that there's a lot of foundational 296 00:20:09,040 --> 00:20:13,840 work that may not feel like active work, but is really important so that when they're moving 297 00:20:13,840 --> 00:20:17,580 forward with the big clinical trial, it's valid. 298 00:20:17,580 --> 00:20:21,160 And that the questions that they're asking, the answers they're getting are actually answers 299 00:20:21,160 --> 00:20:22,520 to the questions they want. 300 00:20:22,520 --> 00:20:23,520 They want. 301 00:20:23,520 --> 00:20:24,520 Yeah, absolutely. 302 00:20:24,520 --> 00:20:32,000 But formative work, it's like underpinning rigor that will pay off the dividends when 303 00:20:32,000 --> 00:20:35,840 you want to use those data over and over again or continue to expand your project or your 304 00:20:35,840 --> 00:20:37,400 research program. 305 00:20:37,400 --> 00:20:44,280 Okay, so it's worth doing it well, especially the first time you're trying to get evidence 306 00:20:44,280 --> 00:20:45,280 for a measure. 307 00:20:45,280 --> 00:20:46,280 I love it. 308 00:20:46,280 --> 00:20:51,960 Okay, so Teresa, one of the things that I think we come up against is that this kind 309 00:20:51,960 --> 00:20:59,080 of work when it comes to PRO measures is not necessarily, I think, as well-funded as perhaps 310 00:20:59,080 --> 00:21:03,200 if we came in just proposing a clinical trial doing some other things. 311 00:21:03,200 --> 00:21:09,160 So I'm curious to know what kind of funding opportunities may exist a clinician researcher 312 00:21:09,160 --> 00:21:12,120 should be thinking about for this kind of work. 313 00:21:12,120 --> 00:21:13,120 Great question. 314 00:21:13,120 --> 00:21:16,720 So here's the deal. 315 00:21:16,720 --> 00:21:25,240 Here's what I have learned slowly over time, primarily in schools of public health or population 316 00:21:25,240 --> 00:21:32,160 health or schools of medicine, what we find is that there's an emphasis on getting funded 317 00:21:32,160 --> 00:21:37,520 through NIH, and that is fantastic. 318 00:21:37,520 --> 00:21:43,360 The deal with NIH is that they're typically looking for studies that improve patient outcomes. 319 00:21:43,360 --> 00:21:50,000 So that could include quality of life outcomes, right? 320 00:21:50,000 --> 00:21:54,200 A lot of times, methodological work is not actually improving the outcome. 321 00:21:54,200 --> 00:21:59,120 It's helping us better assess whatever that quality of life outcome is. 322 00:21:59,120 --> 00:22:07,520 So for studies like what I do, NIH, sometimes we can get funded by NIH depending upon what 323 00:22:07,520 --> 00:22:10,120 is the focus of that RFA. 324 00:22:10,120 --> 00:22:13,880 But it's often not our biggest funder. 325 00:22:13,880 --> 00:22:20,680 Other funders include a myriad of options, and I feel like there's more that I haven't 326 00:22:20,680 --> 00:22:24,640 even discovered yet, but one is FDA. 327 00:22:24,640 --> 00:22:31,880 So FDA is very interested in methodological rigor for the use in clinical trials so that 328 00:22:31,880 --> 00:22:40,840 we're evaluating patient reported outcomes that are meaningful to the patient and with 329 00:22:40,840 --> 00:22:43,200 rigor. 330 00:22:43,200 --> 00:22:48,160 So FDA has what they call the broad agency announcement that comes out in the fall of 331 00:22:48,160 --> 00:22:54,120 every year, and that is a big honking proposal. 332 00:22:54,120 --> 00:22:56,440 So there's two stages to that. 333 00:22:56,440 --> 00:23:03,240 There is a 10-pager, sort of like a LOI, like a letter of intent, and then there's a 50-pager 334 00:23:03,240 --> 00:23:05,480 if you get invited to do the full thing. 335 00:23:05,480 --> 00:23:09,960 And that's just volume one because there's also volume two. 336 00:23:09,960 --> 00:23:19,520 However, the VA's, and I've had two of them, are a really great way to focus in on methodological 337 00:23:19,520 --> 00:23:20,520 issues. 338 00:23:20,520 --> 00:23:25,720 And FDA loves it because they want to support rigor in clinical trials for outcomes measurement. 339 00:23:25,720 --> 00:23:29,120 They're very supportive of that. 340 00:23:29,120 --> 00:23:35,840 Another opportunity with FDA for some institutions is the FDA CERC. 341 00:23:35,840 --> 00:23:40,600 So some institutions, including Duke, UNC, NC State, were in a CERC, and there's other 342 00:23:40,600 --> 00:23:46,360 CERCs with like, I think Yale has one, Mayo has one. 343 00:23:46,360 --> 00:23:51,960 These are opportunities to work directly with FDA and address kind of like smaller research 344 00:23:51,960 --> 00:23:54,160 opportunities. 345 00:23:54,160 --> 00:24:03,040 Maybe I would say between 250,000 to 750,000-ish in size, but they are more frequently coming 346 00:24:03,040 --> 00:24:05,200 through and really great opportunities. 347 00:24:05,200 --> 00:24:06,920 Oh, you'll have to tell us. 348 00:24:06,920 --> 00:24:07,920 What is CERC? 349 00:24:07,920 --> 00:24:09,600 What does that stand for? 350 00:24:09,600 --> 00:24:13,440 Centers of Excellence in Regulatory Science and Innovation, CERCs. 351 00:24:13,440 --> 00:24:14,440 Okay. 352 00:24:14,440 --> 00:24:21,520 So that's University of Maryland, Johns Hopkins, Yale, Mayo, University of California, San 353 00:24:21,520 --> 00:24:22,520 Francisco, and Stanford. 354 00:24:22,520 --> 00:24:29,720 So if any of your listeners are in those institutions, and Duke, and UNC, and NC State, then you're 355 00:24:29,720 --> 00:24:31,160 covered by a CERC. 356 00:24:31,160 --> 00:24:33,800 Yeah, so there's a few of us. 357 00:24:33,800 --> 00:24:34,800 Okay. 358 00:24:34,800 --> 00:24:35,800 Okay. 359 00:24:35,800 --> 00:24:36,800 Okay. 360 00:24:36,800 --> 00:24:41,840 So FDA is a potential funder and thinking about these CERC opportunities, if your institution 361 00:24:41,840 --> 00:24:45,600 is connected to that, what are their opportunities? 362 00:24:45,600 --> 00:24:47,880 Another opportunity is PCORI. 363 00:24:47,880 --> 00:24:55,200 So PCORI focuses on comparative effectiveness research. 364 00:24:55,200 --> 00:25:00,440 So if you are doing a study that might be comparing different interventions and looking 365 00:25:00,440 --> 00:25:04,760 at those outcomes, that might be a good fit for you. 366 00:25:04,760 --> 00:25:08,360 We've looked at the engagement awards together. 367 00:25:08,360 --> 00:25:11,640 Actually one of the things that I did not mention before that's really important to 368 00:25:11,640 --> 00:25:18,160 methodologies for health measurement is inclusion of patients and other stakeholders in what 369 00:25:18,160 --> 00:25:19,160 you're doing. 370 00:25:19,160 --> 00:25:23,400 So a lot of times we have stakeholder panels to kind of guide our measurement approaches, 371 00:25:23,400 --> 00:25:31,040 and PCORI is really a great funder for those types of opportunities. 372 00:25:31,040 --> 00:25:33,600 Other funders include industry. 373 00:25:33,600 --> 00:25:38,120 So I'm actually working with David Leverins from Duke. 374 00:25:38,120 --> 00:25:44,280 He's funded by Pfizer and supporting him on a project where we're developing a super short 375 00:25:44,280 --> 00:25:47,520 questionnaire for use in clinical care. 376 00:25:47,520 --> 00:25:50,960 Sometimes there's AHRQ. 377 00:25:50,960 --> 00:25:53,760 Typically those are very health services related. 378 00:25:53,760 --> 00:26:04,480 So if you have a health services project, let's say you are going to improve resource utilization 379 00:26:04,480 --> 00:26:11,000 using your patient reported outcome measure AHRQ might be a great fit. 380 00:26:11,000 --> 00:26:12,000 That's cool. 381 00:26:12,000 --> 00:26:13,000 Thanks, Teresa. 382 00:26:13,000 --> 00:26:20,480 So NIH, we all know, but also looking at FDA, the PCORI and then industry as well and AHRQ. 383 00:26:20,480 --> 00:26:21,480 Awesome. 384 00:26:21,480 --> 00:26:22,480 Oh my goodness. 385 00:26:22,480 --> 00:26:23,480 It's so funny. 386 00:26:23,480 --> 00:26:25,800 When you started out, you were like, oh, there's lots of options. 387 00:26:25,800 --> 00:26:26,800 I'm like, what? 388 00:26:26,800 --> 00:26:28,800 There are lots of options, but it's good. 389 00:26:28,800 --> 00:26:29,960 It's good to know because you're right. 390 00:26:29,960 --> 00:26:35,280 I think as clinician researchers, we're really focused on NIH and really thinking about which 391 00:26:35,280 --> 00:26:39,360 funder really aligns with what you're looking at. 392 00:26:39,360 --> 00:26:46,200 And it sounds like clinicians and researchers should think about these other potential opportunities. 393 00:26:46,200 --> 00:26:47,200 That is great. 394 00:26:47,200 --> 00:26:48,200 Okay. 395 00:26:48,200 --> 00:26:56,680 So I'm wondering what question should I ask you as someone who's a measurement expert 396 00:26:56,680 --> 00:27:01,280 and I'm a clinician researcher, what question should I be asking you that I might not think 397 00:27:01,280 --> 00:27:02,280 to ask you? 398 00:27:02,280 --> 00:27:03,280 Oh, let's see. 399 00:27:03,280 --> 00:27:14,200 Are you thinking about for like grant submission, study design, what might be happening in clinical 400 00:27:14,200 --> 00:27:15,200 care? 401 00:27:15,200 --> 00:27:20,960 I think all of the above because what I'm hearing you say is that your work is relevant, 402 00:27:20,960 --> 00:27:25,060 not just for research, but even in clinical care as well. 403 00:27:25,060 --> 00:27:27,000 And so maybe we should ask the question two ways. 404 00:27:27,000 --> 00:27:34,200 Maybe the first question is as a clinician, what should I be thinking about and what questions 405 00:27:34,200 --> 00:27:38,500 should I ask my health measurement person who's at my institution that may be relevant 406 00:27:38,500 --> 00:27:39,960 to me in the clinical space? 407 00:27:39,960 --> 00:27:41,540 Oh yeah. 408 00:27:41,540 --> 00:27:48,760 So in the clinical space, one really great kind of low-hanging fruit is to work with 409 00:27:48,760 --> 00:27:54,160 health measurement experts to look at questionnaires you might already be administering in clinical 410 00:27:54,160 --> 00:27:55,760 care. 411 00:27:55,760 --> 00:28:00,240 Are they actually up to the task of doing whatever it is that you want them to do? 412 00:28:00,240 --> 00:28:01,240 Right? 413 00:28:01,240 --> 00:28:06,840 The health measurement expert will give you the support to be able to look at the evidence 414 00:28:06,840 --> 00:28:13,080 as there for those measures and also how you can use those scores to kind of support decision 415 00:28:13,080 --> 00:28:17,680 making within clinical care. 416 00:28:17,680 --> 00:28:24,880 Another opportunity or question that you could work with health measurement folks on is what 417 00:28:24,880 --> 00:28:30,860 questionnaires might I want to administer in clinical care? 418 00:28:30,860 --> 00:28:35,280 And then I can guarantee you that the health measurement person is going to ask you, well, 419 00:28:35,280 --> 00:28:37,400 how would you want to use the scores? 420 00:28:37,400 --> 00:28:38,440 Okay? 421 00:28:38,440 --> 00:28:44,920 So there are a lot of really great opportunities to incorporate screeners in clinical care 422 00:28:44,920 --> 00:28:48,680 to identify patients that really need help at that time. 423 00:28:48,680 --> 00:28:55,960 So if there is, for example, a symptom that you need to know more about, maybe it's pain, 424 00:28:55,960 --> 00:29:02,040 maybe it's itch, we can either implement or identify questionnaires that might help you 425 00:29:02,040 --> 00:29:04,320 be able to do that quickly. 426 00:29:04,320 --> 00:29:12,680 And then a third question that's a little bit more broader is, you know, tell us about 427 00:29:12,680 --> 00:29:17,920 your research program and then where might health measurements kind of support and help 428 00:29:17,920 --> 00:29:21,840 the rigor of what it is that you're trying to do? 429 00:29:21,840 --> 00:29:22,840 I love it. 430 00:29:22,840 --> 00:29:26,280 You know, the first one of the thoughts that came to my mind is, you know, our patients 431 00:29:26,280 --> 00:29:30,200 are sitting in the waiting room on an awful long time. 432 00:29:30,200 --> 00:29:35,600 And so could there be a measure that's actually relevant and valid for that particular question 433 00:29:35,600 --> 00:29:40,040 that the clinician has that they could fill out in the waiting room that actually contributes 434 00:29:40,040 --> 00:29:43,760 to the information the clinician needs to help further their care? 435 00:29:43,760 --> 00:29:50,280 Yeah, that is literally the best time to get patients to ask questions. 436 00:29:50,280 --> 00:29:54,840 We've seen it anecdotally and we've also like evidence about that that's published. 437 00:29:54,840 --> 00:29:55,840 Great. 438 00:29:55,840 --> 00:29:56,840 Okay. 439 00:29:56,840 --> 00:29:57,840 Okay. 440 00:29:57,840 --> 00:29:59,080 How about how about research? 441 00:29:59,080 --> 00:30:00,080 What questions? 442 00:30:00,080 --> 00:30:03,840 I mean, I think we touched on a lot of them earlier, but what question is on asked that 443 00:30:03,840 --> 00:30:06,880 we should we should definitely make sure people are considering? 444 00:30:06,880 --> 00:30:13,280 Well, I'll tell you what's asked that I would like for people to potentially consider modifying. 445 00:30:13,280 --> 00:30:21,560 So people, people might come to us and say, what questionnaire should I include in my 446 00:30:21,560 --> 00:30:23,320 research program? 447 00:30:23,320 --> 00:30:26,000 And it's that's not really the question to be asking. 448 00:30:26,000 --> 00:30:34,280 The question is, how can I measure this thing that I want to measure? 449 00:30:34,280 --> 00:30:38,080 Because it's not so much about it is about the questionnaire, but more importantly, it's 450 00:30:38,080 --> 00:30:40,760 about what it is that you're trying to measure. 451 00:30:40,760 --> 00:30:48,660 So one thing that I would like to communicate is that it's like I don't have a Rolodex of 452 00:30:48,660 --> 00:30:53,720 questionnaires on my desk where I can say, oh, yeah, you know, here's the 50 measures 453 00:30:53,720 --> 00:30:56,720 on fatigue and let's choose one of them. 454 00:30:56,720 --> 00:31:01,880 There's a lot of work that goes into choosing the most appropriate measure for any given 455 00:31:01,880 --> 00:31:02,880 circumstance. 456 00:31:02,880 --> 00:31:07,800 A lot of that is just background literature review, reading what evidence is there, what 457 00:31:07,800 --> 00:31:13,280 work has already been done, and then what we also what we know about our population 458 00:31:13,280 --> 00:31:19,760 of interest and then how that matches to what the questionnaire is actually evaluating. 459 00:31:19,760 --> 00:31:21,760 That is so, so awesome. 460 00:31:21,760 --> 00:31:26,280 And to be honest, I started laughing because I was like, yeah, that's what I thought. 461 00:31:26,280 --> 00:31:32,040 I thought you just you know, all the questionnaires that are out there. 462 00:31:32,040 --> 00:31:33,040 I do. 463 00:31:33,040 --> 00:31:38,760 I think that's a common I think this is a common misperception, you know, misperception, because 464 00:31:38,760 --> 00:31:43,200 I think some people think that we're kind of like librarians of instruments and we're 465 00:31:43,200 --> 00:31:44,200 not. 466 00:31:44,200 --> 00:31:45,200 Thank you. 467 00:31:45,200 --> 00:31:49,000 Thank you for clarifying that you're not. 468 00:31:49,000 --> 00:31:53,800 OK, so what I should do is ask the question is be clear about what I want to measure. 469 00:31:53,800 --> 00:31:56,440 OK, let's use fatigue as an example. 470 00:31:56,440 --> 00:31:59,320 OK, if I say I want to measure fatigue, is that fatigue? 471 00:31:59,320 --> 00:32:01,900 Is that enough? 472 00:32:01,900 --> 00:32:05,320 So here's some of the questions that I will ask you. 473 00:32:05,320 --> 00:32:11,040 If you tell me you want to measure fatigue, I will ask you, well, what is it that you'd 474 00:32:11,040 --> 00:32:12,460 need to know about fatigue? 475 00:32:12,460 --> 00:32:14,560 What is the purpose of measuring? 476 00:32:14,560 --> 00:32:15,560 Right. 477 00:32:15,560 --> 00:32:16,560 Is this a clinical care application? 478 00:32:16,560 --> 00:32:19,280 Is this clinical trials, clinical research? 479 00:32:19,280 --> 00:32:22,400 How often do you want to measure fatigue? 480 00:32:22,400 --> 00:32:25,280 How do you want to use those scores to make decisions? 481 00:32:25,280 --> 00:32:28,660 I think that's like that final question is like zing. 482 00:32:28,660 --> 00:32:31,040 That's what really gets us the answer. 483 00:32:31,040 --> 00:32:37,680 So that helps us really define and zone in to be able to get you the measure that is 484 00:32:37,680 --> 00:32:38,920 going to be most useful. 485 00:32:38,920 --> 00:32:39,920 Sure, sure. 486 00:32:39,920 --> 00:32:41,400 So I'm hearing that. 487 00:32:41,400 --> 00:32:44,600 So you want to, you know, you want to understand the concept you want to measure. 488 00:32:44,600 --> 00:32:48,360 So for example, fatigue, but you also want to be clear about why you want to measure 489 00:32:48,360 --> 00:32:53,320 it, what you're going to do once you find the fatigue and how you're going to respond 490 00:32:53,320 --> 00:32:54,320 to that. 491 00:32:54,320 --> 00:32:55,320 Yeah, absolutely. 492 00:32:55,320 --> 00:32:56,320 OK, all right. 493 00:32:56,320 --> 00:32:57,320 This is very nuanced. 494 00:32:57,320 --> 00:33:00,840 OK, everybody who's listening, if you are trying to use a patient report at outcome 495 00:33:00,840 --> 00:33:05,080 measure in your study, you got to get with an expert. 496 00:33:05,080 --> 00:33:07,200 And they all have this amazing network. 497 00:33:07,200 --> 00:33:09,440 So if you find one, they can find you the right person. 498 00:33:09,440 --> 00:33:13,360 OK, I think there's one more thing I want to ask, because I feel like this is something 499 00:33:13,360 --> 00:33:18,080 that I learned from you too, is so when I think about patient report outcome measures, 500 00:33:18,080 --> 00:33:21,200 I just think of them as just, oh, it's patient reported outcome measures, and they're so 501 00:33:21,200 --> 00:33:22,200 important. 502 00:33:22,200 --> 00:33:27,280 But really, there's a bigger context of clinical outcome assessments under which patient reported 503 00:33:27,280 --> 00:33:30,240 outcome measures are a subset. 504 00:33:30,240 --> 00:33:35,940 So can you just speak to the broader perspective of clinical outcome assessment and how maybe 505 00:33:35,940 --> 00:33:39,520 patient reported outcome measures may not be what we want to use? 506 00:33:39,520 --> 00:33:41,440 Yeah, absolutely. 507 00:33:41,440 --> 00:33:49,360 So I think patient reported outcome measures are typically the most often discussed type 508 00:33:49,360 --> 00:33:52,760 of clinical outcome assessment, but there are so many more. 509 00:33:52,760 --> 00:33:57,440 So actually, on FDA's website, there's a really nice set of definitions for each of these 510 00:33:57,440 --> 00:33:59,660 assessments, but I'll give you an overview. 511 00:33:59,660 --> 00:34:05,360 So we think about clinical outcomes assessments, these are ways that we can measure patient 512 00:34:05,360 --> 00:34:06,360 outcomes. 513 00:34:06,360 --> 00:34:10,120 OK, so there's four primary ways we can do it. 514 00:34:10,120 --> 00:34:15,000 And for some quality of life outcomes, we can use all four of these ways. 515 00:34:15,000 --> 00:34:17,800 So let me go through each one of those, and then we'll talk about it more. 516 00:34:17,800 --> 00:34:22,580 So one that we've already discussed is patient reported outcome measures, or PROMs. 517 00:34:22,580 --> 00:34:25,920 Another one is observer reported outcome measures. 518 00:34:25,920 --> 00:34:28,360 So that is where we are OBSRO. 519 00:34:28,360 --> 00:34:35,160 So that is where an observer tells us about a patient's signs. 520 00:34:35,160 --> 00:34:41,260 So what can they observe about how a patient is functioning, for example? 521 00:34:41,260 --> 00:34:45,760 Another type of clinical outcome assessments that I'm sure a lot of the clinicians here 522 00:34:45,760 --> 00:34:47,440 are used to are ClinRose. 523 00:34:47,440 --> 00:34:50,160 So clinician outcomes assessment. 524 00:34:50,160 --> 00:34:54,440 So that's a clinician reported measure, and there are a number of those in clinical care. 525 00:34:54,440 --> 00:34:59,960 So for those in cancer, like the ECOG, for example, is a clinician reported outcome measure. 526 00:34:59,960 --> 00:35:03,480 Finally, there's performance measures, or PRFOs. 527 00:35:03,480 --> 00:35:08,180 So those are opportunities to essentially test patients on their functioning. 528 00:35:08,180 --> 00:35:15,760 So like the six minute walk test, or stand up and go, those are examples of PRFOs. 529 00:35:15,760 --> 00:35:20,760 All four of those types of clinical outcomes assessments can be used to evaluate patient 530 00:35:20,760 --> 00:35:25,760 health status, and they can be used as primary, secondary, exploratory outcome measures in 531 00:35:25,760 --> 00:35:30,360 clinical trials, or even obviously for use in clinical care as well. 532 00:35:30,360 --> 00:35:33,320 We use these all in clinical care all the time. 533 00:35:33,320 --> 00:35:38,820 The choice of which one of those you use depends upon what it is that you're measuring and 534 00:35:38,820 --> 00:35:40,080 why you're measuring it. 535 00:35:40,080 --> 00:35:43,320 How are you going to use those scores? 536 00:35:43,320 --> 00:35:44,320 That's really good. 537 00:35:44,320 --> 00:35:45,320 That's really good. 538 00:35:45,320 --> 00:35:46,320 Okay. 539 00:35:46,320 --> 00:35:47,320 All right. 540 00:35:47,320 --> 00:35:51,320 So we're at the end of the show, and the last question on my mind, which to some extent 541 00:35:51,320 --> 00:35:58,040 you've answered, but so if there's a clinician researcher out there who's now like, whoa, 542 00:35:58,040 --> 00:36:08,920 I had no idea, where should they start first as they investigate using patient reported 543 00:36:08,920 --> 00:36:12,840 outcome measures or other clinical outcome assessments in their projects? 544 00:36:12,840 --> 00:36:14,000 Wow. 545 00:36:14,000 --> 00:36:22,400 So I think the first thing that I would recommend for that clinician to do is get clear on what 546 00:36:22,400 --> 00:36:26,360 are the outcomes that you think would be important to measure and why. 547 00:36:26,360 --> 00:36:34,560 The next thing that I would do is contact somebody who works in health outcomes. 548 00:36:34,560 --> 00:36:36,620 Sometimes that's fellow clinicians. 549 00:36:36,620 --> 00:36:41,840 There's a lot of fantastic clinician researchers out there who are doing work specifically 550 00:36:41,840 --> 00:36:43,980 in clinical outcomes assessments. 551 00:36:43,980 --> 00:36:45,480 Sometimes it's a clinician. 552 00:36:45,480 --> 00:36:49,880 Sometimes it's PhD trained researcher like myself. 553 00:36:49,880 --> 00:36:52,280 At Duke, for example, we have the Center for Health Measurement. 554 00:36:52,280 --> 00:36:57,240 All of us that are in the center are interested in measuring health and trying to improve 555 00:36:57,240 --> 00:36:59,720 the accuracy of how we do it. 556 00:36:59,720 --> 00:37:05,480 So you can look for somebody who has a background in psychometrics, for example, somebody with 557 00:37:05,480 --> 00:37:11,160 health measurement or questionnaire design background. 558 00:37:11,160 --> 00:37:17,480 The ideal situation is that person either has both qualitative experience and psychometric 559 00:37:17,480 --> 00:37:23,520 experience or has one or the other and then knows somebody else who does the other one. 560 00:37:23,520 --> 00:37:27,240 And just have informal conversations with these folks. 561 00:37:27,240 --> 00:37:32,960 When I first came to Duke, one of my collaborators that I'm now working with now, the whole way 562 00:37:32,960 --> 00:37:37,040 we started collaborating with each other was just random conversations. 563 00:37:37,040 --> 00:37:38,040 That's how we started. 564 00:37:38,040 --> 00:37:43,200 We were thinking, oh, there's an important aspect of hearing health care that we're not 565 00:37:43,200 --> 00:37:45,400 tapping into in clinical care. 566 00:37:45,400 --> 00:37:46,400 Let's measure that. 567 00:37:46,400 --> 00:37:47,400 Let's Sherri Smith. 568 00:37:47,400 --> 00:37:52,560 And then it started this whole series of ideas that we've had for how to improve measurements 569 00:37:52,560 --> 00:37:55,000 in hearing health care. 570 00:37:55,000 --> 00:37:56,000 That is so awesome. 571 00:37:56,000 --> 00:37:58,280 Teresa, thank you so much. 572 00:37:58,280 --> 00:37:59,280 That's been gold. 573 00:37:59,280 --> 00:38:02,600 I've learned new things and I feel like you've been teaching me a lot already. 574 00:38:02,600 --> 00:38:04,160 So that was really awesome. 575 00:38:04,160 --> 00:38:05,820 Thank you for being on the show. 576 00:38:05,820 --> 00:38:06,820 It's my pleasure. 577 00:38:06,820 --> 00:38:08,320 Thank you for having me. 578 00:38:08,320 --> 00:38:09,320 All right, everyone. 579 00:38:09,320 --> 00:38:10,320 You've heard Teresa. 580 00:38:10,320 --> 00:38:15,200 There is a lot to think about when it comes to measuring health. 581 00:38:15,200 --> 00:38:19,680 And as you're designing your clinical studies, you want to make sure that what you're measuring 582 00:38:19,680 --> 00:38:24,240 with whatever outcome measure you're using is actually what you want to measure. 583 00:38:24,240 --> 00:38:28,960 And there is no better person to talk to about it than actually a health measurement expert, 584 00:38:28,960 --> 00:38:30,260 the health measurement expert. 585 00:38:30,260 --> 00:38:36,040 So definitely reach out to someone at your institution and look for ways to collaborate 586 00:38:36,040 --> 00:38:39,880 because collaborations with these experts is awesome. 587 00:38:39,880 --> 00:38:40,880 Okay. 588 00:38:40,880 --> 00:38:42,800 It has been a pleasure to talk with you today. 589 00:38:42,800 --> 00:38:45,800 I look forward to talking with you again the next time. 590 00:38:45,800 --> 00:38:46,800 Bye-bye. 591 00:38:46,800 --> 00:38:58,600 Thanks for listening to this episode of the Clinician Researcher Podcast, where academic 592 00:38:58,600 --> 00:39:03,840 clinicians learn the skills to build their own research program, whether or not they 593 00:39:03,840 --> 00:39:05,400 have a mentor. 594 00:39:05,400 --> 00:39:11,520 If you found the information in this episode to be helpful, don't keep it all to yourself. 595 00:39:11,520 --> 00:39:13,240 Someone else needs to hear it. 596 00:39:13,240 --> 00:39:17,300 So take a minute right now and share it. 597 00:39:17,300 --> 00:39:22,760 As you share this episode, you become part of our mission to help launch a new generation 598 00:39:22,760 --> 00:39:35,640 of clinician researchers who make transformative discoveries that change the way we do healthcare.