ROUGH EDITED COPY

UTAH STATE UNIVERSITY – CASEY JUDD

EHDI 2019 – TOPICAL SESSION 4 & 5 – INTERNATIONAL F
MARCH 5, 2018
10:10 A.M. ET

CART CAPTIONING PROVIDED BY:
ALTERNATIVE COMMUNICATION SERVICES, LLC
WWW.CAPTIONFAMILY.COM
800-335-0911
INFO@ACSCAPTIONS.COM

	(Writer standing by.) 
>> SPEAKER:  Hi.  I'm Craig mason.  Actually, I was hoping to say I was Xidong on a really bad hair day, but she kind of spoiled that opportunity.  That's okay.  Today, we're talking about the expanded data project that some of you will be aware of, working with EHDI programs around the U.S., covering a variety of learning objectives, which are listed here.  I should apologize, Xidong and the other presenters are not able to be here today, so they've, unfortunately, stuck you with me, but it's an exciting project and a nice opportunity to be able to share kind of what's going on.  I am involved with this project as well, so I'm happy to have this opportunity.  The, um, as a kind of background, we, I think most of us know, or many of us will know that since around 2005, CDC has been collecting aggregate data from states as part of the hearing screening to follow‑up survey, the HFSF that's gathered every year, and it's proven to be a valuable tool, as we've seen, over time, documenting and looking at the improvements that we've seen in screening rates and diagnostic rates and rates at which children are entering early intervention, um, but because it's dealing with aggregated data, if we think back to any epi courses we've taken, we're aware of the limitations of aggregated data for firm conclusions about relationships, avoiding things like the ecological fallacy, but also just some of the subtleties that may be getting into the, um, the data in terms of why are some children not going through the one‑three‑six process as quickly as others, what may be some of the issues impacting age of identification the children may experience.  
So, this led to an interest in restructuring how the EHDI data is being collected kind of in this annual format and transitioning from, is it possible to transition from this aggregated data that's collected through the HSFS to individual level data that, um, would provide kind of more robust ways of looking at, um, at children and how EHDI is helping them.  The, um, it started with a pilot project in 2012 to 2014, which was kind of just a strictly demonstration project to look at would it be feasible to work out the mechanisms of states being able to identify their data, obtain the necessary permissions to share child level data at the national level, and that worked out successfully, which then led to, in 2017, this expanded activities project.  Kind of just as background for that, it was a completely voluntary project that, um, a subset of CDC‑funded EHDI programs took part in, and it involves the states developing these limited datasets that contain a subset of the EHDI data, that the states met on, discussed and agreed upon that they would be able to collect and report.  
We worked on the strategies for, um, addressing data quality, because every state will code things differently, and if you're going to pool data, we need to make sure we're all talking the same language in that sense, so that was a lot of the work and also kind of the reporting process, what are the mechanisms that are needed to, um, satisfy the understandable needs of every state that is sharing that data.  Also then working on developing, what are some of the questions that we want to look at with this data in terms of loss to follow‑up and factors that may impact things like loss to follow‑up, but then also more structurally error issues with our data is kind of one of the things we talked about very early on, is working with multiple states together to standardize your data, you inevitably start to learn things about your own data, which is kind of exciting, that, um, how things are being coded, that you may not have realized and factors that may be at play in your data with things like loss to follow‑up and the timeliness of services.  The states that were involved in this, there were ten states, as you can see, kind of spread around the U.S., I think I've kind of talked about this to some degree, the limited dataset that, um, states were collecting, want to be clear, it still involves, um, the public health information, well, protected health information data, it's not completely de‑identified, which, um, adds, you know, kind of layers of checks and works with the state to receive that permission.  It, as part of the project, we worked on, um, privacy and confidentiality agreements between the CDC, that's receiving this, and the states, but also, and kind of here's some drafts we worked on, as well as a baby use agreement, and I know the font is way too small for you guys to read, but just kind of as an illustration, as to stream line the process, kind of a template was developed, with lawyers and everyone involved, and was then shared with the states and then individually tweaked to address the needs or policies of every individual state, but rather than kind of re‑invent the wheel over and over again, kind of worked off of this template that, um, for sharing the data.  
The data items that were identified by the group, it's kind of this group discussion and planning, kind of based off of the pilot data, the pilot project initially, but then, again, kind of revisiting that as we're going along, and still evolving as the project continues to unfold.  We started with 179 standardized data items, which if you think about it, I mean, some of those are easy to standardize, but others, that's a lot of work to go through.  Factors, variables involving their birth, their admission, demographics, risk factors, screening information, diagnostic information, um, some EI data.  There's been kind of two submissions, kind of the April through July 2018 covered nine states.  The ten that we had talked about, um, but without Minnesota.  At that point, Minnesota was, um, you know, just kind of working on getting everything in place.  That involved 354,000 kids, data on those children, and then in October 2018, there was also nine states, but Oregon, um, was funded for one year as part of this, and Minnesota then got, was all finalized and fully onboard at that point.  What's really kind of exciting, if you like numbers, like I'm the biggest number geek you're ever going to meet, 754,000 births in one dataset that's been standardized across demographically, geographically diverse regions.  I mean, that's California and a half, well, more than California and a half. 
 So, very exciting, and you can see kind of the potential that this is going to offer.  If we start to look at the data, so it's kind of like, as you can see, this is, like, almost hot off the press at this point, the screening, starting with 754,000, so part of what we've seen, um, at this point, 96.4 percent, the babies were screened, 1.4 percent were not screened, and we have the famous unknown of that 2.2 percent, around 16755.  Among those who were not screened, the range from state to state went from kind of the lowest with, one state was, um, 0.75 percent up to a little over 3 percent, with an average of about 1.5 percent not screened.  Those are kind of the averages, the state averages.  Among those screened, about 85.5 percent were screened within two days, that was good, and 98.6 percent of those who were screened, almost all of them, well, 98 percent were screened within one month.  Of those who were screened, we had 3.4 percent were referred on their initial screen, and ranging from 1.8 percent, the lowest referral rate state, to 4.8 percent at the highest referral state rate, with an average of 3.5 percent, again, taking the state averages.  
The, um, of those that, um, that were referred off that screen, 69.9 percent, just shy of 70 percent, um, received a follow‑up screen that was documented in the data, that's kind of one of the key things, ranging from 4.8 percent for one state up to 88.6 percent, with an average of about 63 percent across the states.  The, um, for those that were then, as you see, we're kind of working down the pie, those who were in need of additional diagnostic testing, now, this includes both babies that were referred on their second outpatient screen, um, as well as those who may have, um, were referred on an initial screen but received no outpatient screen, or those who received no inpatient screen, but received an outpatient screen, so Xidong has, like, all these maps, and also those kids who were not screened and went straight to diagnostic evaluation, so you kind of combine all those groups, there was a total of 9620 children identified as being in need of diagnostic testing, not that they received it, but they either were referred somewhere or went straight in, which works out to about 1.3 percent of the total births, or ranging from about six‑tenths of 1 percent up to 2.8 percent of births within a state.  So, kind of based on that data, we're now at the point of starting to, okay, let's look at some of the questions that may be coming out of that, kind of what are these issues with, um, the missing data, are we going to be able to identify patterns or, um, combinations of factors that may explain why some children are, um, missing.  Are there timeliness issues in terms of the whole protocol of receiving the data?  Kind of some of the questions that we're looking at, um, again, looking at kind of what happens to these kids, what are the factors that may explain why they're not.  
The, um, kind of looking at the diagnostic follow‑up data, so, again, we have the 9620 that were in need of diagnostic testing, of those, um, 6096 ultimately received some sort of testing, or about 63 percent.  3936, or 40 percent of the 40.9 percent of the total received out within 60 days.  That's great.  4834 received it within 90 days.  Now, that's just being tested, right?  And as we know, the testing may go on for awhile.  If we look at those who actually received a diagnosis in this dataset, we're at 56.4 percent, or about 5400 of these 9600 kids, so a little over half of them have a diagnosis, and of those, um, we saw 3337 received their diagnosis within 60 days, and 4159 received it within 90 days, and overall, kind of looking at that total universe of these 9600 kids, we ended up with about 40 percent at this point loss to follow‑up for diagnose.  We can kind of see the range across the states, I won't just kind of rattle off all those numbers for you, which that kind of immediately, you start to think, okay, we can see, you know, there's great work with some of these kids, you know, these kids are getting in, and they're getting tested, they're getting diagnosed quickly, but there's some that aren't, so what are some of the, kind of what do we, if we can dig into this a little bit more in terms of the timeliness, um, we saw that, again, of those that were, the total that received the testing, that 6096, about 1117 received, um, were in process.  
Now, it's probably actually a little bit more than that, because two of the states didn't report and didn't have information on the in process, so that's kind of something to think about, that as the group's working on kind of the sequencing of the timing of these reports, okay, there's a question of we're getting the data from these states, and the kids are still in process, so they're not lost, we've had these, if you're on the data committee, you know, we've had these conversations a lot about the, um, the in process babies.  There's about 8 percent had an unknown diagnosis.  Looking at kind of the clinical timeliness, um, in terms of the timeframes, the average age for the first diagnostic test, not the diagnosis, but kind of the start of that formal diagnostic process, the average was 66.5 days, which is, that's good to kind of start, at least it seems to me, good, the number of days between the final screening, so now, if we look instead of between their final screening and diagnosis, about 48.5 days between, again, when you're finishing the screening process to starting the diagnostic process, on average, a little over a month and a half, ranging from 37 days to 68.7 days as an average.  Of those that, about 491 where there was an unknown diagnosis, um, we can see that the dates for their testing, um, about 88.8 percent have their, were tested earlier than February, so, again, we're kind of getting this data late, you know, in October, so you're seeing, okay, October's good, most of these children, you know, they've been in that process since February, 88.1 percent, so kind of the timing of when do we collect this data, that seems to suggest, okay, that's pretty good there.  Only 3.5 percent after July had had their first diagnosis.  
So, again, as we're collecting the data from the states, these are kind of the questions the states were raising, like, you know, as we're still pulling in data, as the kids are still working through the system, what are the implications of when we time these uploads of the 750,000 births?  So, for kind of that diagnosis, it seems that, you know, collecting, by October, collecting the previous year, it's not, I mean, yeah, there's some unknowns, but it doesn't seem so much to be a time, it's not, the October timeframe, kind of the conclusion, seems to be okay for that reporting, that the kids have already, you know, they're already in the system, and they may still be in the process of being diagnosed, but, um, we're not just missing kids that are just now getting the data for diagnosis.  The, um, when we look at the diagnoses, of that total 5400 who received a complete diagnosis, 1128 have permanent hearing loss, and the average age of diagnosis for those with permanent hearing loss is about 97, a little over three months, and again, that's those with permanent hearing loss, which may involve a little bit longer, more elaborate diagnostic testing process.  
If we look at those, um, kind of broken down in different ways, that works out to about 20.8 percent of those that, um, were diagnosed, down to, you know, you can kind of slice and dice that a variety of different ways, or about 11.7 percent of those that were in need of a diagnosis, but are still in the process.  So, again, of those who received a complete diagnosis, we're seeing about 20 percent, which is, I think, consistent, though the range, it did range.  Kind of demographic factors for these kids, um, who received diagnostic evaluation, and this is just kind of, Xidong wanted to make kind of an illustration, um, and you can't see the numbers, but this is just a variation based on mother's education, and the idea was that, and part of the process, so let's look at our individual, you know, we've pulled all the data, but we still want to look at individual states to see are there meaningful differences in what we're seeing across states, and you can kind of see here kind of a general, in terms of mother's education, among those that received a diagnostic evaluation, it's pretty, you know, there's kind of the consistent pattern that we're seeing across the nine different states, with, um, the highest levels of, you know, kind of broken down based on below high school, each of these are the farthest to the left, to high school graduate, associate degree or some college and bachelor's degree or above, kind of more in the higher end.  
Um, so, kind of based on this, you start to think, okay, what are some of the diagnostic related questions that can be addressed with this pool dataset?  And kind of pragmatically, okay, are there, let's look at ways to, um, look at performance, kind of how do we look at kind of performance measures, um, in terms of testing versus diagnosis, right, because we're knowing that you start the testing process before you actually have the formal diagnosis.  Looking at timeliness, um, kind of the timeframes between birth and diagnosis or the screening and diagnosis, and I've kind of touched on the timeliness of the reporting, kind of what's the impact of, it would be nice to get very timely reporting and be able to say, okay, let's turn this data around very quickly, at the same time, what's the cost in terms of providers that are still getting data in the system, so you want to, you know, you want to give providers and reporters enough time to submit their data, but you don't, you know, you don't want to also report the data when it's five years old, so kind of trying to strike that balance of a give and take; the more timely you are, the more immediate you are to the birth, the more in process, the more unknown, the more kids are out there, getting services, and everything's fine, but it just hasn't worked its way through the system.  So, that's kind of an important question I think some of this can help to answer.  
Um, kind of related to that is efficiency and effectiveness.  What's the difference between, I mean, if you, um, you can have a very efficient program by just not screening any babies, you know, you can, oh, I'm running out of time.  So, you can have a very efficient program by just not doing anything, but that's not effective, so kind of how do you balance those.  I'm running out of time, and I told her she can throw things at me, um, at zero, so let me keep going.  EI enrollment, I'm just going to jump through this quickly.  Lots of numbers.  Part of what we've seen is, one, and this is stuff Xidong and Suhana have been, um, and Ashley had been working on, um, you're starting to see, we do a lot of talk about we know NICU babies are kind of slower through the process, because they're in the NICU, in fact, they're able to start now documenting what are the differences, what is the impact that, um, NICU babies have in the system, so that we kind of recognize that and can quantify it and understand it rather than it's just kind of a gut feeling we all have.  
Another factor is kind of getting at this issue of the timeliness and EI, and without kind of jumping to the conclusion, so, part of what, um, Xidong has seen, based on a very elaborate system of classifying the kids kind of through all the different steps and pathways you can get to EI, is that that seems to be, EI seems to be taking more time, no shock there, no Nobel Prize, for that data to get into the EHDI system, and if we report too soon, then we're going to miss EI kids that may be in EI, but the state just hasn't got that data from EI, so what's the sweet spot?  I don't think they know what the sweet spot is yet, and the data committee actually was, if you're, again, on the list serve, that's one of the things we want to kind of talk about, I think, to inform that, because we want the data to be timely, but we don't want to then look like no kids are in EI, when they are, we're just waiting for that information to come.  So, again, kind of some of the valuable information we can get from this.  Um, just acknowledgments, they wanted to make sure to thank all the states involved in this.  It's been a lot of work, bless your hearts, but it's been an exciting project.  
You can see kind of some of the ways the states described it; collaboration, challenging, but it's improving and has a lot of potential and is useful as well.  Thank you.  If you have questions, you can toss them to me.  Feel free to contact Xidong as well.  Thank you.  I think we're, like, out of time.  Are we?  But I can hang around, if people have questions.  Okay, so, I can hang around.  Marcus is here as well.  Oh, look, there's a few of us from CDC you can ask, if you have questions.  Thank you.  I guess people can feel free to leave, and others can, I mean, however you want to do it.  Don't feel bad, if you get up and leave.  It will not hurt my feelings.
(Off mic.)
>> SPEAKER:  No, and I think that's one of the exciting things about pooling this, because we talk about different states have different policies, and until you're able to start to pool the data, I mean, that's kind of one of the, so, we haven't looked at that yet, but that's going to be one of the questions, and particularly, at some point, you know, um, if more states were kind of in this pool, you'd be able to start to look at, okay, what are, can we identify and describe at the state level.  Multi‑level modeling?  I'm a statistician.  Um, you can start to identify program characteristics, maybe even some subtle ranges of state policy and what are the implications those have on this whole process.  So, the fact, we don't have that yet, but good eye, that you can see, this tells us there's the potential to do that and hopefully start to understand what are those policies and procedures that can be implemented, that have the biggest impact.  Did you want to say something?  I saw Marcus kind of fidget.  Did you want to say something?
(Off mic.)
>> SPEAKER:  Yeah, and I think the issue, rural/non‑rural, is a distinction we're interested in in particular.
(Off mic.)
>> SPEAKER:  Um, that is, I mean, I don't know.  I think the next step is to start to analyze, you know, kind of do more with the data and kind of generate and document the value of being able to pool, because it's, for these very reasons that you're able to look at the differences across states, it's better than just having one big state data, I mean, it's bigger than California, because we're able to start to look at some of these differences in other ways, so it's, I think the next immediate step is to start to answer some of the questions that the group has proposed, okay, here's kind of what we're interested in looking at with this, and, um, see where that can take us.  Other questions?
(Off mic.)
>> SPEAKER:  I think the, um, for me, as a stat geek, the notion of being able to, and it'll depend on the specific variable, but being able to start to do what's called a multi‑level HLM sort of model, where we're looking at, it's not quite exactly perfect, because we're not randomly, it's not a true random variable, but anyway, that we're able to start to look at those differences, and we may not know, we may not be able to say this is why states have a lower loss to follow‑up, because of this policy versus that policy, at this point, we may not be able to know that, but we should be able to start to document how big of a role, how big of a difference are we seeing state to state to state, and so we know that there's some differences, and it's this big, that's related to something about the states, and that can then lead us to, maybe, now, okay, what are those differences that may be, and why is it that, okay, these ones are here, and these ones are there, and what are the differences between them that may be playing a role?  So, I think that's kind of one of the exciting things, that we can start to look at policy in a different way, practice and policy in a different way than we have in the past.  Question?
(Off mic.) 
>> SPEAKER:  I was going to say, so you're a number‑cruncher too.
(Off mic.) 
>> SPEAKER:  And this just gives you a way of doing it, that you can't, from individual state discussion to individual state discussion.  Others?  Thank you very much.  Have a good rest of your day.  

This is being provided in a rough-draft format. Communication Access Realtime Translation(CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.