ROUGH EDITED COPY

UTAH STATE UNIVERSITY – CASEY JUDD

EHDI 2019 – TOPICAL SESSION 4 & 5 – INTERNATIONAL F
MARCH 5, 2018
10:10 A.M. ET

CART CAPTIONING PROVIDED BY:
ALTERNATIVE COMMUNICATION SERVICES, LLC
WWW.CAPTIONFAMILY.COM
800-335-0911
INFO@ACSCAPTIONS.COM

	(Writer standing by.) 
>> SPEAKER:  Hi, everyone.  I'm Brian Shakespeare.  I'm the EHDI coordinator in Idaho and here to talk to you a little bit about data visualization.  We're going to start with how to choose which data you can display, some different tools that you can use to display that data, how to use the data to influence your stakeholders, but the first thing I want you to do is forget about data.  What you want to think about is who's your audience.  How many people here are EHDI coordinators?  Audiologists?  Man, I don't know what else.  What are the rest of you guys?
(Off mic.)
>> SPEAKER:  Oh, parents.  Thank you.  All right, so, depending on who you're trying to influence, you really want to target the data you use and how you display it to be the most impactful with the audience.  So, what motivates your audience?  What do they care about?  You know, like, the way I like to think about it is what if this little girl up here on the left was holding fish instead of ice cream, how happy would she be?  Or what if the man with the fish is in the bottom left picture in a shopping mall?  You know, you need to find out what your audience cares about and display the data that is going to motivate them, and then you want to use that data to tell your story.  You want to use the data that tells the story you want to do to influence your stakeholders.  The way I like to think about it is you want to use, think about your data like it's a children's book, like an illustrated children's book, like you wouldn't have Aladdin in The Jungle Book.  You could, but it would be a way different story, and then you want to make sure that the data you're using clearly tells that story, like that you look at this graph, and I know you guys aren't all audiologists, but this will clearly show you that the child has a high‑frequency hearing loss, so use this graph, and it definitely shows the story you're trying to get across.  Then you want to make it actionable.  How many of you guys are data geeks in here?  
All right, we got a couple.  So, you guys want to talk about data all day, but I'll bet the people you're trying to influence don't, so you want to get the data, you want to show it to them and then move on to how you're trying to influence them and, um, you know, talk about the things that they care about and use the data that will show them why you want to make a change.  How can you move forward with the data that you just described?  You show them the data, and then what are you going to do with that?  How is it going to change your program?  So, you want to keep your designs simple.  I just wanted to give you guys a second to look at this.  I know it's small, but can anybody tell me what that graph says by now?  Basically, the data you display, your audience should be able to discern what that means within 10 seconds of looking at it.  All right, so, our last slide, take 10 seconds, and you can walk somebody through this graph, and it's got great information on it, but you're not going to be able to look at it and know exactly what it means within a 10, 15 second period.  
This one, you can look at, and I know it's small, but, basically, what this graph is saying is after you get to bar three from the bottom, we probably start, we probably need to start using hearing protection, and you can see that very quickly.  So now, we're going to talk about different tools you can use to display the data.  So, a column chart is generally used to show comparisons between groups or different sets of data or to show the same dataset over a period of time.  So, this is some Idaho data from 2015 and our loss to follow‑up rates, and we can look at this data and show right here, we're not doing so well.  So, I can target education to that program, I can make sure that I focus in those areas, and I can show this data to the people in that area and show, like, hey guys, we need to bring this up a little bit.  Um, and then on the right, the orange graphs, this is our loss to follow‑up rates in a region, Idaho's health region three over time, and we can see, right here, that they had a significant drop.  So, what happened there, you know?  I can tell you what actually did happen; their screening machine broke, but, um, but you can look at this data and see, like, it's very consistent, and then there's a drop, and then you need to look at it and find out why that drop happened and what you can do to help.  
So, bar graph is basically the same thing as a column chart, it's just sideways.  What it's useful for is when you have long names for your dataset, so that way, like, the bottom of your chart isn't reading up and moving everything over.  Oh, the other thing that you can use, this graph isn't very descriptive of it, but you can use negative numbers in a bar graph, so you could actually have your data go to the left.  All right, a line graph, Allison likes to refer to these as run charts.  This is the same data we showed in the first slide with the column charts, so, again, see, I like this one better, it shows very clearly that we had this drop in region three, where their machine broke.  A couple things you want to pay attention to is you want your vertical axis to, like, your data needs to be about two‑thirds of your total vertical axis.  So, when I first made this, as you can see right down here, I should have made these numbers bigger for you guys, but that starts at 30 percent.  If we had it at 0 percent and then up to a hundred, so this goes 30, sorry, guys, this goes 30 percent to 90 percent, so if we had it 0 percent to a hundred percent, it's not as impactful when you look at it, and we have a dual access chart.  This lets you have two sets of data on your Y axis and shows the relationship between them.  So, this is our 2015 loss to follow‑up rates as opposed to, we implemented a program where we're trying to get hospitals to schedule diagnostic appointments when their kids fail their hearing screen before they leave the hospital, so the blue is the loss to follow‑up rate in each of Idaho's health regions, and the red is the number of appointments made prior to leaving the hospital.  
So, in regions one and two, our loss to follow‑up rate is a little less than 10 percent, but they made 50 percent of, 50 percent of the children born in that hospital and failed their screening had a scheduled appointment prior to leaving, and our regions one and two, they had zero appointments, and their loss to follow‑up rate is 55 percent or so.  All right, so we can show how effective it was, there's an inverse relationship between the loss to follow‑up rate and who got scheduled for diagnostic follow‑up.  All right, we have a stack bar chart.  What these things are good for is showing comparisons of data from the part to the whole.  So, this chart shows, um, we started tracking why parents weren't showing up for appointments, so we called in, we actually collected data from audiology clinics where the appointment was scheduled, and as you can see, in 2013, we weren't tracking that data, so everybody was unable to contact, basically, but you can see, once we started tracking and we started implementing PDSAs, or study act plans, basically, small tests of change, once we started doing those with our audiology clinics, our unable to contact dropped significantly over time, and this bar chart clearly shows that.  You can also see, like, this orange right here.  I think that's transportation.  Idaho is huge.  There's 80,000 square miles in Idaho, and I think our population is just over 1.5 million, so from the population centers to get to an audiologist is sometimes up to two to three hours, and, so, you can see, as we started tracking, we've found that, and we are working right now to do some tele‑audiology, we're doing outreach to try to get audiologists to move a little bit closer to the population centers.  
All right, a pie chart is kind of the same thing as a bar chart, just makes it look, um, I find that people are more familiar with pie charts, so depending on who you're displaying your data to, this is a good way to do it.  A couple tips with pie charts are make sure that you run your, um, show your categories in descending order, so here's your biggest one, that's our transportation, and there's our unable to contact, and then you can see, as it goes around the pie chart, the percentages get smaller and smaller.  This is one of my favorites.  I didn't know about this a couple years ago, but it's really effective in showing how, um, a certain set of your population goes through your chart.  So, um, I hope you guys can see these numbers, but these, this top is how many kids in Idaho referred or failed their newborn hearing screening, and this is the amount of kids that actually sought diagnostic follow‑up.  In a perfect world, these two lines would be exactly the same length, and then, so, once we dropped down, we had 66 kids that were diagnosed with hearing loss, and 43 of those kids, um, enrolled in early intervention programs, and, again, ideally, these two lines would be exactly the same lengths, but it's a, I really like the way it funnels it through and shows how your population works through your system.  
Then there's infographics.  So, they're really good, because they combine visuals with, um, with words.  The words allow you to have in‑depth meaning to your chart, and the graphics hopefully make it easy to understand.  So, in this particular chart, this one's done by Hearing First, I just grabbed an example, so Hearing First thinks that having your kid, having the dream job, going to college, reading, writing, they think that the base of that is on hearing, and this infographic clearly shows that.  Not technically advising any type of communication option, I just found a good example for this one.  All right, and, so, this is the DASHboard, this is what I said I was going to talk about in my abstract.  So, DASHboards are great, because they provide on‑demand access of your data.  So, this DASHboard is actually live data, this is a screen‑shot, but I'm going to show you the live data in a second, but it's connected right to our EHDI IS, so, up here, this is a date range, so you can type in any date range, and you can look at the numbers.  These are our key performance indicators.  Sorry it's small, but, basically, it's Idaho's birth count, Idaho's screening rate, their refer rate, Idaho's loss to follow‑up rate, and then their EI enrollment rate, early intervention enrollment rate.  
So, you can take a look at it right at the start and see exactly how our program's doing on what, you know, what we have to report to our grantholders.  It gives us an overview of your program in one glance, shows you what you're doing right.  So, we've indicated what we're doing right in green, so these are our health regions that are doing pretty good, and it also shows you that, um, we have some challenges and opportunities to improve our program in these other health regions, and they're also interactive, so your audience can feel involved, they can say, um, well, they did okay in 2015, how did they do in 2016, so it helps them out.  All right, so, I wanted to walk through, back from the beginning.  Know your audience.  So, who's our audience, um, with our EHDI DASHboard?  In general, it's Idaho population.  Anybody, this is a public‑facing website, so that's our general audience, but what I have been using it to is sending this data to hospital administrators and saying don't you want to be green?
(Laughing.)
>> SPEAKER:  Like, oh, let's see if we can get you guys to be green over here.  And it tells our story; it shows us what we've been doing right, what we've been doing wrong, it helps us target areas for improvement.  So, this bottom line here, that's our Idaho data, so overall, as a state, we're doing all right, but, obviously, our different health regions, where there's no audiologist and things like that, we can do some improvement.  Make it actionable.  Again, we can pick specific regions in our state and, um, go talk to them and say what's going on, what can we do to help you improve these things.  They can look at the data, less than 10 seconds, and see, like, oh, we're not up to par here, now what can we do about it?  And keep it simple.  Maybe this one isn't quite so simple, because we have every single one of our metrics here, but what we did that does make it look simple is point out exactly what we want them to look at.  Like, we have a bunch of data, so if their care about our screening, right, if their care about our refer rate, um, that's fine, they can take a look, but it clearly draws your attention to our loss to follow‑up rate, and this is what our programs are judged on in general.  
Um, before we go to questions, I wanted to jump back and actually show you how it works, because it's kind of cool.  So, this is the actual website that anybody can go look at.  We have our dates.  Can we see this?  So, these are our date range right now.  I have it set for six months ago, data to just pop up, because, whoa, sorry, guys.  So, I have it set, not a lot of you raised your hands when you said you were EHDI coordinators, but, um, EHDI data usually takes about six months to be accurate, because after a child's born, refers on their hearing screen, it takes time to get an audiology appointment, and then once they get an appointment, we obviously have to get the data reported back to us.  So, after about six months, the data is actually pretty correct, so that's how I have it set, but, basically, if you look at our data, if we had current data, like last month, every single thing on there would be red, because the kids haven't had the time to go through the process.  Um, but anyway, if you would like to go through, you can check out 2014, 2015, and then look how we've progressed and how we've improved in some places and, you know, some places had some other challenges.  So, you can also hover over the regions, so it comes through.  
It also works the opposite way, and, so, this took about a year, working with my state partners to get it done.  There's a lot of things that go into it; getting IT resources was a particular struggle, getting them to understand why we're doing this was another one.  They're like, what are you talking about?  That took me a couple weeks, because they're IT people, and they speak a completely different language.  Um, but, yeah, so we modeled this after the CDC.  The CDC has their own, um, DASHboard, which is much more extensive than ours, we just wanted to pull out, you know, what our program's judged on and, you know, and just show to our program partners, this is what we're trying to protect.  So, really, that's what I have for you guys.  Do we have any questions?
(Off mic.)
>> SPEAKER:  You know, I honestly don't know.  Like I said, IT people speak a completely different language.  I'm sure they told me at one point, but if you give me your card, I'll get back to you.
(Off mic.)
>> SPEAKER:  We used CDC funding for this.  Basically, correct me if I'm wrong, Eric, but my understanding is pretty much having to do with data improvement, you can use to, um, to improve your program, using data improvement.  Um, I think you can technically use NCHAM, or, sorry, not NCHAM, but HRSA funding for your grant.
(Off mic.)
>> SPEAKER:  Have a good time.  Thanks for coming.
(Laughing.) 
>> SPEAKER:  Anybody else? 
(Off mic.) 
>> SPEAKER:  So, we have an EHDI IS, which sometimes is manually entered, but generally, it, um, gets entered, um, through a data link with the hospitals, and the DASHboard is connected directly to the EHDI IS, so, like, here, let me ‑‑
(Off mic.)
>> SPEAKER:  Yeah.  So, it takes a minute, but if we just, so, this was six months ago, so during 2018, I'm just going to change it to 2017, and then it'll take a minute to go through, because it has to connect to the EHDI IS and then pull up, but you're going to see a change in a moment here.
(Off mic.) 
>> SPEAKER:  You can do it, but that introduces its own challenges.  We decided not to do it that way, because we didn't want hospitals feeling like we were singling them out and their data was open to the public, like, so we kept it within regions, you know, because that way, what happens is programmatic rather than one specific hospital, and I didn't want people feeling like we were calling them out, like I wanted to maintain a good relationship with them.  I mean, honestly, I really would like to do it that way, I just don't think it would be as effective as a program, but, yes, you can totally do that.  We had that talk in our planning committee, why we were doing this, and, you know, the committee thought we should do it this way.
(Off mic.)
>> SPEAKER:  Yeah, it might be a good idea to make it so hospitals had a log‑in that might take them to a different page.  That's a great idea.  I'll probably work on that.  You know, that way, if they really are interested, which I find most of them aren't that interested, but I can make it available, so if they do care, they could check.  We'd probably have to put in some sort of log‑in, so the other hospitals can't, like, snoop on their neighbors kind of idea.  Yes?
(Off mic.) 
>> SPEAKER:  Yes.  I just changed it, it'll pop up in a second here, so, really, this date range, you can move it to look at whatever data you want.  Again, like I said, if you look at last week, everything's going to be re‑read, you know.
(Off mic.)
>> SPEAKER:  Yeah, as long as the data is entered into the EHDI IS, it'll display through this DASHboard.  Yes, ma'am?
(Off mic.) 
>> SPEAKER:  I'm sorry, could you say that one more time so I can repeat it in the mic?
(Off mic.) 
>> SPEAKER:  She says power BI is what it looks like.
(Off mic.)
>> SPEAKER:  Maybe.  Again ‑‑
(Off mic.) 
>> SPEAKER:  You can use a variety of programs that will do that.  I used Excel for that one.  Shoot, what's that program?  I can't remember.  My data people, oh, Tableau, that's what a lot of people use to do these charts, I'm just, um, I learned about Tableau about six months ago, so I'm working with our data people to show me how to use it more effectively, but, yeah, I just used Excel for that.
(Off mic.)
>> SPEAKER:  Tableau.  Yes, sir?
(Off mic.) 
>> SPEAKER:  We still report to them, mostly because we don't think they pay enough attention to us, so we try to be like a squeaky wheel.  They could just go and look, I just don't think they do, so we still send them our reports.
(Off mic.)
>> SPEAKER:  No, it was designed just to have transparency to our public, and also, like I said, I'm trying to create a little competition, so, like, I have e‑mailed all the hospital administrators in each of these regions and just said, hey, you might want to take a look at how your program is doing in comparison to the rest of the state.  So, I worded it nicely, but hopefully, they looked.  Anybody else?  All right, thank you guys so much for coming.
(Applause.) 

This is being provided in a rough-draft format. Communication Access Realtime Translation(CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.