So, welcome to our session on the future of patient safety and quality improvement education and practice, a national collaboration among psychiatry residency training programs. My name is Ray Shaw. I'm one of the co-chairs for this meeting, along with Jackie Hobbs. And we also have Tim Kreider and Michelle Dick with us. And we're looking forward to having a very interactive session with all of you here in the audience and also online to talk about this national collaborate that we've been doing for the past few years. So we do not have any conflicts of interest to declare. The learning objectives, we're going to talk to you about the training and practice gaps in patient safety and quality care. And we're going to describe the program directors in patient safety and quality improvement PDPQ Educators Network and its role in faculty development. And we're going to talk about how you can go about collaborating and connecting with colleagues in a psychiatry open forum to share and solve problems related to QIPS in education and clinical practice. And we're also going to talk about a model using technology to really promote distance learning to help facilitate the development of a standardized curricula. And we'll also share with you some opportunities that's coming up through APER, ACGME, and APA to really drive the future of innovation in QI patient safety training in psychiatry and psychiatric training. So the PDPQ is a partnership between ACGME, APTA, APTA is the Organization of Program Directors Association, and also Project ECHO. So how many of you are familiar with Project ECHO, raise your hand. The new concept to you, okay. So Project ECHO is a project that started out in New Mexico, and the next slide will show you how it works. So it started about 20 years ago by a hepatologist, Dr. Sanjeev Arora. And his inspiration was that he met this woman who was 43, a widow, a mother of two kids. And when she met Dr. Arora, she had stage four liver cancer. And the part that really shocked Dr. Arora, she had been diagnosed with hepatitis C eight years ago, but never got treatment. So Dr. Arora was going, well, what happened? And she said, I can't, as a single mom, I can't afford to drive five hours each way just to get care. Because all the hepatologists in New Mexico at that time were all in Albuquerque area. And so when Dr. Arora looked up into the stats, can you guess what percent of hepatitis C patients were receiving treatment, evidence-based treatment in New Mexico at that time? Any guesses? Spot on, 5%. Only 5%. So 95 out of 100 people were dying unnecessarily because they could not get the care they needed. So what Dr. Arora decided to do, he wanted to really train more primary care providers to do this. But what they found is that when you just send them protocols, guidelines, it didn't really work. So he decided to create this whole thing called Project ECHO. So the concept behind Project ECHO is the following, one, amplification using technology. And this is back before we're all so familiar with using Zoom after COVID. So this is really advanced 20 years ago. And they also want to be shared best practice to reduce disparity. And they want to use a more case-based learning model to help clinicians master complexity. And they also maintain a web-based database to monitor outcomes to make sure. So this is what it looks like. So they believe in what's called a hub and spokes model. So the picture on the left side is where all the experts were located in Albuquerque. And the right side would be two different clinical sites in New Mexico that were actually participating. So concept behind it is actually all the participating groups in the spokes, they would actually present cases. And also, they would do peer teaching. And then the experts at Albuquerque would then actually provide additional consultation and kind of help them take the case to the next level. And our goal is to actually try to model that for you today into how we have done it in psychiatry using a Project ECHO model. And so in the Project ECHO model, the way they thought about it is actually making sure that patients were getting the right care at the right place at the right time. And other providers were able to acquire new knowledge and then treat more patients and build a community of practice. And in a community, there'll be less disparity in treatment, and then they'll be able to give providers more satisfactory treatment experience and then keeping the patients local so they don't have to travel five hours just to see Dr. Arora. And overall, as a system, they'll be able to increase access, improve quality, and then reduce costs. So what we have done here is for the educational route is we have decided, so the patient in this case will be more QI patient safety educational programming and trainees. And then the providers will be program directors or the faculty involved in QI patient safety training. The community will be a much larger program directors association. And the overall system would be like entire GME community. And as you are well aware of thinking about the time when you did training versus what you're doing in clinical practice, a lot of times what your clinical learning environment want you to be able to know and do, it does not actually gel with what's going on with your learning. Because, for example, when you're a trainee, you need to have post-call days. But what does a hospital want? They don't want anyone to have post-call days. They want you to be working constantly. In fact, when Michelle was one of my trainees, we had this experience where Michelle was doing a training in DBT, and the DBT really emphasized continuity. But yet, on the weeks when Michelle was doing night flow, she could not be seeing her patients. And so then we had this innate tension. And this is what the curves are meant to see. So you see on the right-hand side is the GME curve, on the left-hand side is what the clinical learning environment like the hospital or outpatient clinic might dictate. And so our goal is to figure out ways to bridge the two gaps and get the two curves closer together. So ACGME developed a PDPQ, and the way they structured its curriculum was that they had a six-months-long cross-specialty learning. And then after that, each specialty would go into their own specialty-based continuing learning. And what we're trying to model for you today is actually component number two, with some references to component one. And we had a large participation. So psychiatry has been involved for the last three rounds. We ended up having 18 teams, and you'll see it's a diverse group. We have the traditional large universities, like the University of Washington, Michigan, and Florida. We also have some new programs that were brand new. In fact, the Family Health Center of San Diego is just about to recruit their first class of residents. We also have other programs that's more based in the community and a more rural area. So it's a really wide, diverse presentation. All right, I'm going to turn things over to Jackie. Thanks, Ray. Yeah, I'm Jackie Hobbs. I'm the new program director at the University of Washington, formerly at the University of Florida. And I'm going to tell you a little bit about some of the resources, just some of the, you know, kind of the structure of the PDPQ, just to give you an idea and to help you understand, like, what some of the, kind of what's in it for you if you were to participate in something like this. So it really operates on, you know, trying to develop curriculum for quality improvement and patient safety. And it's broken down into four major buckets. The first is stakeholder engagement, and that includes the other bucket of faculty development. So that's a big part of PDPQ. And of course, curriculum development, which is what everybody's usually very interested in if you're involved in training. And then also learner assessment and program evaluation. And I just want to keep stressing that PDPQ has a lot of resources to help in developing curriculum and quality improvement and patient safety. They have tons of worksheets and presentation packets for you, as well as a workbook. And I'm just going to show you some examples of some of those worksheets to help you with your curriculum development. So this is the typical QI curriculum worksheet, and it just is a way to kind of help you organize your thinking about what you currently have going on in your programs or your facilities even, because you may be doing training that's more in the realm of even CMEs. But it really helps you to think through, kind of, what are your challenges and your opportunities? And there are some examples here for you. These worksheets are all in the app. We uploaded those, so you can take a look at those even after the session. But again, just trying to get you to think about, and as Ray was sort of alluding to, trying to think about how can you get this curriculum to your trainees or to whomever you're teaching sooner, earlier, so that they can really start to understand this and be able to function to improve the quality of patient care. The next one is the patient safety curriculum, and that's something to always keep in mind. These are sort of, even though they often go hand-in-hand, patient safety and quality improvement, they are different. And they have certain different aspects, and you want to make sure that you're thinking about both of them. So there's some examples here. Like, for patient safety, we often think of, you know, looking at an event that may have happened, an adverse event, and sort of drilling down on what the root causes of that are. This is the assessment worksheet, and again, it sort of wants you to really work through how are you assessing your learners and at what level, and what are you doing now versus what do you want to do in the future. How can you improve the assessment of your learners? But also thinking about your overall learning program. And then there's a faculty development worksheet, and that really takes you through thinking about who are your different faculty, who are your different teachers. You know, you will have obvious leaders in quality improvement and patient safety, but you'll want to have coaches who might be able to coach your trainees through, you know, a quality improvement project. But you'll also just have general clinical faculty who may not have a lot of expertise, but they can definitely help support. And I think everybody's interested at some point along the way of how can we make care better for our patients. So those are just some examples, again, kind of a little teaser of what's available from PDPQ. And really we want to think about, before we sort of dive into some cases, you know, why are we doing this? That's the big thing, right. I think a lot of times we're going through, we kind of go through the motions of knowing what we're doing, how we're doing it, all the different steps. But the core thing is why are we doing this. And really it's all about the quadruple aim, right. We want to make patient care better, we want to make it safer, we want the experience for patients to be better, and we want our experience to be better. If we have good quality, safe care, it's good for our well-being as providers. All right, and we're going to start with our first case to get us going this morning. Thank you. Hi, my name is Tim Kreider. I'm from New York, Zucker Hillside Hospital, Zucker School of Medicine at Northwell and Hofstra. And I came to this group by being a participant in PDPQ, that structured course sponsored by ACGME and others. And it's a pretty intense, weekly, over six months course where we, you know, learned how to use worksheets like the ones Jackie just showed you and tried to really think broadly about how to implement a curriculum in quality improvement and patient safety. And since I did that course, then I've been involved in this group that we're calling the Open Forum, where we meet monthly, virtually, and we continue to support each other. And so what I'm going to show you now is an example of how we do that, you know, inspired by that Project ECHO in terms of the model and recognizing that, you know, that there's group sort of support and consultation, and then there are also some programs that are further along and really serve as experts or hub in that hub-spoke analogy. So what I'm going to demonstrate is how we might present a case and support each other in one of these monthly meetings. So I will describe a problem at my hospital residency that I'm trying to solve. And it's a curriculum problem. And I'm going to try to bring a consultation question to the group, just like you would for a clinical consultation. And the group will try to problem solve with me. And so, the general format is that I, as the presenter, will speak for about 10 minutes about my problem and give the context. And then the group will ask clarifying questions to make sure they understand it. And if we have both experienced participants and less experienced participants, we structure it where the more junior folks will ask clarifying first, and then the more senior folks will weigh in maybe with sort of higher level questions. And then we do a problem-solving session where I, as a presenter, sort of try to step back and let the group talk about it as if it's their own problem, and then I sort of come in and hear the solutions and respond to the ideas after letting the group discuss. So let me jump into the case, and with the caveat that this case was brought by a colleague of mine in the same health system, Dr. Peter Steen, Sister Hospital and Residency Program within Northwell Health and Zucker School of Medicine. And so I start the case by giving a little bit of background because, as you heard, we've got programs of all sorts of sizes and ages in this group. So at Staten Island, the residency program is relatively young and relatively small. The medical school is only a couple years older than the program. And as of the time coming to join this group, the QI and patient safety curriculum was quite unstructured and really had a lot of development to do. That's why Peter came representing the program to first do the structured PDPQ course and then join the open forum. Now, in addition to the training program being young, the hospital service was not. And in fact, there were a lot of attendings who'd been around for quite some time. So there's a bit of a culture change that's been happening here. And part of that is a lack of enthusiasm for patient safety and quality improvement among some of the clinical staff. Of course, you know, the training program knows that we need to do this to train our residents in this. But even some of the leadership in the clinical services is not fully bought into supporting resident involvement in QI or even sort of formal QI as we understand it. And so in this context, we're trying to get residents to do projects and support them. And here's an example of a project that the residents proposed that ran into some trouble. So the residents identified communication between inpatient and outpatient providers. Here's some facts about our psychiatric unit. Two attendings, two NPs, residents, med students, social workers, nursing, as you would expect in an academic service, busy. And then the outpatient providers just did not consistently get handoff, able to give collateral. Sometimes they didn't even know the patients were admitted until the discharge appointment came up. And the residents, you know, were stuck in the middle of this and identified this as a great problem. And we loved it because it's right at that sweet spot where it's a real patient care impacting issue and it's also a quality of life issue for the residents. And so they were very enthusiastic. And that was one of the real strengths of this project was that resident enthusiasm because that's not always been the case for us with residents being told to do QI. And it's a great topic for going over the process, you know, using the Institute for Healthcare Improvement model of clarifying the, you know, an AIM statement, determining your measures, doing a PDSA cycle. And so it was good for that. And in fact, it was a great one for identifying stakeholders and getting the residents to go around getting those different perspectives. You know, in my experience, residents jump straight to what the solution is and we've got to encourage them to slow down and really understand the baseline and different perspectives on the problem. But then there were a lot of challenges for this project that are representative of what we're dealing with here. The unit leadership, while like they recognize this is an important issue, they were not terribly interested in changes to their workflow brought by resident ideas. They've got a system in place and even though that's not working terribly well, they're just sort of resistant to change. And then furthermore, some of the solutions involved use of email and other sort of communication methods that were, frankly, a little too advanced for some of the senior faculty. It just sort of didn't fit. So, you know, this was challenging for me supporting the residents. And you know, of course, where I want to be is a place where the attendings and the leadership not just care about QIPS, but care about including residents in that and really welcome their involvement. Residents can bring a lot of energy and enthusiasm. We want QIA to be part of the culture rather than something that, you know, education is insisting on that doesn't fit. And we need attendings not just to be open to it, but really to help share some of the responsibility for training and mentoring the residents in QI and PS. So I bring all this to the group with these questions in mind. How do we shift the culture of a department to embrace QI and patient safety as a process that's valuable to their everyday work? How can we make it feel like it's something they need and want? And what do you do when you run into resistance to these kinds of initiatives? So rather than break out, I think we'll all be together. And maybe what we'll start with, are there any clarifying questions that people have? And after the clarifying questions, then we can start doing brainstorming solutions. Yes, I have a question. What is the attitude of the other staff on the unit, like the nurse practitioners, the nurses, and the social workers? What's their attitude about the project? About the resident project, it was, you know, there were some people who were really excited about it, and others that, frankly, rolled their eyes. And unfortunately, there was more eye-rolling at, like, the physician level. And so it was really hard to sort of get the whole team on board as a result of that. For this specific project, social workers were all about it, because their life was more difficult. But it would work. And can you go to the microphone, so the folks who are online can hear you? Was a QI a mandatory for scholarly activities for anyone in terms of the residents or for the faculty? That's a great question. Just because it can change motivation, that's all. Absolutely. Yeah, thank you. You know, it is required for the residents, you know, they've got a graduation requirement. It's not required for the faculty in the same way, which maybe is part of the problem. So did the attending see this as a problem, that the outpatient provider is not being notified until discharge date? Do they think this is okay, this is a good way to operate? Because you would think they'd want to contact the outpatient provider on day one to get some clinical information about the patient when the patient was admitted, instead of waiting until the patient is discharged. I mean, this happens to me all the time, is that the outpatient provider, I get a call, we need an appointment for this patient who's been in the hospital 10 days. Yeah, I agree with you as an outpatient provider myself, I would just love to get the call. And you know, of course, the inpatient providers are very busy, I try to have empathy for them. And so I guess, you know, part of the problem is how do we get people on board with this particular project, sure, but more generally, about quality improvement as something that's needed. Are the outpatient providers part of the same medical system, like the same faculty? In this case, it's a mix. Sometimes they're not, but often enough, they are, you know, faculty or residents within system. I'm just wondering, you know, if I imagine the outpatient faculty are going to be much more in favor of this, and I wonder if that holds any sway for the inpatient faculty that like my co-worker once would love this. Yeah, thank you, if we can try to connect a sense of fellow feeling and community as working together. Yeah, within the same system, that would be ideal, because the inpatient team would also want to be able to get a hold of the outpatient. There are frustration for many inpatient doctors, they call out outpatient folks, they say, I'm in session all the way until seven. So how can they go about them? How do you go about making major decisions about medication changes, for example? So it's a frustration both ways. So if we can incentivize both in that regard. I just want to ask, what, what, from what branch of leadership is this initiated? Or has there been like guidance set forth? Like, if there is deficiency, which is, I mean, quite understandable, like, is there any costs associated with it that can lead more of to a team that will allow the outpatient and the inpatient providers to communicate or to see or to incentivize a need as a measurable outcome that we could reduce other than, of course, improving quality care? I think that's a great point. Can we tie the motivation of doing this not just to the educational mandate, but to outcomes that different people really care about, whether they're financial, or other things that hit the bottom line? It sounded like that's actually approaching problem solving. So why don't we get to problem solving? Yeah, absolutely. So yeah, so open the floor to, you know, specific ideas. And let me encourage, encourage you, if you can, to think not just about this given problem with the, you know, the residents we're trying to deal with, with communication, you know, because I bet you think of a lot of solutions about solving that problem. But rather, the problem is, like, the culture shift, right? Like, that we've got people who've been there a while and don't see it as part of their job. How do we get them on board with the program? So maybe we could get folks to get closer together and start having a discussion. And we'll come join you, too. I'm thinking asynchronous. I'm thinking eight, because I'm seeing the barriers from both sides, typically, because I have been an inpatient attending. You have multiple admissions every day, as well as multiple discharges. And because of the structure of payers, often the patient is only in the unit for three days. So by the time you actually get to talk to the outpatient person, it's already post-discharge. And it's not because that's how it was intended. The outpatient is in session until late, and the inpatient is busy sort of cranking the machine up. So I'm thinking, in order to, because one of the barriers, I think, is just time and more work. And one way to decrease that, it's perhaps asynchronous use of communication, right? Like, for, like, we do that for our patients on messaging in MyChart. But it doesn't need to be MyChart. I'm thinking any type of asynchronous, where you can send a, how do you call it, private and protected, HIPAA-protected message. As soon as the patient gets admitted, a little brief summary to the person who treats that patient outside, and ask for any important information that they want us to know, in terms of, you know, what could have led to the patient coming in. And so this way, there's no burden of having to have a real-time conversation. Yes, it's easier, but the way the system is, that's not friendly. Other thoughts? I think one thing that I've learned is that when you start having a lot of ideas about how things can be done differently, it can be very overwhelming to people who are not necessarily excited about changing. So if you, I don't, and I don't know if the residents did that in this case, but to come up with the system and sort of mock it up for how it would look, and then present that to the attending. So instead of it being like, oh, we could do this, or this, or this, or this, which feels like too much, then you have something to show, and it can be a lot easier to see how it would save time and effort, instead of just be like another burden. This is an age-old problem when you have people who have been in the system 20 years and this is the way we have been doing things for 20 years and we don't want to change. This happens all the time. But I think you have to approach it from the standpoint of the lack of communication between outpatient and inpatient is causing problems. It may result in more readmissions, putting people on the wrong medication, patients staying in the hospital longer. And so it's got to be something that you have to address as this is a problem that's causing bad patient outcomes here and all the science out there is that the less communication you have with outpatient and inpatient, the poorer the outcome. So we need to address this problem and it's not just, you know, your system is working perfectly and doesn't need to change. That's never the case. So you're really talking about giving the inpatient docs, the folks who are more resistant towards change, a value proposition for why they should change and it can come in different ways. So earlier you were alluding to are there incentives that can come out this way? For example, for many hospitals, they want to reduce the rate of readmission. So in that case, that would be a proposition saying, hey, doctor so and so, if this patient comes back again, what's the reason for doing that? And after a while, they'll say, well, it's probably going to waste my time to call the outpatient doc so I don't have to fill out another hundred forms to justify why this patient is readmitted or the community might have complaints. So again, the goal behind this whole case, we actually prepared three cases for you. So the first one, we're trying to just give you a sense of this is how it works. So like technically, so like Tim was a presenter. So he's one of the spokes. And then Michelle will be another spokes participant. And then Jackie as an expert would then actually come in with like the last clarifying question. And this part is where the peers would actually have a conversation. And then the experts would then provide the last part. And this group is so good. You're already jumping into it. So we didn't have to demonstrate a first case. So next one, we're actually going to give you a more complex case. And what we want all of you to start going, okay, this is how we go. They actually did. So then we actually gave them a lot of, so with Peter's case, we actually talked a lot about incentives. And also they actually brought in a nurse practitioner. So they were able to show the nurse practitioner and the attendings will do things differently. And then so also they got a social worker to be more involved to kind of help facilitate. So Peter was very happy. All right, move on to case two. All right. So this case to kind of mix thing up a bit is more of a clinical situation. It has a lot to do with equity of care, especially for patients who receive care in a language other than English. And this case is coming from our Child and Adolescent Latino Mental Health Assessment and Treatment Clinic that we have at Seattle Children's. And this was formed by two prior graduates of our training programs, Dr. Laura Black and Dr. Cindy Trevino. And so they noticed that there was a huge gap and need for culturally and linguistically adapted care for our Spanish-speaking families, not just in Seattle Children's, but in the community around us as a whole. So over time, what we've been able to offer has developed. We offer diagnostic evaluations, medication management, group interventions for depression, anxiety, and disruptive behaviors, and short-term individual therapy. Every single clinician who has direct patient contact is certified to provide care in Spanish. Seattle Children's requires us to pass that test in order to assure appropriate level of proficiency. And a lot of our support staff are also certified as well. At this point, the majority of our appointments are telemedicine, and this is because Spanish-speaking families exist all across Washington State. I'm not sure how familiar you are with the geography of our state, but there's that huge mountain range that separates us from eastern Washington, where a lot of families live. And it's hard for them to cross and get over to see us, especially in the wintertime. So we're really thinking that this model or method of care is going to continue, at least to some extent, to make sure that we can reach those families. So at this point at Seattle Children's, there's been a big push to – sorry about that. So at this point in Seattle Children's, we use Epic for our EMR and have been really trying to push using MyChart or our patient portal for every single family. So once they start getting set up for an appointment, someone is trying to help them with that. And that was to improve communication with the clinicians, to help them log into those virtual appointments, and also give them access to records. The nice thing is that MyChart actually can be changed into another language, but that is only after you go through the entire setup in English. So it requires a lot of time and effort to guide our families through how to get to that point, and that was a huge barrier for us, actually. Another problem is that the language certification is just for verbal communication only. So I am not certified, and they don't have a process to certify me to write or technically read in Spanish, even though all of the clinicians can. And so what that means is if a family wants to send us a MyChart message, or if I want to send something to them, I have to call them and tell them the message, which means that we spend a lot more time and effort communicating with families than our colleagues who aren't seeing Spanish-speaking families. And you know, translation services or language services gave us a number of different reasons why this was, one of which is safety, getting your translation done faster. We can't do that as quickly as you can call them, but there are plenty of times where families want some type of instruction, contact information provided in a place they can read it again, and mail might take too long. So in addition to starting to use MyChart or pushing that for all of the patients at Seattle Children's, we've also been experiencing some changes in our EMR, and we're at a point where measures are automatically included in the chart during an appointment. There's a tab that we can just start filling out, so the PHQ-9 is in the chart, and then it can be dragged into or brought into any note, and you have the values from before to really track. And then if a patient has MyChart available, I might send them a message, and the measure is included in that message, so then they can fill it out, and then that's also saved in the chart. But for those of us who are seeing families who do not speak English, none of those things are readily available in EPIC in Spanish. It's just English at this point. So what that means is in order to do measurement-based care with these families, we're using appointment time to collect measures, because otherwise there's no real way to ensure that those are going to be done. And we'll share the screen during the virtual visit, read the measures aloud, the questions, and then write them down. And then maybe later after the appointment, scan those into the EMR, or transfer the answers from the Spanish version to the English version in the chart. So it's a long process, and these are all workarounds, which isn't effective, and it just leads to more potential places for error, losing measures, forgetting to put them in. So we really want a more long-term solution. So in our clinic, in the Kalma Clinic, what we have started doing is deciding that, gee, maybe MyChart is not something that we need to push for all our families, because the benefits that we're supposed to be getting from that, like ease of communication and access to records, they can't get that through this. So we've started to decide that maybe we're going to back away from pushing that for all of the families that we see. At this point, most of our educational materials and questionnaires that we would want to use are translated into Spanish and other language as well, which is good. So when I'm in an in-person visit, I can readily hand these things to families and ask them to come back and bring me the completed one at the next appointment. But it's at the virtual appointments where this is really difficult. Also, some of my colleagues who run the groups have started using RedCap to send messages or measures to the group participants to track their progress as they go through the groups. But a lot of families have kind of struggled with getting those emails, getting into those emails, and understanding the meaning of that or why we're doing it. So just to kind of put this all together and think of our strengths, one strength that I'm really appreciative of is we have a lot of support from our colleagues and department leadership. They're passionate about this. They have a desire to help with diversity, equity, and inclusion work. The problem is when we go and work with language services, there's some, it seems, conflict or difference in what our needs are. We're also lucky that we have all these questionnaires and all of these educational materials translated. It's just a matter of we would like them available electronically in the EMR. And then we also have support staff who are incredibly passionate about doing this work. They have spent hours sometimes trying to talk a family through MyChart in Spanish and like click this thing that says this in English. So it's a process and they're passionate and they care about these things. So we're really lucky to have these colleagues. The challenges with language services, we already talked about what had happened with MyChart sending messages, things like that. And I can appreciate the situation that they're in. And it costs, I looked this up recently, it can cost anywhere from $0.10 to $0.40 for every word when you send something out to translation. So over time, that's really going to add up. So I get that. I can appreciate that. Another problem here is the current system is significantly increasing the workload for bilingual clinicians just to have the measures done. So there's a huge difference in how many steps there are for us compared to other families and physicians who do only see patients who speak English for the most part. We're also losing critical time and appointments reading those measures aloud. And most of our families access these visits on a telephone or on their smartphone via Zoom. So what that means is when I share the screen for the PDF, they're really not able to read it. So what ends up happening is I'm reading it aloud and they're trying to remember what I said and then remember the options, which makes me wonder, like, is this even valid? Are we getting what we need? And then finally, another big concern is a lot of parents don't feel comfortable accessing email. They might eventually get a MyChart account, but they don't remember and they need their kids' help to get in, either because of the language barrier or because they don't understand the technology, which our patients have to do that a lot for their families already. So adding to that burden and responsibility to them is not something that we want to do. So what we're hoping for when we try to find a solution to this problem is to improve patient access to measures that's on par with the access that English-speaking families have and decrease the number of steps and time that's required of us bilingual physicians and clinicians to use measure-based care. So it increases the likelihood that we will and increases the chance that those things are going to end up in the chart and be able to be tracked over time. And then also, finally, to create a more collaborative relationship with language services and other non-clinical departments like the EMR so that we can identify solutions that have those lower costs while also addressing those two goals above. And so the questions that we have for you that we want to start talking about is how have you built relationships with non-clinical departments and partners such as the EMR system, language services, et cetera, and how have you approached collaboration when the goals or the needs of these partners may directly conflict with the needs of what may be a very small subset of clinicians and patients? Compared to the total volume of patients seen at Seattle Children's in psychiatry, we're only a fraction of that. So that begs the question, when the conflict is just a small number of patients, how have you been able to push that forward? And then also, what's worked for you to increase participation from clinicians and patients in projects like measurement-based care? All right, at this time, go ahead. Okay, so now this is where you get to ask clarifying questions of Michelle. Go ahead. Have you had anyone like submitted a request to EPIC to have the measures translated? Not yet. Because EPIC does have that, you can, EPIC has that for anything for other languages where you can have the same questionnaire and the responses will go into the same flowsheet. Okay, thank you. I mean, it has to be done, but. Yeah. No, we didn't know that. Okay, thank you. So was there not a process to get certified to be able to respond in written Spanish? So I looked it up and there is. The same language services that certify us for verbal communication also have a written test that we can take, but that is for general, like conversational Spanish. So there might be an issue or a gap in terms of being able to certify the medical Spanish. Because physicians take, the test is more of a medical Spanish test. For some of the psychologists and therapists, it's a little different. So I think there might be a gap in what they want tested if we take that written test for just general conversational. Other clarifying questions? Would MAs be available to review these questions with them prior to the visits? So when we're doing the telemedicine visits, we generally do not have MA support. I go in person once a week and I do get MA support in those situations, but as far as I've been told, having the MAs go through those questionnaires with them isn't part of the job description. But also, when I do see them in person, I'm asking them to complete them outside of the appointment. So in that situation, it really isn't an issue. The problem is when we have these virtual visits, mostly. Have you asked the institution for additional time due to the complexity of this, or have you billed for complexity? That has been brought up, and that has been a concern about the increased amount of time and work that we do, and are we being given enough time to do that appropriately? So that has been a question that has been raised, but we haven't really got an answer. I'm sorry, I forget if you mentioned this, but are there also therapists there? Yes. Because at my community mental health center, sometimes I have the therapists go through the measures with the patients during their sessions, because they spend more time with the patients. So when the process that we have is I might be completing an evaluation, and our therapist might also be doing the same thing. So we don't typically see the same patients, unless after I've completed the evaluation, I might refer them. Because Seattle Children's also used a step care model, so unless there is a significant clinical reason to not start them in group treatment. Everybody starts group treatment and then if they need individual therapy after that and the when they've looked at the data it's a small percentage of kids that actually need individual therapy after the groups so that's where the individual therapy comes in or if the patient's suicidal or having self injurious behavior in those situations we want them to have a therapist and so we might put them in therapy before putting them in a group. Do you have any peers in your clinic or family peers? We just hired two community health workers. As far as what they're going to be doing I think they're just starting to train and my understanding is that their role is going to be more helping the families on the wait list so not after they've come into contact with us to establish care. I wonder if even even then maybe one of the roles that they could have would be to just to help the family set up MyChart and sort of explain how the process goes mm-hmm especially if you're also like using zoom and so then there are some other computer things to work out. That's a good idea. I had a similar issue with a different EMR when I worked with the technical department it was it was an issue with the password how you reset the password required manual so is it possible that you could record maybe it's just verbal right like a physician could record themselves going step-by-step a presentation of how to use it in you know and then that could be accessible to them that could be sent out so therefore you don't have to take the extra time and do go through it and have like a frequently asked question on their website that could be a potential that's what I would I basically did with the paper format you're referring to like and kind of like a job aid or instructions on how to yeah I sign up for the initial transition from we have to set up to epic in English so you guide that step-by-step with infographics and then have that as a handout or something that you can have a video where they could replay it use it as many times as they need to so it sounds like we're doing some problem-solving so why don't we go back to a previous line Michelle so folks can see what helps you're looking for so any suggestion for Michelle with the the EMR thing ideally you have connections with I mean like for epic they have physician builders and you then you have like a clinical connection who can help let you know what I mean because epic has so many things they offer there's no reason that like the average user would know that they exist and so I think that's something that is really helpful to have like a direct contact I agree other thoughts feel free to come on and comment on any of the questions that's posed up there usually in in technical situations like this I used to lead a residency group in multiple specialties and the issue is the technological like bandwidth of accepting new tech or reaching out to your epic kind of support staff so I think it'd be good to involve them in any of the key why meetings or and it is a simple kind of ask a to be part of the physician group and have representation from your clinic in the physician groups that that you could readily build this within 48 hours even if you want personalization of that and due to the cost of this I would you know I would you know what isn't measured as a managed so therefore I would extrapolate this would be what would cost us if we don't continue to improve it in a readily manner and this could add to the physician burnout having to deal with this excessive amount of time on top of your documentation I want to comment on question number two you can fall into the trap of spending a significant amount of time efforts and money off the partners just to circumvent like do it a much longer path to treat this small so I think it's actually a very easy sell for the partners is to say it this is consuming so much time effort and money if we find a shorter path it does benefit them even though it's a small subset of the population so you probably if you show in FTS or the the amount of people that are being using the things that are being done and say look like you we I could be seeing this many more patients and I'm not because I'm having to do this thing on the side usually money talks that's my yes and for the last one the best thing is to make it automatic so if it's an epic translated document that automatically gets sent to the patient the week before or three days before and so you can access when they come in then it totally bypasses the need for you to be certified in in the written language and also would increase the chance that they will do it because they would have a significant amount of time to actually access it at home so if it's anything that's automatic in the places that I have worked that the patients can do without you even telling them really increases the number of people who do it and I do want to advocate for getting certified even in colloquial Spanish I'll tell you why the emphasis in clinical care is to speak to patients in a language that accessible to them so like from a standpoint of patient care what's what's the matter with you being able to use big words in medicine when that might not be actually the language that you would use you know like so I would say get certified in colloquial Spanish so you we know for a fact that somebody can communicate with patients yeah I think part of the reason is maybe a liability issue or something like that at least that's what we've been told in terms of writing outside of the policy but I agree I think to speak to the last point as well I think that clinicians are probably more likely to do the measurement based care if they see the benefits from it so like you said tracking the phq9 or whatever over time and again from a technological standpoint like you can work with people at EPIC to build these things out so that they don't have to like remember the dot phrase to do to pull in the last scores or whatever they don't have to go to the flow sheet they can build things that will dynamically show progression over time and again I think that's makes people more likely to put the data in if they feel like they're getting a benefit from it the bigger issue here is been a lot of articles written about these EMRs and all these systems have been put in place that require a lot of documentation a lot of time and the question a lot of physicians are getting burnt out with this I saw an article the other day 57% of physicians say they would go into medicine again if they had to do it again that's a huge number of people that are saying I wouldn't do this again because it's not what I signed up for when I decided to go to medical school and the EMR people have to be working for you you don't work for them they work for you and they should be making it as easy as possible for you to do your job and if you can't do measurement based care without spending twice as much time seeing the patients it's not worth doing it yeah I think part of the solution isn't convincing them that you know it's worth making these changes for our patients those a lot of that is is accessible and ready to go and epic in English this might be more of a clarification question but does how frequently does this need to be done the PHQ measure of this and every visit it's usually they can only get compensated for four times during the year I do it maybe every couple of months approximately it depends on the patient but I'm not doing it at every visit for sure I am I also use a lot of measures and screening forms to help with diagnoses you so you know I might give a parent a scared to complete I might give the kid one or Vanderbilt said something I use all the time because that's a primary translate them there are there are translations available but you do that during the session to the scared you know I if I'm seeing them in person I try to send it with them home I've tried to send like my chart blank messages and just attach it so you end up seeing like a blank encounter and an attachment to see if maybe that will work it doesn't usually yeah I'm gonna add on to previous loop point before she goes on it's I agree with the colloquial kind of certification I think you know just as you said it's like money talks but legal walks and so there's more exactly my point thank you great assist and I did not plan that question I was gonna I was gonna tell you that have you tried to measure patients that fell through the crack because of lack of communications and how that has impacted the system because it doesn't matter the size of the marginalized population because that's what we're talking about it is a small number but it's also marginalized if they are creating enough of a pool of resources and potentially putting the institution in Lego in legal jeopardy so has anybody measured that like like how many hospitalizations or how many you know like just looking ER visits because because they didn't have access to a translation mm-hmm not particularly but bringing that up there's been issues with safety plans and then having like being discharged without a safety plan in Spanish and things like that and I've tested out the mechanism that we were supposed to have I either I'm not finding the right job aid but it did not work we did speak to the the lawyers who because we work for University of Washington and but the clinical site is Seattle Children's so you know we have spoken with Seattle Children's legal department and then we also spoke to UW and their answer was if it's a safety issue do what you need to because the patient safety comes first and that's really what you should be concerned about so in those like acute situations they told us just go ahead and write whatever you need to write thank you all for being so enthusiastic so hopefully Michelle has some good stuff to take home and you'll see where we're very intentional and how we structure the cases for you so first case is the classic complaint people have inpatient complain about outpatient outpatient complain about inpatient so it's all about stakeholder engagement this case now we added more complexity to for you so you'll see we have the classic big one everybody struggles about you talk about burnout EHR burning people out you talk about working with special population our whole emphasis on diversity equity inclusion and also measurement based care so this one's also again about stakeholder engagement about faculty development because you have to train your workforce on learning how to do that so you see the buckets that Jackie was talking about earlier we'll get tie it all together so now we're getting to a key point Michelle if you can advance the slides a couple here's an interesting part so we have actually prepared a third case for you in case none of you have any urgent cases urgent problems like Michelle's that you want us to solve does anyone have any this group is so advanced I said we could if you have a case you really like to speak up we'll be glad to help you otherwise Jackie also hang out these handouts we do think about and you can actually reach out to us and we'll be glad to invite you to join our open forum where we can help you do some more problem-solving so anyone have any case they want to bring up otherwise we're gonna have things have Jackie finish the last case okay all right if anything comes up just let us know so what I'm gonna be talking about today is I think a case that's probably a little more well it can be relevant to a teaching practice but also to just general practice so what I'll be describing is my experience obviously I'm in academics so you know I'm going to describe it from that point of view but I want you to think about like if you were out outside of academics as well so just some background we have an outpatient practice with obviously faculty and residents and fellows and as of July 1st 2018 Florida passed a bill that's what that went into effect requiring that all physicians check the prescription drug monitoring program prior to prescribing a controlled substance to any individuals age 16 or older and there was also a required CME two hours and you know it I think I think the the system sort of worked very hard to to sort of make this as easy as possible and so our prescriptions and our electronic medical records you could click easily click a little button that said that you had checked the PDMP you could also link right there in the visit to the actual PDMP for Florida and you could click another button that said that you had you you had reviewed it so that was your way of documenting those things and also you could see who the last provider was and what date they it had been checked before but what I experienced recently was that you know a resident asked me to sign their controlled substance prescription they basically pended the prescription for me and I had to sign it because the patient had either Medicare or Medicaid and that's required to have an attending signature and I always read or at least I think I always recheck the the PDMP even though the resident may have checked it because I want to make sure you know it's part of my job to as oversight of the resident but also just to make sure nothing slips through the cracks. So this time I found that the resident had not checked the PDMP so the last time it was checked was two months ago by another provider and that does not meet what is required you have to check at least the same day of the visit. So it made me wonder like how often does this happen and I also asked myself have I ever forgotten to click that I checked the PDMP. So what's working, what's not? So our strengths are that, you know, our EMR makes it easy for us to do this. We sort of have two reminders. We have two places we can click. And again, in academics, fortunately, we have kind of double checks, right? We have a resident and an attending who can do this. Some of our challenges, you know, I think it can be easy to look at and review the PDMP, but then sort of forget to click that box that says that we actually reviewed it. And I think this could especially happen if we get very busy and distracted. Also I think it's been, you know, so it's obviously been about five years since the law went into effect. So as one of my colleagues always said, there's a voltage drop off. And people aren't thinking about it as much, don't think about, you know, how important it is or what the consequences may be. Also, you know, I started thinking about, well, the consequences of this, of not doing this because we aren't following statute, for instance, they may not come until much later down the road. And if something, you know, heaven forbid that there should be some unrelated board complaint or something come, and then they start looking at your practice, and then they come across this. So where are we and how would we think about this? And really this is kind of a proposal, a proposed QI process improvement. You know, some of the things I've sort of thought about, you know, can we run EMR reports to check compliance? How easy is that to do? You know, could we have some best practices, some alerts, some hard stops, all these sorts of things sort of came into my mind. You know, can we reeducate about the bill and sort of, you know, do some retraining sorts of things? You know, and what I'd really hope for is that we would really see this, first and foremost, I mean, why do we have the statute to begin with? I mean, it is a patient safety issue. You know, that's why we as physicians should be checking the PDMP. You know, and we also need to recognize this as a practice, both individual and group risk management concern. And we want to think about how to reduce liability. So basically, I'm coming to you to hear if any of you have had similar experiences, if you have, you know, come across anything like this, you know, or, you know, have you learned anything that may help with this and just any suggestions you may have for further improvement. So I'll open it up. This is actually an ongoing QI project that we're working on in my hospital. So there is a very simple solution to this other hospital systems have if you have EPIC. There is an integration you could do with the PDMP. So it's within the EPIC, but the problem is the hospital has to pay for it. It's about $1,000 for 500, like, membership pass. That's the simplest solution because it forces you as a four-stop before you order anything controlled, that would be one. The other way is that I've created a kind of a dot phrase that also has when was the controlled substance agreement signed, but it forces you to take a screenshot. It asks you to put a screenshot. So therefore, you can tell. But this also educates other providers as well that might not check just to see when it was not only prescribed but picked up. So we don't have excessive supply of this. So I think it's a requirement probably even more for legal reasons. The reason why you want to have a screenshot, that's my preference, is because legislation or consequences might come later on, and these PDMPs do not store the information past two or three years. So therefore, you want to have something in your documentation or to justify prior MAD doses were this level at this time. That's interesting. We have it integrated into Epic, but again, the issue is it's sort of like it's a manual sort of thing, and it's not a hard stop. I think that's part of the problem, and we have a lot of trouble in our system asking for any hard stops. There's a hard stop to say that, yes, I checked it, but there's not sort of that hard stop that says when you're actually in the PDMP. So I think there's kind of this little bit of disconnect there in the technology. I was going to say, we have that in my system, and so when I sign a controlled script, you get the pop-up that asks for your password or whatever, but before that, I get a pop-up showing me the last things that have been in the PDMP and asking, do you want to mark this as reviewed? And so then I mark it as reviewed, and then I sign it. I like the pop-up, and that's why I was saying kind of a best practice advisory BPA sort of thing where it flashes in front of you. So we sort of have parallel sort of views, and it's a little more manual, I think. It also comes, I think, in that pop-up, it also tells me, well, it tells you the morphine equivalents if you're doing an opiate. If you're prescribing Suboxone, which is the only opiate I prescribe, it tells you it can't do a morphine equivalent, so it's actually not very useful to us, but I wonder if from a hospital standpoint, if you can make the argument of like, you know, they're coming down really hard on opiate prescriptions and stuff, like this will help people understand like how much they're prescribing and that sort of thing. Yeah. Yeah, that's great. So I think the other thing that we ran into, your other point about like having sort of that screenshot sort of thing, because we went through some different iterations of like having like a smart phrase, dot phrase sort of thing, and you could say, you know, we had kind of said like it did or did not match what sort of like what the patient had told us sort of thing, but our legal group said that we could not put such information into our documentation, because they said that whatever information is in, and we couldn't print out copies, couldn't do any of these things from the PDMP, because that is very protected patient information, and it was not for us to distribute or disseminate in any way. So again, I don't know if that's just our attorneys, you know, because every, you ask 10 attorneys, you know, sort of thing, but I would just, you know, I would just give you that information, because that's something, you know, like we thought, why not? Like why wouldn't we, you know, make documentation of, you know, discrepancies, for instance? So that was another thing we sort of stumbled over as well during implementation, yeah. »» So my PMDP, PDMP, experience is »» PDPQ, PDMP, say that 10 times fast. »» It's all the P's, all the P's, yeah. It's with college mental health and prescription of stimulant medication. I actually have an interesting, I prescribe and sometimes I see them every three months, and then I realize they're actually not using the medication. So I actually started tracking the system just to see are they even using what I'm prescribing, just to have that conversation with them, you know, okay, so you're struggling in your classes. We have talked about that. We have talked about all of the props that you need to do and establish, and here's the medication, and that was prompted because a pharmacy called me and says, you can please stop sending prescriptions, like we have seven in the system and they're just not picking it up. So now I check to decrease my work, you know, it's like maybe I don't need to be sending the prescriptions. »» Plus they might not get it given the shortage. »» That's right. »» Yep. All right. Great. This has been wonderful feedback. »» All right. Again, thank you all for being so enthusiastic in participating, makes this process much easier. Again, it's like using our collective wisdom, it's better than just having one consultant. So thank you. So we want to welcome you to come join us to this community. As Tim mentioned earlier, we have this psychiatry open forum that Tim organizes. It's a monthly one-hour session. We meet on a second Friday of most months from 1 to 2 p.m. Eastern Standard Time via Zoom. And then again, we take turns similar to the whole Project ECHO setup. We take turns presenting cases and then curriculum. And we also, sometimes we end up doing a group project together, like the presentation you see today. We have presented other places. And in fact, our group is so advanced, we have actually been invited to the 20th anniversary of the International META ECHO Conference. So either Jackie or myself will be representing psychiatry, along with ACGME to talk about this whole PDPQ project that we're doing. So we feel very blessed. And so APRA is looking at how to go about doing this for the community, ACGME is looking at for GME community. But APRA is also looking to how do we go about helping our members in the community who might not be in a medical education training setting, but you're also educating other people. You're educating co-workers. Many of you might be medical directors for community mental health agency. Do you need to train your physician assistants? Do you need to train your nurse practitioners? Or do you need to actually train your mental therapist as well in measurement-based care so that you can actually do more QI patient safety work? So that's a future we're looking at. So Jackie and I are looking into different ways for how we might be able to develop this. And we're currently working with the APA's council on quality care to look at maybe potentially rolling out curriculum. So we'll look at whether we can actually develop something similar to the structure ECHO that Tim mentioned earlier. So weekly meeting for like about five months or so for an hour each time where we give you some basic little nuggets of information, what they call theory burst. So that might be on stakeholder engagement. It might be on assessment. And then we actually spend most of the time doing a group discussion about what the concept is all about, this homework in between. And then after that's done, then we can transition over to the open ECHO, which you saw today. And so we have included references here. Now we're trying to stick to the APA dictated timeline. So we have to make sure you have time to fill out your evaluations and ask questions. So anyone have any questions? I guess this is about your experience just with QI in general, and I guess measurement-based care too. So I feel like, you know, there were, who's that internal medicine doctor who wrote all the books about like the checklists and stuff, like, yes, Atul Gawande. And so that, I feel like, and that was what, like 15 years ago or something like that. And so I feel like the medical community got very sort of interested in QI. But psychiatry, at least the people that I've worked with still seem like just so skeptical of it. Like, you know, a scale can never beat my clinical judgment. And that's, I mean, like, which I don't disagree with, but I'm just wondering how you all have gotten other people on board with thinking this way. »» Yeah. I don't know if this is, can you hear me? So I think that that's a great point. And I think that, you know, in general, like in education, when it comes to quality improvement patient safety, you know, sometimes it really does kind of become this like it's a check box. We have to just, you know, kind of make sure that, you know, we've educated people in this. But that's the whole point of PDPQ is really to try to get this into clinical practice from day one and really trying to develop faculty who can teach in this way so that we have, you know, next generation that really gets this and understands it. It really does have to become the culture and like it has to, it's like what I feel like I do in my job is just constantly talk about it. And I just get the vocabulary out there anytime I can, you know, my residents are always making fun of me because like here comes Dr. Hobbs with her quality improvement stuff again. You know, any place, I don't care if I'm talking about psychotherapy, I will talk about root cause analysis, right? I mean, you can merge the two. So that's kind of my thought about it. Yeah. So actually, we can probably have a whole conference just talking about implementation and measurement-based care. The rationale behind it, so it's actually all these resource document that's actually being developed by the APA to kind of looking into it. So it should be available in the near future. And a key take-home is actually, many of you alluded to earlier, follow the money. So guess what? That's what the payers are doing. So more just like CMS, there's all these meaningful use initially, and now it's all about improvement. So we need to have quality metrics in psychiatry, because otherwise, we'll be falling behind the rest of medicine. And people continue to question what we do as a profession. And more and more people want to actually do what we do. So we have nurse practitioners with physician assistants and psychologists are saying, hey, we could do the same things these doctors are doing. But in reality, as you noted, it's not. We have to demonstrate why these other people cannot do what we do. And that's where the metrics really come into play. So that's why in one of the cases we presented today, it was a little bit about measurement-based care. And also people who do it have also found it to be helpful in other settings. So for example, if you're a child psychiatrist like I am, so I'm the training director for Child Psychiatry Fellowship at University of Washington. And I do clinic with like fellows, like when Michelle was my trainee. And sometimes we have some patients who are just difficult teenagers who don't want to open up. And sometimes the best way to have a conversation is actually doing the measures. Because when you do a measure, they say, fine, one, two, three. And then eventually, you're able to launch it. Why a three? Why a two? And it's a great way to do that. So there's many different ways for you to actually make the measures make sense to you. And also a lot of times I work with minority population as well. So I actually spend one half day a week at Asian Counseling Referral Service, which is a community mental health agency serving Asian-Americans in Seattle. So it's a premier agency for Asian-Americans in the entire state. And a lot of times the families, they're kind of going, hey, it's kind of hard for me to understand what improvement is like. But once you start showing the measures, and you're showing how the reduction in scores has actually been demonstrated based on medication, often they go, wow, OK, fine, we'll keep on taking medication. Otherwise, their default position oftentimes is, why are we taking these medications? Why can't I just take supplements? So there's different ways. If you're a creative clinician, you'll be able to figure out how to make measurement-based care work. But again, also, we discuss a lot about administrative hassles. So the trade-off has to work. But everything goes back to a value proposition. So any other questions? Any questions from online folks? Well, again, thank you all for being here today, especially on the last day of the conference. So safe travels, everybody. And then we hope that you'll reach out to Jackie and join us for our open echo in the future.

patient safety

quality improvement

psychiatry residency

collaborative initiative

PDPQ network

Project ECHO

case-based learning

technology integration

prescription monitoring

clinical practice

education innovation

healthcare disparities