Well, today, you're going to be hearing from the four of us on our series of QI projects looking at advancing mental health service access through equity-driven quality improvement initiatives. QI Made Simple-ish. My name is Lucy Obunwabedo, and I'm one of the Public Psychiatry Fellows at UCSF. So we don't have any financial relationships to disclose for our talk, and just a quick overview of what today's session is going to look like. We're going to give a very rapid review of QI. I'm sure many of you here might be experts in quality improvement, so please pardon us if we're missing some of the nuances that you may know from your work. And then we'll describe each of our projects that really look at implementing quality improvement at community health centers throughout the areas. So my colleagues, Heminda, Sam, Paul, and I will go through our work. And then we hope to really have an interactive component for this talk where we can maybe break out in smaller groups and brainstorm ideas of how we may be able to address certain things in our individual places. Sounds like fun? Yeah. Awesome. And just a quick overview, the context in which all of this happened was really through the UCSF Public Psychiatry Fellowship. We have the beautiful Beyonce in our program, too, so if you want to come and join us, hey. And so this program really is open to fourth-year residents at psychiatry residency programs, usually ones that are regionally located around here. So we have fellows that are coming in for fast-tracking from UCSF, from Stanford, from San Mateo, from UC Davis, but we've had other people that were a little bit further out, too. You could also do it as a fifth year after you finish residency. And in my case, I'm doing it as an early career psychiatrist after residency, doing it while working as an attendant. So the fellowship really is really structured to help have an immersive experience during the year where our fellows are located at community health centers around the city. So about four days a week, they spend their time doing clinical work and working with their teams there. And then we have a day where we're working on the actual scholarship of the fellowship, which includes didactics, having supervisory engagements, and working on our QI capstone project that we'll talk about today. And each of the team really is comprised of the fellow, their research supervisor, which is an assistant who works with them throughout the course of the year, and their clinic supervisor, which is some of you in the audience. Thanks for joining us. So why QI, right? Why do we do quality improvement work? As I mentioned, some of you may be really well-versed in this work. Really I think much of this got a lot of attention in 2006 when Institute of Medicine published their Quality Chasm series that really highlighted the importance of QI in healthcare system. So we know that QI enhances patient safety by focusing on how we improve processes, systems, and practices, and not limiting it to just individual providers, but extends to the entire healthcare system at large. So it emphasizes collaboration, coordination, and sharing best practices among different stakeholders. So clinicians, administrators, and patients, and policy makers can come together to drive improvements in healthcare delivery. And I think this leads to better health outcomes overall, and enhances all of our efficiency. When you improve access to care, when you improve workflow, when you improve cost and efficiency, it can help actually enhance patient satisfaction and their engagement in their care. And all of us here are lifelong learners. We're at an APA conference getting all our CME credits. QI really fosters that continuous learning and innovation model for all of us. By regularly having us measure our performances, measuring our data, collecting data, implementing feedback into systems, we can actually identify areas of improvement through our own successes, through our own failures, right? And then helping us drive ongoing enhancement in how we give care. And I think it's important to promote transparency and promote accountability for all of us. When you have evidence-informed decision makers by stakeholders, you can really drive good care. Some of us are trainees here, and much of the accrediting bodies for healthcare professional training programs are requiring that trainees have some basic foundation and quality improvement mechanisms. So this is really important for all of us. And so what does QI actually look like in terms of the timeline, especially for our trainees here? So basically, what we start off with is we need to figure out what it is that we're addressing in terms of our quality improvement process. So we identify the issue. This is done pretty early on. During our time with the fellowship, this was July, August, September, we were looking at different questions within the system. We'll talk more about that in a moment. After that, we start thinking about how do we design the study and actually answer the questions that... Oh, what happened there? What just happened? Uh-oh. Oh. I touched something. Okay. Oh, snap. Okay. Let's see. We may need that AV... Oh, there he is. Okay. Okay. I shall not move. So we design the study, actually try to figure out how do we answer the questions that we're posing for ourselves, our systems that we're in. Then we start collecting the data and looking at that data and analyzing it. And then afterwards, we talk about the findings that we have with our peers, our colleagues, the systems that we're in, and hopefully try to make some action on that. And so when we're looking at identifying the issue, we typically want to choose a topic that is meaningful to us. And it's important to the systems that we're in, the clinics that we're in, and hopefully also the larger field of public psychiatry or whatever fields that everyone is in specifically. The question has to be something that is answerable, not some existential question. I think that's important in order to be able to provide some sort of action at the very end. And then we start looking at has anyone else done this? Do literature review, what kind of things have already been implemented, have not been done? Are there any similarities anywhere else that maybe someone else is already doing that we can learn from? All righty, so once you've kind of identified that problem you want to investigate, it's time to design your study. So asking questions and learning what sort of data are missing is kind of a crucial part of that process. We each kind of did so collaboratively within each of our fellowship clinic sites and oftentimes it reveals that your project may need to answer some basic questions before you can get on and kind of design some major intervention. In addition you may want to consider what kind of data you need. Oftentimes qualitative data is overlooked as being really informative but just know that whatever sort of method and data you choose make sure you take into account a realistic time frame for doing all that because there's oftentimes competing clinical demands. And finally you want to consider anchoring your QI project to some sort of framework so as not to get lost as to what to do next. PDSA is one framework that I'm sure many of you have heard of that involves plan, do, study, act. We've kind of already been talking about the plan part of that so that's you know choosing the topics, what kind of data and methods you'll pursue and we'll kind of cover the rest of the PDSA cycle throughout this presentation but just know that there are other frameworks to consider. REAIM is one of those. Okay so once you've got your study design you start collecting and analyzing the data. This is the do part of the PDSA. Some important things to keep in mind here. The role, the really important role that a lot of the clinic leaders and administrative staff play in terms of helping you collect this data feasibly. So that might involve joining different planning committees, joining meetings, listservs and all that is really to help you sort of get the word out about your project. And then you can move into the S or study phase of the PDSA and that is kind of like you know the analyzing of the results, seeing whether or not they match up with your conclusions. And assuming you've collected some sort of quantitative data I found that we often stick with sort of more basic descriptive statistics and statistical tests. They often suffice. A lot of the times we don't really have too much capacity to go into like deep sort of complex statistical analysis given kind of like low end and limited bandwidth. And then yeah we once we've kind of finished the analysis you then enter the act phase of the PDSA. So you can ask what did you learn? Do you have enough data to make recommendations or do you need more? And if you do have enough data don't be scared or underestimate the power of having a null result if you do get one. Sometimes in our field this is kind of seen as a as a negative sort of thing but actually that can be just as informative for leadership to hear about future interventions. And also just like acknowledging the success of like completing a project right like that's a really big thing it shouldn't be minimized at all. So you know you want to celebrate that and and yeah you know it will definitely generate more leadership opportunities I'm sure. You may even end up getting an APA presentation like we did. So now we'll transition to talking about our QI projects. You'll notice that three of our projects deal with the topic of substance use disorders. So before we start talking about the projects themselves we wanted to provide some context to what catalyzed these projects. Like virtually every community around the United States the Bay Area has been massively impacted by rising opioid and meth use. It's nearly impossible to be a community psychiatrist and not bear witness to this every day. So this slide here shows how in 2019 specifically San Francisco started experiencing a rapid uptick in overdose deaths specifically related to fentanyl as well as meth use. The number of deaths tripled from 2017 to 2020 in San Francisco. You can see from this graph that fentanyl the gray line was complicit in most of these deaths and actually meth was on board for over half these deaths in San Francisco as well. In fact the plurality of overdose deaths in San Francisco involved both fentanyl and meth use concurrently. This map here shows you the neighborhoods adjacent to the Moscone Center. So our red dot here. So for those of you who aren't from San Francisco you have the Tenderloin and South of Market neighborhoods here. These have been the hardest hit by the overdose epidemic. I'll also point out Bayview Hunters Point here which is the neighborhood in San Francisco the highest number of black residents which has been extremely hard hit as well. And then this slide here it deserves a whole 90-minute talk of its own but for the sake of time I'll just let this graph speak for itself. In San Francisco the overdose epidemic has been disproportionately affecting black residents of the city to an enormous degree. And finally I'll note that the fentanyl and meth epidemics have affected the other Bay Area counties similarly including San Mateo County where my co-fellow Harmander works and where a QI project took place. So I'll pass the mic to Harmander and have her explain a little bit about her project. All righty so like Paul was mentioning I work just south of San Francisco and so that county is called San Mateo County and for those who are not from this area you can kind of think of it so San Francisco Airport is actually in San Mateo County so look essentially from that area all the way down to Stanford is kind of the area that I'm working in. And so my project similarly to kind of every three of all three of us here does look at kind of what can we do within our community to really bolster the care of the folks who are living there. And so San Mateo County is both urban and rural in terms of kind of the diversity of the communities we do in our clinics see folks who have severe mental illness have substance use issues as well as housing and financial insecurities. And so when I was thinking about the kind of a lot of the questions that were coming up looking at the clinics that that I'm working in with regards to substance use there were a lot of folks who are coming in and they were using substances but we weren't able to necessarily clarify exactly what kind of how is it that they're using these substances a disorder is that not and I want to understand better like how are we actually screening folks for substance use disorders within our community and then also try to understand what is it that we're providing for these folks when they come in and what kind of things are influencing the clinic staff in terms of what we are able to provide or not provide to our patients. And so this all those kind of those questions and during the planning phase we converted to these aims which helped guide the rest of the project going forward. And so the study design that I decided to take on was both a qualitative as well as a quantitative approach with the idea that they will complement each other. And so one half of the project was to do a survey which was sent out to the clinic providers where there were questions about kind of what kind of tools are they using to screen folks for substance use disorders. What are the prescribing practices resource referral practices and really gauging what is what are people's awareness and comfort levels with each of these specifically for alcohol opioid and tobacco use disorders. And I also want to understand more about kind of what kind of barriers and facilitators exist for folks in the clinic and with regards referring to individual resources or MAT. And so this was fairly subjective and this like I said complemented the chart analysis part of my project which looked at the intakes that we had within our clinic and kind of pulled out what are the ICD-10 codes that people are getting diagnosed with both psychiatrically as well as substance use related. And what are the treatments that are documented during that intake process and whether any harm reduction sorry was discussed. And so this was the idea being kind of our folks reporting differently than what's actually being documented. And I think that's important when we think about some of the results that came up. But before we get to that there were a number of different challenges that did come up during not just kind of my project but I think all of us will be able to discuss. Balancing clinical and quality improvement work can be difficult at times especially given how busy some of our clinics can be. And this comes up in the results to like time is often a barrier for many things. And so just being able to to find a good balance of that. With regards to the chart analysis piece kind of when we enter data into the EMR it's for patient care. It's for communication. It's not necessarily for the emphasis of a project. And so being able to filter through that as much as possible was also kind of challenging at times. And then getting staff buy-in too. Which at first was actually I thought would be trickier than it was. And we'll go into some of the reasons why. But if people need to feel like they there's a they have some say and there's some interest in what the project is. And so you know kind of given all this stuff we could give up. Or you know we could go on which we decided to do. And you know with the wisdom of Beyonce. And so overcoming some of these challenges. I think that one of the biggest things was just being transparent. But what is the project looking for? What am I doing? Why am I doing this? And ultimately the goal was to improve patient care and how we're delivering it. And everyone was on board with that idea. And so it was fairly easy to get buy-in for the survey piece. And being able to kind of connect with staff directly too. And really talking about what are some of the concerns that are coming up. And also emphasizing confidentiality. Like some you know sometimes there are ideas or thoughts or concerns that maybe you know they're not like people aren't doing something that they should be doing. Or maybe there's some guilt. And so really just emphasizing it's okay this is confidential. This is really just so that we can improve what we're doing as a whole. And so kind of being able to overcome those was was fantastic. And does kind of give a sense of being able to survive. And pride too. And kind of what Sam was saying earlier. To be able to get go through this process and create something that for our communities. So with regards to some of the results which I won't go into too much now. Just for the sake of today's talk. But what I did find was that folks are talking about substance use. Which pretty much most of the visits. Like 90 plus percent of the visits substance use is coming up. Now how that's being talked about is a little bit different. They're not a lot of folks are using structured screening tools. And so there's a possibility we may be missing some folks who are using substances on a more kind of a level that we would consider a disorder. But the fact that we're talking about is a huge success. And that was something that I emphasized when I kind of brought this up later on to to our system as well. So we're talking about it. Let's take it a step further. There's a lot of variability in terms of kind of what resources are being provided to folks. I think that kind of gets to another issue within our system. Is that you know sometimes things can be really siloed or fragmented. And so how do we join those things together. And then you know another thing I want to kind of point out is that our EMR just the one that we're using right now and I think a lot of folks can agree in many settings that you know our EMR doesn't always take into account patient care. It's more about billing. It's about communication. And so how do we really optimize that whole process to be able to actually understand what it is that our patients are going through and what each provider within the system is seeing with that person. And so this is kind of like I said the the preliminary findings. And so we use these findings to really study and then thinking about what we could do to take action steps going forward. Education is a big piece. So presentations, handouts, about resources, about what is that we're doing, what we're not doing, trainings around MAT would be essential and something that we've already kind of started to implement within our system. And providing support to the clinicians in order to be able to prescribe MAT whether that be a consultation group or or kind of colleague support and supervision. And then trying to create embed a system within the clinic that involves screeners without burdening the staff which is tricky just because time and a lot of these folks are very complex in terms of their history and and their living situations. And so that's still kind of a work in progress and a better EMR of course is something I think we all desire at some point in our careers. So with that I'll pass it on to to Paul to kind of talk about his project. All right so my QI project considered how substance use disorders are addressed in an intensive case management setting. So my project was inspired by my clinical site which is a large intensive case management clinic in the heart of San Francisco where a large proportion of our patients struggle with substance use disorders along with mental illness. My clinic is made up of multidisciplinary teams that consist of case managers, nurses, psychiatric prescribers including MDs and NPs and peer specialists along with non-clinical staff. We work with a population that is a high rate of homelessness or housing instability. Our clinic has been tremendously impacted by the overdose epidemic with 27 of our clients dying of drug overdoses in 2020 as well as 83 of our clients passing away altogether that year. And not that you need more proof at this point that the overdose epidemic has been devastating San Francisco but to further contextualize things in 2020 there were 725 overdose deaths in San Francisco which was almost triple the number of COVID deaths the same year. So I sought to better understand the challenges facing an ICM clinic such as mine. Few clinics can provide such a high level of support to its clients and yet at times we still seem to be no match for the devastating power of the fentanyl and meth epidemics we face. There's tremendous work going on at my clinic every day in terms of helping our clients with substance use disorders and I sought to better understand what we were doing effectively and what we might be able to improve. So you can see in the bottom right corner here we're in the plan phase at this point. So to design this QI project I met with leaders and clinical staff at my clinic to discuss this topic and how best to gain the information I sought. The public psychiatry fellowship research team and faculty provided a lot of crucial support as well. We settled on a format of interviewing 10 ICM providers across multiple disciplines and teams within our clinic. With the help of others I designed a standardized interview that asked providers about their experiences addressing substance use disorders among their clientele. The UCSF and SFDPH IRBs gave their approval and I began recruiting participants. Ultimately I was able to hit my recruitment goal and completed 10 interviews. Then our fellowships research assistant and I coded these interviews first separately and then together to reach consensus on our coding. From these 10 interviews we generated hundreds of data points from the interview responses and analyzed these data points to pull out major findings and patterns among them. And finally I created presentations like this one to disseminate to stakeholders and other interested parties. It's beyond the scope of our workshop today to talk about a length about our project's findings since the main purpose of this workshop is to talk about the process of creating and executing our QI projects within the context of our Public Psychiatry Fellowship. But I wanted to share briefly some findings and I encourage you to seek me out if you want to talk more at length about some of the findings of my project. So some of the challenges my study participants described in addressing substance use disorders at my ICM clinic included systems level challenges, challenges addressing the underlying drivers of substance use disorders, and challenges related to client insight in stages of change. And some best practices my study participants identified included rapport building, shared decision making, striking while the iron's hot, harm reduction, and I think really crucially recognizing that abstinence isn't the only marker of success. One thing that came up for multiple people is feeling really discouraged based on how challenging the overdose epidemic has been, but others acknowledged that even reduction in use or using more safely is something that's worth being celebrated. And then some improvement ideas that were generated by my study participants included ways of improving access to substance use treatment, additional training on substance use disorders for clinicians at an ICM clinic such as ours, and considerations on how we might advocate for improved conditions in the communities where our clients live. This is something that came up over and over again is that our clients live in environments where it's really hard to support healthy lifestyles and abstinence or even reduced use. And of course the PDSA cycle is just that. It's a cycle, not a straight line with a beginning and end. So ultimately my QI project generated numerous findings and ideas for improvement, and each of those could become a QI project of their own. So thus the cycle would start anew. So that concludes a summary of my QI project to date, and next you'll hear from my co-fellow Sam. So I work in a public mental health clinic in San Francisco's Mission District, where we aim to provide mental health services for the Latinx and LGBTQI plus populations of the city. Much like other places in the city, there's a lot of rich history and culture, oftentimes depicted in murals like the ones that you see here. And yeah, for those that are not from San Francisco, I hope you've had some time to be able to get to know the city and the history a little bit more. I know that was definitely a big part of my journey when I came to fellowship, especially in wanting to serve La Comunidad and being able to understand the culture where folks come from. And when I did arrive in fellowship here, I was actually surprised to learn that, you know, as we've been discussing, there's this huge high prevalence of substance use disorders or SUD in the city. But many of my colleagues were not actually prescribing as many of the FDA approved treatments for those disorders. And I thought that that could be a really huge opportunity to expand treatment capacity within our mental health clinics. So I wanted to investigate that more. So using the QI model that we've been talking about, I first did some planning and identifying of my problem. And so when looking at some existing data within our city, our leadership reported that there's a minimal number of prescribers in the entire behavioral health system who were actively prescribing these medications. And this is across, you know, the board. Opiate use disorder, alcohol use disorder, nicotine dependence, Narcan prescriptions is fairly low. So for me, that was, you know, confirmation actually that, you know, at least in the mental health setting, we were not prescribing as much of the FDA approved medications. And it's not that our prescribers weren't having these conversations and asking their clients about SUD at all. We definitely were. About 85% of our prescribers routinely inquire about SUD. So something was going on in between asking about it and prescribing. And so that's kind of where I wanted to do more of my investigation. So this is where my project kind of comes in. I wanted to understand the barriers and facilitators to mental health providers and what challenges they face when prescribing these FDA approved medications to treat SUD. And for the purposes of my project, I would focus specifically on opiate use disorder, OUD, alcohol use disorder, and nicotine dependence, ND. And with this information, hopefully develop some recommendations to increase the prescription rates among all the providers, hopefully lower rates of SUD and overdoses in severity of comorbid mental illness. And for me, again, I really think that it was an opportunity to kind of realize this no door being a wrong door for substance use treatment for a lot of our clients who we know really need that help. So what did I actually do? There were two pieces to my project. The first was I put together a brief online written survey for prescribers to take in our systems, and I asked them questions about, you know, how much are you prescribing for SUD, what's enabling you, what's impeding you to do so. And in this process, it was actually really helpful. A lot of my co-fellows actually helped test the survey, and we kind of got it down to a point where it took five minutes to fill out. And to get the word out, what I did was I actually attended different prescriber meetings at the different clinics, and I had them fill out the survey during those meetings so that folks could have protected time to do that survey. And actually, I think that really helped with my response rate. It was pretty high. The second piece that I did was I put together a one-on-one sort of structured interview. So I put together an interview guide, and I reached out to kind of key stakeholders in various clinics, and I was hoping to get the more nuanced details about what was going on, suggestions for how I could potentially improve substance use prescriptions. And that kind of entailed, again, identifying these key folks in the system, coordinating a time to meet, conducting the interview, coding, analyzing. So for me, I'm kind of burying the punchline here, but this was pretty ambitious to kind of do both of these methods. I'll speak more about it in a minute, but I just wanted to highlight that now and then also kind of highlight just some brief results of what I was able to find. So here's some initial survey data. For me, the key takeaway from all of this is that prescribers are prescribing less for opiate use disorder and are less confident in doing so for opiate use disorder relative to nicotine dependence or alcohol use. So you can see here, when you ask providers, this is the chart on the left here, when you ask providers if they prescribe medications to treat these conditions, only about 20 percent of them prescribe for opiate use disorder, whereas about 65, 75 percent of providers prescribe for nicotine dependence and alcohol use disorder. I also compiled this chart on the right, which was the rate of prescriptions. So this is so kind of like among all the encounters with patients with a diagnosis of SUD, sort of like how much are we actually prescribing among those encounters. And actually what surprised me, I found that these numbers were a little higher than what I was expecting, 41 percent for alcohol use disorder, 34 percent for nicotine dependence, and 25 percent for opiate use disorder. But in general, that sort of lines up that pattern. I'm not going to present to you confidence data here, but I'll just say that it pretty much mimics what you're seeing in terms of how much we're actually prescribing. Less confident for opiate use disorder. And that's a pattern I actually saw for both addiction-trained providers and non-addiction-trained providers. So for me, a key takeaway is that when we do see substance use disorder in our mental health clinics, especially opiate use disorder, our providers feel less comfortable and prescribe less. So, oh, this is a little bit of my qualitative analysis here. And when you ask them kind of like, again, what's getting in the way, obviously they don't feel as comfortable with these medications. And so what they recommended was just more individual-level support. Can we get more trainings around it? Can we have an expert consultation model where maybe we can talk to folks who have more experience in being able to manage these issues? And can we have straightforward guidelines so that we can follow things and just better manage SUD? And then there was also some systemic challenges. Obtaining lab work was one big thing that folks often cited. It was difficult for a lot of our patients, many times because the services are disconnected, as we were referencing earlier. Our workflow is also very non-standardized across different systems, and we oftentimes lack administrative support to kind of manage things like paperwork. I know probably a lot of you are familiar with how challenging prior auths can be. So, yeah, that can be a lot. So where does this all leave me now? So I'm still wrapping up my analysis, so I'll be finalizing that in this coming month, presenting at various working groups, clinic leadership meetings, where I can get some more feedback on what's missing from my data before I do make formal recommendations. But I can at least say I have enough data to start a dialogue around what are some of these initial recommendations, maybe talking about an expert consultation model, making that more formalized, standardizing our treatment protocols and integrating more services to facilitate handoffs and transitions of care. Here's what I really wanted to talk to you about, is the challenges and lessons learned along the way. As we've already kind of been talking already, time is a very limited resource. Working around schedules of busy providers at various clinic sites, including my own, was very challenging for me. I had to find strategies that were going to give me the best bang for my buck. And if I had to say, like, I actually really do think it was worthwhile for me to attend the different clinic prescriber meetings, although it was challenging to do that, it really helped with my response rate. Like, if you can imagine, like, sending out a survey just online, how many people actually fill that out, it's not very hard. But when you give people protected time to do so, it actually helps. So I would recommend that. I also had to give myself enough time to do it. For context, for my project, that took me about two to three months to actually attend all these different meetings. And then there's the competing demands of, like, clinical and QI work, which for me, I had to learn that the hard way. I referenced earlier, I think I bit off a bit more than I could chew in terms of collecting my data between two different, you know, quantitative and qualitative methods. And so I kind of had to put the qualitative stuff for myself on hold. I got maybe, like, a few one-on-one structured interviews. And then it just became very challenging for me to kind of coordinate meeting with the next person, scheduling all that, and then competing clinical duties. So, yeah, I limited that and just mostly focused on my survey. And then finally, one kind of, like, other major lesson learned was kind of just knowing my audience. So I kind of came into this project with a bias, right? I wanted to make sure there was no wrong door for, you know, folks to come in and get substance use treatment. But it was important for me to be open to competing perspectives or alternative perspectives as to understanding why the system is set up the way that it is. And so in that vein, understanding, like, the funding streams, for example, was super critical. Because then you get to know, like, how your organization works, what's funded. And also, like, those organizational charts. Like, how many times, like, have you actually, like, looked at an organizational chart? Like, it's so incredibly helpful to do that because it can help guide you. Okay, who's the person that I need to talk to? Who are the relevant people I need to discuss and ask some questions to? So that's what I would say, you know, was really helpful for me in terms of knowing my audience. Anyways, so I'm going to wrap up here. I'll let Lucy speak, and then we'll kind of get into our group discussion. Great. Thank you all for your attention so far. One of the things that I'm really passionate about is looking at our healthcare workforce. We've all heard this time and time again that there's challenges in really recruiting, retaining, and actually ensuring that those that are doing all the work that my colleagues have outlined are actually there. And if you're in any system, if you're in a leadership role, if you're, you know, someone that's an employee in a system, I'm sure that the issue of workforce is one that you're probably wrangling with as we have a huge attrition in mental healthcare, and I think in healthcare in general, and COVID did not do us any favors. So the purpose of my, you know, project within all of this is really to look at how do we advance diversity in a DPH, Department of Public Health system, looking at behavioral health services. And so just briefly, you know, we looked, the goal was really for us in the planning stage of the QI, the question was who is in our psychiatric prescriber workforce? So our NPs, our psychiatric nurse practitioners, our MDs, our DOs, those that are prescribing medications and treating our clients and our patients in our community, who are they, right? Because we want to understand what are the demographics of these folks, and what are the ways that we can actually understand their values, their goals, to ensure that we're able to actually retain and not just recruit them, right, but retain them. Because if we have this information, it can really help inform our initiatives, including how do we actually, you know, provide culturally congruent care for our patients in our community. So really briefly, you know, a lot of my colleagues have talked about, you know, the methods and the planning that they did, and mine was very similar in the sense that one of the things that I needed to understand is how does one actually ask these demographic information, right? Death by survey, we're all, if you get one more email. But when you're asking people about really personal, you know, information about who they are, whether it's their race, their ethnicity, their sexual orientation, all these sort of kind of potentially complex and challenging questions, what are the current best practices and evidence-informed guidelines to actually do demographic workforce survey? I was not as well-versed in that, so I had to do some really important research to understand and talk to people who are doing this kind of work to ensure that in my approach, right, I'm not inadvertently causing harm. Because whenever I get some of these surveys in front of my face, I know I have certain reactions, like what is this for, where is this going, how would this be used? And it was really important to do a lot of community and stakeholder engagement. So just like Sam and others have talked about, I went to about over a dozen clinics where these prescribers are located and actually had meetings with them, talking about what we're trying to do, but asking some of their questions and reflections. Are there reticence about this? What are they concerned about? You know, issues around confidentiality came up a lot, like if I tell you these things, what are you going to do with it? Where is it going? Who gets access to it? Can it be traced back to me? Is there potential negative consequences for me if I'm really honest about these things? What are you asking? How are you asking it? So these things really gave us a lot of good information, and I think it allowed the community to be part of this process and actually help us in creating the survey and then disseminating it in a way that allowed them to actually have protected time, like Sam was doing, to do it. We had a lot of the clinic management team really get engaged early on with me and offered their prescribers time during their monthly meeting to take the survey, and I was able to come and, you know, watch folks take it live. Like, it's five minutes, just like people were talking about, because who wants to do an hour-long survey? I know I don't. So that was a really sort of a stepwise progression in how we came to this point. And, you know, just really briefly, what did we access? Like, you know, what were we actually asking? And we won't get into everything in as much depth into the results of all of this, but we were looking at race and ethnicity, time at DPH. How long have you been working here, right? What are the roles, right? Are you a psychiatric practitioner? Are you an MD? Are you a DO? Where are you in terms of your roles there? Gender, sexual orientation, and linguistic diversity, like languages spoken, which is really important because San Francisco is a really linguistically diverse county and city. And we also looked at first-gen status. So, like, are you a first-generation student? Are you a first-generation health care provider? Because we know that those coming into health care systems from different backgrounds encounter unique challenges in their work and in their pathways. So understanding that was really helpful for us. And then looking at recruitment and retention factors, and I'll show you briefly what that may look like, like what brought people to DPH and what is actually keeping them, because that could be a different thing, right? What brought you here may be different from what's actually keeping you there. And then we looked at these engagement and inclusion factors. So these were like domain-based statements from a validated tool looking at certain things like sense of community, looking at trust, access to opportunities in a system, appreciation for individual characteristics. And so for example, under trust, you may have a statement like, I trust that my institution will be fair to employees. If I raise a concern around discrimination, I feel like my institution would do something about it. You know, things like that, gauging like how do people really feel about certain characteristics of domains, about diversity, inclusion, and community, and sense of belonging within that system, and seeing what we were noting, and is it differences based on people's backgrounds? And so briefly, again, we won't get into all the results from every different component there, but we saw, for example, what brought people to DPH. So a huge part of that was wanting to work with a specific community or area of interest, and the environment in the place was also a huge factor, colleagues, and social and cultural context that they were working in. And benefits was, you know, interestingly, you know, third in line, and the diversity of patient population, and salary was the fifth one. But interestingly, as you can see, as folks have been there for some time, what is keeping them there switches up a little bit. We have a tie now in benefits, and opportunity to work with a specific population. I thought it was interesting for me, because I thought the benefits and salary would be really related to each other, but they're actually quite different in this particular response rate. And then workload expectations was also in the top five. So I think as people are thinking about how do we recruit, how do we retain, knowing what factors are bringing people in a workforce is really important, because if you can optimize those environments, if you can optimize those benefits, optimize who they get to work with, that may actually help us in mitigating what we know is a huge burnout and attrition in mental health care right now. And then just really briefly, some early takeaways. So we saw that some of our race and ethnicity demographic data so far actually tend to mirror the San Francisco County demographic information. So for example, we have our prescribers, about 52% were Asian American. And in San Francisco County, that's about, we have about 16% of those accessing care in SF County were Asian American. So we have a huge number of Asian providers who are providing care in a county that has folks from Asian American accessing care. Our white demographics for our prescribers were about 20% or so, with about 20% of those in our healthcare system being white. We had some challenges with our black prescribers. We only have about 6%. And the county's accessing care folks were about 16%. So not everything was closely mirrored. In our Latinx, we had about 12% of those who were prescribers in our system were Latinx Hispanic, while our county had about 16% of those that were accessing care were Latinx background. So there's some congruence, but not really fully. And we're gonna look more into those datas closely to see how we can understand what it's actually telling us. And interestingly, most people that we surveyed on a scale of one to five were very happy with their work at SFDPH at about four, over four and five. So four being the second highest, five being the highest, people were around four and five in satisfaction rate, which is really notable to know that people actually were happy with their work. And some growth areas that I've seen so far, again, we're in the early analysis period of this, is that we need higher linguistic diversity in this region. So for example, we have a huge amount of folks accessing care needing Spanish language speakers, and we don't have a high percentage of our prescribers who speak Spanish. We do have some challenges around our Cantonese and Mandarin speakers as well, but Spanish showed the highest disparity in base of who we have in our system and who's actually in our county. So some of those things we're gonna look at more closely. And then the factor around trust came up, right? On a scale of one through five, most people were around three-ish when we're looking at, do you feel like the system is able to handle issues around discrimination, which is really notable because most of the scores were about four and five. So this was the one area that we actually saw people being like, maybe trust in the system around this particular factor is not as high as it should be, something for us to pay attention to. And then, as we've all talked about, there's some challenges in this work, and I had my own unique ones as well. And I think one of it is really doing this partnership between UCSF being an academic center and a public health. All of these clinics that I worked with were part of DPH, and they're very unique systems and governances around each place. And so when you're coming in to do a project, you have to really navigate each of their own systems. So like, for example, UCSF had their own IRB system that I had to clear, and I had to clear the DPH version, but it wasn't really an IRB type of setting. I think it was a much more complex system of approving quality improvement projects that actually changed after my colleague got theirs passed. They changed their process and their systems, and I had to go through a whole different kind of approval process. So some of those constraints, system-wise, can really present unique challenges, but I think it also could be really enriched to be able to have the diversity of your sectors in your work. So having academia and public health together can really create unique, interesting opportunities for our system in terms of healthcare delivery. And then, like I mentioned before, understanding how you actually do this. I can't say that I've mastered or really have any sense of expertise in this particular space yet. These are evolving things. Every other month, it feels like there's another paper coming out on how you assess this, how do you ask these questions. And so being able to keep yourself as much as possible up-to-date is gonna be really important. And I think I had about maybe 13, 14 sites that I was working across, so trying to implement work across multiple systems is really challenging, both time, both engagement, and just each place has their own unique needs and unique stakeholders and having to really make sure that you're reconciling all of that at the same time. So it was not an easy process, but it was one filled with growth and learning opportunities for me. And then, next steps, we're now in the study and act cycle of the PDSA, really understanding what are we actually seeing in our surveys? Are there unique differences based on satisfaction, based on where people are coming from and their background? Are we seeing certain things around, some of the engagement and inclusion factors that we can look at and understand what it's telling us? So we're in that analysis phase right now, interpreting all these things, and then presenting our findings to the stakeholders and getting them to actually take what we've given them to then execute and implement in their systems. And really, could we collate some strategies based on everything that we've learned so far on how they may want to consider approaching this? Nobody has a magic wand or a magic tool in any of this work, but are there actual strategies that we can actually offer them to help them begin the work? Because when you are gathering the confidence and the time and the trust of folks in doing something like this, you wanna ensure that it actually materializes into something tangible. So a big part of my goal is that at the end of all of this, those in power actually do something with it to help change the system for our prescribers, but also for our patients and our communities. And so, I wanna just take a pause and say we've had four back-to-back talks, a lot of talking at you, but wanted to just ask if there was any questions so far that's come up, any present reflections, happy to engage with that briefly before we go into our breakout session, because we'll also have time at the end for a much more larger group, nuanced reflection time as well. So if there's anything coming up right now, please feel free to come up to the mic and share your thoughts before we do the breakout. Hey, good morning. Hi. Good to see you guys. Thanks so much for sharing this information. One thing that was just coming up for me, particularly I think during Paul and Sam's talk about the differential treatment or prescribing for MAT with OUD versus other substance use disorders, I was just thinking about, one, if there was any thought about there being a possible role of people have OUD, but maybe they're getting treatment for it in a clinic, like in a methadone clinic, and so it's not being caught in the population that you're looking at or the clinic that you're looking at. And then also, I was thinking about whether factors might be at play. And to my mind, the biggest difference is that MAT for OUD is either like a super controlled substance like methadone, or it's like naltrexone or buprenorphine, which are gonna have a direct interaction with the substance itself, whereas like for alcohol use disorder, for nicotine replacement, it's like something that you're taking kind of alongside use and it may not have a direct interaction with the substance itself. And so, I don't know, I was just wondering if like those factors were anything that was, if that was like thought of or looked at, I know it's kind of not totally in the scope of the QI project, but I'm just kind of curious what your thoughts were on that. Yeah, thanks for that question, Sean. Definitely some things that came up in my interviews, for instance, a lot of our clients aren't linked to like specialty clinics and the referral process can just be, can be pretty onerous. And I know a lot of the clinics actually try to make that as like, you know, open office hours, but something that comes up is for our clients, there can be like a small window of opportunity where they're feeling motivated and just to get them physically to another building can be onerous. And then they have to go, you know, maybe every day if it's methadone or routinely if it's buprenorphine and then going to multiple clinics is really challenging. So talking about like, how do we break down those barriers? And obviously that's within the scope of Sam's project as well. So those are a couple of things that came up in terms of why, you know, despite the presence of like these specialty clinics, a lot of our folks aren't getting care. And then the topic of just, I think sort of similar to what you're saying about some of the limitations of taking like MAT for OUD is not all of our clients are in a place where they're really super interested in taking those medications. So thinking about like, how do we, you know, whether it's using MI or different tools, like how do we get them to feel more motivated to seek treatment for these disorders? And I'll just, thank you, Sean, for the question. I'll just add that for the part that you were saying, kind of like, are they getting treatment elsewhere? It's like, yes, definitely. There are a lot of places that are kind of set up within our system that are, you know, absolutely, you know, doing that care. For me, I found actually a lot of it is happening in primary care settings. And part of the reason why they do it in those settings is because they have capacity to kind of accommodate like drop-in sort of hours, which is something that in mental health clinics aren't exactly, you know, feasible. But, you know, to Paul's point, it's kind of like, you know, I think ultimately we wanna be able to build up our treatment capacity because, I mean, I can just say kind of like, you know, anecdotally, I've definitely come across a lot of patients who, you know, will be like ready, or maybe they really like being in a particular clinic and it's too inconvenient to go to multiple different places and have their care in different areas. So it's like being able to kind of like capitalize on that and just say like, look, if you really wanted to start Suboxone, we could do it today. I could put it into the pharmacy right now for you to pick up. You know, that makes such like a huge difference for people instead of saying, well, look, you know, I'm sorry, we're gonna have to call the methadone clinic. You're gonna have to schedule an appointment. You're gonna have to do a Utox and all that stuff. You know, it's like, there's so much, you know, barriers that I think once you start to see that, and once, you know, the patient's kind of aware of those like systemic barriers, they're kind of like, ah, forget it. Like, you know, we're not really gonna go for that. So yeah, I think that's kind of how I've seen things. But yeah, I mean, it's tough, right? You know, because these other places are also really, you know, well equipped to handle this stuff too. But yeah, thanks for the question, yeah. Any other questions? Well, we'll have more time for that. So, you know, hang tight. So, you know, we have us kind of relatively small groups. So we can actually do this, I think, in a really cool and intimate way. So our goal is to maybe have about 12 or 13 people right now break up and maybe to a group of three or so. And maybe as a group, you know, maybe find people that you're not working with, that would be helpful. Although, of course, this might be hard because we have a smaller group. But it would be nice to kind of maybe just talk as a group. Is there maybe one common issue in your specific setting, wherever you're in, whether it's at a nonprofit, whether it's a clinic, institution, that perhaps you may want to address for a QI, right? You can, you know, as you've learned from us, when we can get really ambitious, we can narrow it down. We can think about what is tangible for this exercise. But think about, like, this PDSA cycle that we talked about. How would you design this care intervention for this issue? From the planning stage of it, like, what is your question? Who are your stakeholders? Like, how do you even plan on even beginning to get that, what you need for information? And then discussing the challenges that you may be able to actually have to wrangle with at each step of this PDSA cycle. Like, how might you overcome it? We've discussed some of our own strategies, or in our case, just like acceptance, radical acceptance about what we're dealing with. And how might these vary based on where you are? This could be, again, very informal, very sort of relaxed discussion. Just think about, maybe you already have something that you're working on that's related to a QI process that you want to share with the group. But I wanted to kind of give us a time to kind of play with this, see how this feels, and come back together as a group to talk about it and see what we learned. Maybe about 10, 15 minutes or so. We can just, we have enough room. Maybe you can have one, you know, some group in the middle, you know, in the front, some in the back, some in the middle, and have it happen organically. Any questions? I'll give a 15 minute, a five minute warning when we're at 10 minutes. How was that, guys? That flew by fast. That was 15 minutes. We'll love to hear how each of the three groups engaged with this exercise. If there's any volunteers who want to come up to the mic and share what your group discussed. Or, I guess, I don't know if the room is big enough. Maybe you can sit in your chair and just say, we can probably hear you if you don't want to move. We, wow, yeah, we talked about a couple of different things. So, you know, our group is made up of mostly folks who are in residency at UCSF. So, I'm here to join us, which is really nice too. We're at the VA in Chicago. And we were talking about, mainly through the lens of what our experience has been at our clinics at UCSF, so the academic clinics associated with UCSF, the academic clinics associated with the Medical Center. And one of the big things that we, you know, we're needing to be improved would be access to psychotherapy services for the patients at UC, which is mainly general and adult psychiatry, largely commercially in a shared population, just who we're working with. But we have a really limited range of options that we can direct people to. And there's a lot of variability in terms, there's a lot of variability and also, I think, just low total knowledge about what options are available outside of our institution to connect people to. And so one of the, you know, pains or issues we were looking to work at was to, you know, identify, you know, what people do know about what, like how they can connect people to therapy. Are they the same across different individuals who would be surveyed? So when we talk about that. And also getting a sense of how people view it, who they use as a problem, and how significant of a problem it is for them and the agents. And we kind of jumped the wheel a little bit, but just a couple of things that came up was potentially using findings from that survey or that study to either create some kind of more standardized, consolidated information resource that can be used, or potentially trying to develop more access internally that would be either done by trying to arrange to get more people on board if that's the source of the problem, or to come up with any kind of useful strategies to increase access, like with group therapy, for instance, rather than individual therapy. I really appreciate that, because you can substitute psychotherapy for any service that people need, and some of those processes that you describe can be really applicable. And I'm curious, what are some of the challenges that you guys anticipated? Because I imagine that there will be some, and you probably came up with some ideas of what those could be. One of your dollars is structural development. money one word right yeah I'm curious like who are some other stakeholders I think I imagine residents right in the patients but well they identified any other stakeholders that may be part of optimizing this initiative for you and that you may collaborate with around this the other thing that I think is also important for the perspective of teaching clinic is also that you know making decisions for what residents do or how they spend their time or how they care for patients also kind of by default ties in our supervisors and so that's a really big stakeholder pool that I think is kind of unique to our situation. Yeah the residency program, your PD, your APDs, the vice chair of education, the chair, site directors, I can imagine all those folks having a lot of thoughts about even GME right how you fund hours of resident spend and so really really cool question and I'm I can we do it? Thank you guys. Can I just quickly say to like again like I think it's such a great sort of topic that you guys kind of focused on and sometimes this is something that I know we've already said this but it just kind of like bears emphasizing that it might feel like QI is okay like why are we starting here we know that more people need therapy you know let's just go and ask higher-ups you know for more funding or more streamlined resources of doing it but like you know if you were to do some some kind of survey like that and then be able to go to them and say look I'm just gonna make up stats like one in four clients here have to wait six plus months for therapy services and during that time you know we found that their depression scores you know raised significantly to this point like that like is a much like heavier ask that you can then kind of like put back on them as opposed to just kind of being like well look we know therapy works so they everybody knows like it works right but to have that to quantify it in that sort of way can really can kind of like you know begin to kind of like really directly characterize a lot of things. So yeah, great work. Data. Money and data. Awesome. How about the group, the smaller group to the right? I'm happy to talk about what we discussed. So I do a lot of my clinical work in emergency services at San Francisco General Hospital and one of the things I've been thinking about recently when we were discussing is the use of sheriff's department deputies there and the conflict that arises between public staff and the sheriff's department. Deputies there and the conflict that arises between public staff and sheriff's deputies and our dependence on them when we have individuals who may become violent. So we talked about other ways to be able to quantify the issue and be able to get perspectives from both the clinical staff, be able to get perspectives from patients, be able to get perspectives from the sheriffs, deputies, as well as somebody else's ideas on what works well now, what some of the issues are now. There's a way to compare to a system that uses in-house security staff or training or something like that. It would be interesting to be able to compare what that looks like and what's more than that. So we talked mainly about the planning phase, and maybe I should go further than that. But there's talk a little bit about key stakeholders. And largely, this is hospital administration. It's the same time. I don't know what that looks like, that whole process, where you have to talk to patients. That's a whole different Google process to be able to get patient data as well. So there's a number of barriers, and that's why this hasn't been done before, because this is more to be able to do with something like this. Those are a few of the things that we talked about. Y'all went there. That's a third-wheel discussion in psychiatric services now, right? The involvement of this carceral system in our hospitals, and the fact that you didn't shy away from actually approaching that. Because I think one of the things I've seen is sometimes when you have a topic that's this hot, literally, both in our societal dialogue and our discourse, it can cause people to just be like, you know what? I'm good. I'm not even going to go there. It's just how it has to be. I think having the courage to sometimes ask questions in what can be uncomfortable context is really important, and having the humility to know that it's going to be, for all stakeholders, a really complicated question. One of the things I was thinking about in terms of presenting some rationale for folks around, if the argument is that we need less sheriff's involvement, we go back to why we had them there in the first place. They were there maybe, what are the reasons why? Are they there to protect us? Are they there to keep us safe and all those things? If that's actually not happening, if their placement and their role in that situation is not helpful, can we then reimagine what other ways to get to that goal without that involvement? Maybe framing some of those things could be helpful, but I can already imagine that when that topic will be broached anywhere, you're going to have a lot of people with really strong feelings about it, and some of that emotionality can get in the way of the PDSA cycle, can get in the way of doing this work. Anticipating that is a huge barrier as well. I think it would be important to identify what the goals are and have an open mind about what the solutions might be. Exactly. It may look like making adjustments to how the system looks like now, it may look like redesigning everything from the ground up. Having an open mind and trying to be objective and help with that. Yeah. One thing that came up for me hearing about your idea, which I think is obviously a really important topic, is you were talking about getting the perspectives of our patients as the most important stakeholders. I didn't talk about this with my project, but my initial idea involved interviewing our patients, our clients, about their experiences accessing substance use treatment within our clinic. Ultimately, basically, I learned that to interview patients would be a really arduous IRB process, which maybe is a QI project in its own right. The fact that we don't do certain projects because we have some protections in place for folks which have these historical reasons which are really important, but they prevent us from having certain conversations. Even conducting research can be a really lengthy process. How can we break down some barriers and leave up the barriers that are there for good reasons so that we're asking the right questions to the right people? Awesome. Then last but not least, the large group in the back. I am the Chief Medical Officer for the Oregon State Hospital. A recent case came up where one of our patients, who is shortly going to be discharging to the single most populous county in Oregon, which is called Nova, which is Portland, Oregon. The County Mental Health Program asked us to take her off clozapine, which she had been stabilized on because they did not keep her on clozapine in the community, which seems like a pretty egregious thing for the biggest county in Oregon to be unable to support a patient on clozapine. Really, the question that I have is, what are the barriers? I can focus on our five most populous counties who send us most of our patients. We have 400 people in the hospital, many at a time, who have been sent to us for comprehensive restorations. What's keeping them from being able to support our patients once they leave our hospital, either in the jail or in the county or both, on a medication that, frankly, we don't choose first, but when we do choose it, it's usually because nothing else is working. I know that my in-house prescribers do not prescribe clozapine when they would otherwise would because they are convinced that patients will not be able to be maintained in the community. It really feels like the first step is asking county mental health programs, asking jails, asking CCOs, the Affordable Care Organizations, which are the payers in Oregon, what are the barriers? I would need to partner with my parent organization, the Health Authority. I would need the assistance of jail commanders and a lot of folks throughout the counties just to even ask the question. But I don't think we get anywhere near solutions until we do. Interestingly, despite the fact that they're extremely upset with us, judges and DAs would probably be allies there because if we are able to maintain treatment that keeps people housed and in treatment and out of jails and out of our hospital and safer for the community, I can imagine a lot more connection would be eager to help us. Yeah, that's a multi-sector, multi-system QI project and one that I think has a lot of societal, I think, connections. The fact that this one patient is like patient zero, but this is happening all around and clozapine is one medication, but I can imagine that could be any of the numbers of things. I appreciate the fact that you're actually thinking about maybe undertaking this because one of the things I can imagine are people, when it comes to this kind of work, it seems overwhelming. I think a large part of what QI can be can feel like, oh my gosh, these are such big existential questions. Is this tangible? Is this bigger than I can chew? But are there elements of this that actually could be done? We may not fix societal ills with this QI project, but we can make a difference in one element, one domain that can actually help our patients. But this is a really, I'm so happy to get to actually did this exercise and wrangle with these challenging topics. And because we're close to time, I don't want to keep going, but we can talk about this for a very long time. But if you just want to stick around, have smaller, informal discussions, we're happy to stick around for that. But this has been a really rich exercise for us, and I hope it was for you as well. And we are so grateful that you came to spend the morning with us and to engage with this discussion. Thank you so much. Thank you.

Quality Improvement

Public Psychiatry

UCSF

mental health services

equity-driven initiatives

substance use disorders

opioid epidemic

provider training

systemic barriers

stakeholder involvement

data-driven decision-making