kick off the morning with some measurement-based care talk. So you should all know that there's an expectation that everybody participate in the discussion at the end. So start getting your ideas in your head and your questions because we really want a lot of participation. Cool. I'm Jess Thackerberry. I'm a psychiatrist at UCSD. I'm the head of our tele-mental health program and the informatics program within the Department of Psychiatry at UCSD. So I'm gonna let our panelists introduce themselves. I'm gonna be moderating for us today. Hi, good morning, everyone. My name is Andres Schneeberger. I'm also at UCSD where I'm the medical director of the outpatient psychiatry department La Jolla. And originally, I'm from Switzerland. I joined UCSD like a year ago. I was working at a psychiatric network, also head of a clinic there. And I've been trying to deal with measurement-based care in two different countries now. So I'm excited about discussing everything with you. Yeah, good morning. My name is Rao Gupta. I'm a psychiatrist and medical director of a psychiatric clinic in Switzerland. And I'm very excited to be here today to share some of the things that we do in Switzerland and compare them to what you're doing here. So I'm really excited, good to be here. And I'm Eric Vanderlip. I'm a physician, a psychiatrist, and I'm speaking in my capacity as co-chair of a work group in the Council on Quality Care on implementation of measurement-based care. We have a white paper that's coming out to help support small group and solo practitioners in the implementation of measurement-based care. And I have some experience with this as former chief medical officer of a mental health startup or healthcare startup in general called ZoomCare that started about five, six years ago. Thank you. Thank you. All right, we're gonna go ahead and get started. Who here is from not the US? A couple of folks. Excellent. Oh, okay, nevermind then, we're done. Thank you. So which one is it? Question time. There you go. Well, sorry for that. Again, we're trying to be very tech-savvy here. Yeah, we need to disclose that we have no conflict of interest. So we don't own any measurement-based care companies or something. There we go. All right. So I'm gonna be talking about integrating information system clinical work and communications. And this talk is more about giving you like a little input because we want to have the discussion going later about the why and what we're doing. Why measurement-based care? What is measurement-based care? I'm sure most of you know that already, but just to give you like a little input again, treatment outcomes and patient satisfaction. And then I think the most important part, the information services and Dr. Thackerberry's lead of IS at UCSD, the Department of Psychiatry, will chime in if I'm not saying the right things. And then, of course, engagement of patient and staff. In this whole process of measurement-based care. And that's where we really want your input and how you're doing it. I think there's no like definite recipe on how to do it. We need to discuss that together. So why measurement-based care? An immediate way for patients and providers to understand symptomatology at any given time. Symptomatology at any given moment. That could be like a sentence of why we're doing measurement-based care. I know there have been a lot of talks about measurement-based care and there's much more to it. But I think what we have to discuss and what have been discussed yesterday is that there's a lot of barriers to implementing measurement-based care. Despite the fact that measurement-based care is a good thing and that we know from studies now that it is working and that this communication improves the outcomes of patient care. So the barriers are that the patient might need time to complete the measures. Providers have maybe different attitudes. The standardized measures are not better than my clinical judgment. I've been practicing psychiatry for 20 years and this is like not really needed. I have the experience. I don't think that's gonna help me in the process. What is measurement-based care? Again, it's the systematic evaluation of patient symptoms before or during an encounter to inform behavioral health treatment. And I think that's what we're gonna be discussing. Also, there's a lot of questions that come up when you talk about this. Which instruments do you use? How do you do this whole thing? The evidence-based practice of using systematic and routine assessments of the patient's perspective through patient-reported progress and outcome such as symptoms and functioning throughout their course of mental and behavioral care to inform treatment decisions and engage patients in their treatment. Treatment outcomes and patient satisfaction, quality of patient experiences associated with high satisfaction which in turn is positively associated with improved treatment outcomes. So we know that. So the question is, and that's what we wanna know from you guys, why don't we do it? I think Eric told us yesterday that's like 20% of people use measurement-based care in their practice. So that's the question. How can we improve that? What's the difference between using it or just working in a clinical setting without structured measures? Treatment outcomes and patient satisfaction, including patients in the decision-making process led to a slightly better treatment outcome and lower dropout rates when receiving the preferred treatment. So we know that. That's one of the reasons that we should be doing measurement-based care. But again, we have barriers. Patient reported outcomes, for example, satisfaction survey might affect the relationship with the practitioner. And we know patient relationship is one of the most central aspects of our treatment. So a patient might be scared that it is gonna impact the way we see them or the other way around. And the provider might think that there's a lack of clarity on the clinical utility. Like what does the scale even measure? Does it measure depression? Or what is it if I provide a measure to a person with personality disorder who always scores it super high? And is that person chronically depressed or not? These are very important points that need to be discussed. So now we're coming to the matrix of it all, information services and measurement-based care. I think that's a very central point. We can use paper-pencil measures. We can use other ways of implementing measurement-based care. But I think nowadays it is, and it's gonna be crucial, how do we integrate measurement-based care in our electronic medical records? How do we present the data? How do we work with the data? How does the provider access it and is able to show it to the patient? So we've learned from people who are specialists in information science that not specifically to measurement-based care, but in general, transferring information in an electronic form needs to follow these five rules, the five R's. The right information to the right person in the right intervention format through the right channel at the right time. This is like a general rule to how information scientists want us to transfer the information to people. And if we do it that way, our chances for success are higher. So what does that mean? And this is just an overview. The right information. So here I listed a few things. So the right information. So here I listed a few points that could be needed in order to have the right information. First of all, it has to be brief. The shorter, the better, because if you transfer too much information, people are overloaded. We're constantly receiving all this information day to day, so we need to make it short and sweet. It obviously needs to be cost-effective. It needs to be available and accessible. It needs to be valid and reliable. It needs to be sensitive to change. We need to be able to benchmark it. It needs to be relevant to clinicians and to clients. It could be used as performance indicator. For reimbursement, it needs to be possible to integrate it in electronic healthcare records and has to have key points of high-priority data elements. So these are a lot of points that you have to consider, but I think it's just good to at least have an overview in what way we want to present the information. To the right person. Here, one of the possibilities to look at it is we have this patient-provider dyad, and this patient-provider dyad can vary depending on how the value of MBC, of measurement-based care, is viewed, whether the clinician values it more, whether the patient values it more, or if there's a mix. And I think depending on that, and we need to know our clinic, we need to know our population to implement it. So this is not like a static process. It's like a process that needs to be in motion, and we need to adjust our system to it. Obviously, the right person is also based on the visit type, medication management, for example, psychotherapy case management. Psychotherapists have also worked on these aspects quite a lot of measurement-based care on how to inform their treatment. Then the right information format. There is a lot of things that can be looked at. I know we've been talking mostly about PHQ-9, GAD-7, things like that. But in the end, if you think about it, measurement-based care is not limited to that. I mean, we need to also step away from that idea that it's just some instruments that we're looking at. It's basically we're trying to operationalize our whole treatment and look at which parts of the treatment, the diagnostic part, the therapeutic part, the outcome part, is important to us for what we want to deliver to the patient, and then try to integrate it into measurement-based care. So this can be from personal data about aims and risks, service preferences, intervention, progress mechanism, and contextual data. It can be all of that. It's up to us to make a concept of how we're implementing measurement-based care and how it's gonna be, and what's gonna be measuring, actually. Then it has to go through the right channel. We all know that. Numbers, simple line, graph, directionality, interpretive guidance, use of colors, treatment milestones. I think this is really, it seems very banal, very simple, but it actually is important to look at because if it doesn't come nicely packaged, people might not use it. And it has to be at the right time. I think that's also a very important part. Speed is everything. People don't like to wait in our modern world. It has to be there when we need it, when the provider needs it, when you can show it to people. It has to kind of anticipate the need of the providers, deliver it in real time, fit it into the user's workflow, and I think that's the important aspect, that it needs to be, it doesn't have, it cannot be an additional burden. It should be something that's helping us in the process, and little things can make a big difference in that sense. Good, let's talk about the last point. And again, this is just like a little primer so we can go into the discussion later. Engagement of patients and staff. I'm sure everybody here who has been implementing measurement-based care is aware that this is not always that easy. Patients, why we're doing, try to engage them by information, provide a strong rationale for MBC, motivational enhancements, like we have to do this, this is good. And it's important to transmit the message that it's also for them, it's helpful. They gain control over their own treatment. They're engaged in their own treatment if they see what is happening with them. It's important to discuss it, obviously, every time a measure is administered, and have it be part of the discussion of measurement-based care. And for example, use graphs to show progress to patients. But there's barriers to it. Administrative burden, time, you know all that. When do they fill it out? Should they fill it out at home, on their phone? Should they come in 15 minutes early, fill it out on a tablet, or even on paper, and then be put in, so there's a lot of barriers that might influence this whole process. Then engaging clinical staff, yeah, again, which selections, which measures are you selecting? That's the important part. It should be, it can't be just one fits all. It has to be part of your clinical practice. Which part, what do you need for your practice, for your clinic, and you should select it accordingly. Obviously, accessing data has to be easy, again. And when you present it, ensuring that the measures are available when they use it, it should not be an additional burden to them. Engaging patients in measurement-based care, adapting platform if needed. It has to be like a living process. That's the important part. It cannot be like you've started the measurement-based care once, and then that's it. We know now, post-COVID, telepsychiatry is an important part, and dealing with suicidal risk, for example, is also like the challenges that we're facing with this. There's barriers, provider need to provide resources for training, staff turnover, leadership support. It needs to be, the whole clinic leadership needs to be part of it. They need to be in support, otherwise it's not really gonna work. Well, in summary, measurement-based care improves patient satisfaction and treatment outcomes, so we really want to do this from a rational point of view. If you are implementing it, if you're working with your information services, focus on the five R's. This is a path to success for the system, and it's important to engage patients and staff. Thank you for your attention. Thank you. All right. There we go. Yeah. So I have the pleasure of talking to you about the Swiss way of measurement-based care. So we were asked to disclose any conflict of interest, and I just came up with one, because I've been assigned German citizenship up in birth, and now I identify as Swiss, and the relationship between Switzerland and Germany is a little bit like the relationship between the States and Canada. So there might be a little conflict of interest in that. But yeah, so just give me an impression from where we come from now or where we are in Switzerland. We have to keep up with the beaches that Andrew showed on his last slide. So that's what we compare to in the mountains. You see the Matterhorn. And so you can understand what we are doing in Switzerland. Maybe it is important that I give you a quick run-up on what our situation is like, because you have to keep in mind Switzerland is a tiny, tiny, tiny country in the middle of Europe. So with a population just about the size of New York City, really, it is there in a very diverse situation. There are four language populations in that country, so everything has to be in four languages, making things a little more difficult. There is a lot of population coming from outside the state. About a quarter of the population is actually a migrant population. The average life expectancy in Switzerland is 83.6 years, so people do get really old. And the population has doubled in the last century. So you have an idea what a country Switzerland is. It is really a rich country, you have to say that. It has a very high domestic product. There is a low unemployment rate right now, around 2%. It has a high level of education with a lot of educational institutions, international institutions in the country. There is a low crime rate, so at the place where I live you can still leave your car unlocked and things like that, things that probably in New York you wouldn't do. There is a high rate of renewable energy that people are using, water energy and stuff like that. And tourism, as you have seen before, plays a very important and crucial role in that country. The Swiss healthcare system is pretty unique, because on the one hand you have a universal coverage for everybody, you have an insurance system that requires everybody to be insured, yet there is also a part where people can choose what kind of care they want to have. So they can do added insurances on top of the mandatory health insurance that is in the system. The system is very costly, there are very high costs in the system, and healthcare funding is actually split between what people pay as their insurances and what is subsidised by the government and out of taxes. Although it is a very comprehensive system of coverage with a lot of different services that are offered in that system. Care is very accessible and there is a very high quality of care, and to maintain that high quality of care, the Swiss have done a lot of things to actually measure care quality, and that's what we are talking about today. We don't have to go into the details of that slide, it just shows that the healthcare system is quite complicated, so there are a lot of different levels that play into the system in Switzerland, making it sometimes not too easy to navigate around. The health expenditure is just below the American health expenditure per capita, but as you can see here, it is the highest in Europe, and the number of hospital beds that you see is also on the high end of the European scale, as you can see on that scale here. The number of psychiatrists is also almost by far the highest in Europe, and still there seems to be a very large demand for psychiatric services, so there are still waiting times to get into therapy. At the moment for outpatient care, that will probably average at about six to eight weeks, I would say. So now, getting to the way of measurement-based care, I was very impressed yesterday to hear a couple of sessions on measurement-based care and what you're doing here in the States, because what I really like about that is that you talk about it. As you will see later on, we do have a couple of things that we do in Switzerland, but it comes more from an administrative level, and it's not so much that we do actually make the link into the clinical process, although we have beautiful data, as you will see later, but we don't use it clinically as much as we could. So I think in the States, you're much further at that discussion, and if I see how many sessions you have at that Congress, it's really good that you get into the discussion of measurement-based care and how you can use it. So there's a Swiss National Association for Quality Development in Hospital and Clinics that was actually founded in 2009, and it is an association of all main players in the health care system. So you have the health care providers, you have the insurers, you have government bodies all coming together in that association, and they're doing a lot of things. They do comparison reports, they have more than 380 hospital and clinics actually working together in that association, there are a lot of training courses that they're offering through that, and a lot of specialists actually working on the field of quality in medicine. And in psychiatry, what do we do? We actually do measurements to the burden of symptoms and to the restrictive measures of liberty that we apply in psychiatry. So today I'm going to focus on the burden of symptoms, because that's really that field of measurement-based care that we're talking about. So we have different quality indicators, I'm going to get to what they are a little later, and they're used for nationally standardized measurements, so that'll happen in all of the state of Switzerland, they will be using the same measures at the same time for adequate patients, for all patients. There is transparent information and publication of that data, and through that there is a good comparability of the results throughout the nation. That should trigger improvement process in hospitals and clinics, and it should also ease the dialogue between providers and payers, because if you have that data there's a different base that you can talk about care. And it should also help to bring international comparability in that. It shouldn't be used to actually allocate resources, so it shouldn't be a system of enabling pay for performance, although there are some tendencies in that, so that is one of the barriers that you actually have in that system. It shouldn't be used as a measure for sanctions against individual hospitals with poor measurement results, and it shouldn't be used as a rating or ranking scale for the hospitals. Why are they meaningful, those quality indicators? They are an instrument for monitoring. They are an instrument for evaluating important functions, and they could be used to sharpen one's view of potential problem areas, if you see in which areas there are deficits. It is, however, not a direct measure of quality all over. So what do we do with the individual patient? We have this measurement plan in adult psychiatry, like you see here. So we will do a patient satisfaction survey, which is administered through a short questionnaire in paper form still. As for the symptomatology, we have two measures. We have a self-rating scale, which is a brief symptoms inventory that you're probably all familiar with, and we have a third-party rating scale, which is the HONOS, that's the Health of the Nation Outcome Scale, which comes from the UK and comprises about 12 items. And the freedom-limiting measures are also measured through a special questionnaire. So that's the Health of the Nation Outcome Scale, which was developed for primary care settings, a little like the PHQ-9, which was originally designed for primary care. So the HONOS sort of, yeah, goes in that direction, and it has different subscales, and this one here for mental disorders has 12 items. They can be scored with values from 0 to 4, and they have some subscales on that. The brief symptom inventory, so the short form of the symptom checklist with 90 items, you have the brief symptom inventory, which has 53 items, showing you that, depending on the state the patient is in, that can be quite challenging. So if you're giving a patient who is in severe distress, a rating scale or a survey with 53 questions, that puts quite a lot of pressure on the patient. The results are then given to each individual hospital, and that's like the overview for the whole country. Those are the results of 2020. There was a total number of 83,000 patients included in the measurements. The average length of stay was 29.6 days. There were 50.7% females and 49.3 males included. About 33% had affective disorders, 20% addiction disorders, and about 16% had disorders from the schizophrenia section. And all patients are actually measured up in admission and on discharge, and that's the overall results. So in the external assessment that is done through the HONUS rating scale, you had a starting point of around 19, going down to 11.5 or 11.4. And on the self-assessment, you had a reduction from around 69 to around 38. So in both categories or both dimensions, there was quite a reduction of the burden of symptoms over the course of the treatment. So how does that actually work in the individual hospital? You have to include every patient that you see in the inpatient, so that's only for inpatient setting at this point of time. You measure at admission and on discharge, and you use the self-rating brief symptom inventory with 53 items and the professional rating on the Health of the Nation outcome scale with 12 items. And that is actually done as a team effort. It's not just a single-person effort, the professional rating. It should be a multidisciplinary effort. The challenges are quite a few, because you have to build that into your daily routine, so it has to actually be implemented in the hospital's routine. And the rating should be a multidisciplinary effort on both points, so also on admission and on discharge. And it is crucial to actually motivate patients and staff, that's what we saw before. So people have to actually believe that there is something important that they are doing and that it's worthwhile doing that. And what we found, another challenge, is to manage the dropout, because there are quite a few dropouts, and you have to be aware of that and manage that so you still receive the necessary data. So those are the results, what they look like for an individual hospital. The red sign is actually the results for one of our hospitals, where you see it's slightly above average on the professional rating scale. And here are the results for the patient-reported rating scale, showing also that it is pretty much an average of what the other hospitals reported. So to give you a quick summary, in Switzerland they have established a compulsory nationwide system of measurement-based care, where there is a measurement in each psychiatric hospital, on every patient, on admission and on discharge. All results are published, and it builds a routine of regularly measuring patient state through instruments like I've shown you. The challenging thing is that it actually anchors the quality control in patient care settings. The challenging thing is that it is very crucial to choose the relevant indicators, to find the routine to implement that into your practice, so to make it part of your daily practice, and to obtain the comprehensive data. And I think the most important point, that's what I want to stress and maybe discuss with you later on, is that how can you actually use that data later on for patient care? Because I think that's the point where you've made big advances, and where you have established good practice. And we still have to do that, because now we have that data on every patient that we see. We see those 80,000 sets of data, but it doesn't really reach out into the patient's life as far as consequences for their therapy. So that will be, yeah, a challenging part of that. Thank you very much. Wonderful. It's great. Nice to meet you all. Happy morning. I don't have an accent, so I'm going to do my best to speak about international implementation of measurement-based care, actually from a for-profit health care startup perspective in the Pacific Northwest. I think the unique thing about this session, in my mind, is the contrast between a nationalized system, inpatient, hospital-based, and an ambulatory, for-profit, non-nationalized system in the US. So the contrast, I think, will come out as I talk through several of these experiences. So I used to be chief medical officer of a company called ZoomCare. ZoomCare is an on-demand, like Amazon or Netflix, health care multi-specialty retail clinic based out of Portland, Oregon, spread now to Seattle and now in Idaho and Colorado. And ZoomCare operated about 60 clinics, everything from emergency room to dentistry at one point, physical therapy and behavioral health, as well as primary care. In-person and virtual care was provided at ZoomCare. They accepted private commercial insurance, no public insurance, but most contracted with most plans. It was built uniquely, I think, in health care for a niche audience, millennials, and not really for everybody. And that's an interesting tact from a business perspective. Nonetheless, the focus audience was millennials. They owned their own providers. Nobody was contracted. They had their own provider workforce internally in-house. They had their own tech and their own electronic medical record, as well as an app and other things. All of it was in-house, the tech. And then when I joined ZoomCare, originally we had our own health care insurance on the marketplace. So you could purchase ZoomCare insurance, go to a ZoomCare clinic, see a ZoomCare provider, start your care on your app. We had a pharmacy, labs, even CTs in our emergency department. So it was a very interesting health care system and compelling for me as a young health services person to go join this system and try to build out their behavioral health program initially and then becoming their chief medical officer. So I came into ZoomCare to build their behavioral health program. And I apologize for the blandness of the slides, but I'll just tell you what happened. Originally, I arrived and there were a couple of nurse practitioners providing behavioral health care. The wait was very long to get in to see anybody, and there was no aspect of measurement-based care there. And so we just took paper and pencil and formulated a process for everybody to get a PHQ-9, a GAD-7, a CIDI, which I kind of like as a bipolar disorder screen, and an ASRS during every initial mental health appointment. And then depending on the constellation of their symptoms and diagnoses at every follow-up appointment, they would get a PHQ-9 and a GAD-7 and potentially an ASRS to look at ADHD symptomatology burden. And it was just paper and pencil. Like, that was it. We had a check-in associate who would see them. Most of the clinic visits were in person at that time. This was pre-pandemic. It was easy to schedule and find mental health visits because we changed up some other workflows and other things. So it was immediately on demand. People could get in within a day or less to see mental health providers. We'd give them this battery of instruments. It'd take variably five to seven minutes into a 30-minute appointment. Then we would scan that, literally, into the electronic medical record. And the associate at the front would chat the provider in the back so that they could review the scanned PHQ-9, et cetera, before they went out and greeted the patient and took them back for their appointment. And that was literally our workflow that we had for a couple of years. The scores were tracked manually. So if you wanted to go back and review the scores, you had to literally look at the PDFs and see where they were. But there was a timestamp. It was just one little tab on a click in the EMR. And it wasn't that hard to kind of trend it out. You could kind of do it. It didn't take an exorbitantly long time. And the pros were that it was almost obligatory or mandatory participation. You know, the patient would show up. We'd capture the data right there. And almost every visit had this data. And that was really nice. It was relatively quick. These measures are brief. They're patient-reported. And though it could eat into a substantial portion of your clinic visit, there would be times when I'd have to just go out and grab the patient because we're 10 minutes into the time. It was their time. And they're still hemming and hawing over what item they should select in the PHQ-9 for their symptom burden. I'd say, let's just come on back and talk. So that was a pro, was that it was generally relatively pretty quick. And then there was room for error. And different practitioners may or may not be better about enforcing this workflow in their own permutation of a ZoomCare clinic. But that being said, ZoomCare is a very standardized system. And we took a lot of pride in the standardization between providers. That was one of the unique aspects of it. And so there was room for error. Human error, scores not being scanned, people getting lazy and not doing them. But I'd say that was actually fairly minimal because we had a very modular, team-based approach to care. Okay. And then this pandemic thing hit. And we pivoted rapidly to telemedicine. And suddenly this workflow was majorly disrupted. People weren't coming in to in-person visits anymore. And even though we had our own tech stack, it was hard for us to prioritize making a PHQ-9 part of the app and the patient portal. That was an investment. And we had a lot of other distracting investments that we were trying to make at the time, like, I don't know, securing N95 masks and all these other things that were going on. You remember the pandemic? It was bananas, right? And so we went all virtual. We tried to implement the PHQ-9 solely. We had some of our tech team working on a user-centered design interface that would get PHQ-9 data from behavioral health patients coming for behavioral health visits. And basically we disaggregated the process of getting a PHQ-9 from check-in for a behavioral health appointment, making it so that persons who had a ZoomCare account could log in and complete their PHQ-9 virtually anytime they wanted to. And that was kind of nice. It was very person-centered. But it also made it more difficult to ensure that the PHQ-9 was returned to us as a clinician in a timely manner and in the format of making decisions for the person at the time. And so that was a challenge. And our compliance team kind of had a fit about the ninth item, suicidal ideation, on the PHQ-9. But we pushed through that. And that was a barrier for a brief hot second. And then I said, no, I'm not going to tolerate any of this kind of legal pushback about asking about suicidal ideation and what we're going to do about it. We just have to ask about it. So that was just, you have to have good clinical leadership to assuage their concerns. But the pro of the virtual stuff was that it was scalable, definitely. It was easier. It was less hands-on, less room for error. It was more patient-centered. And obviously, we could trend then the digital results of their PHQ-9 scores that we couldn't do as easily when it's all scanned in in pixels on a PDF, clearly less manual. But the big challenge for us then at that point was an issue with data integrity. Like we were just not getting as much PHQ-9 data, and the data wasn't as meaningfully useful because it could be stale, or the patient hadn't completed it prior to the virtual visit. And that was more of a challenge. So I know this has been a little bit brief, but in reviewing this as a case study of implementation of measurement-based care in a for-profit, tech-enabled health care startup in the U.S., I have some takeaways. It is a lot of work to design for a user and to engage them and to get them to engage in participating in measurement-based care, to design for them, to make it easy for them to participate and complete a PHQ-9. It just takes a lot of work, and that takes a lot of commitment, not only from the clinical team who understand. I mean, you have to get the clinical team on board, but even if you have that, that is insufficient. You have to have your tech partners and your product partners on board with the rationale for why we need this. And as a health system, even Zoom care, which was relatively small, you know, I needed to do a lot of convincing amongst even executive team that this was a major priority for us, that depression was something that was important enough to invest other things and resources towards when, frankly, we were losing money in the pandemic and we had a bunch of other things that were important to deal with. And it became difficult for me to advocate sometimes for better measurement-based care in the virtual space because of all these competing interests in a rapidly moving for-profit, you know, healthcare startup. So that was one thing. Zoom care was started in 2006, 2007, I think, before software engineers knew what the hell they were doing, or at least the ones at Zoom care didn't really know what the hell they were doing in 2006 and 2007, I guess, because we had a ton of tech debt. And if you all aren't familiar what that term means, it's like the burden of legacy tech products and code and security and stability in the platform that was just very burdensome. And Zoom care had been towing that along for years, and I don't know if anybody knows Zoom care goes to Zoom care, and I don't work there anymore, so I feel free to say this, but tech debt was a major burden for us, and I had underestimated the burden that posed. And that's just a random thing, but you might think, looking from the outside, that a for-profit healthcare startup that's tech-enabled and has its own tech would just be able to add a PHQ-9, it's a couple of damn questions, and some stereo buttons that the person picks, and a simple calculation. It can't be that hard. Turns out it's actually fairly complicated. And to design it well for really good engagement is a lot harder than it looks on the surface. And so that was a learning process for me, to understand the priorities of stabilization in the platform and security from real security threats and other things had to, by necessity, be prioritized against some of these other more innovative product ideas that we had. And then I think this has been a takeaway from some of the other sessions we've already had, but yoking the behavior of completing like a measurement-based outcome or a patient-reported outcome like a PHQ-9-2, a must-have for the patient, greatly improves data integrity and the chances that they'll actually complete the thing, because there's a lot of disengagement out there and a lot of people who just fall off the map or just don't care or don't see the utility of it, even if they get to graph their own PHQ-9 score out over time and they see it themselves. That can help, but there's still a lot of disengagement. What do you know? It's depression you're working with, and people just have a lack of motivation and follow-through, and so you have to make it as stupid simple and easy and compelling as possible without being overly coercive. That's my reflections. Thank you. Okay, so it's discussion time that I warned you all about. So we want to try to make this as engaging as possible with all of you. I kind of put some discussion topics up here just in case we couldn't come up with any, but we definitely want to hear about your struggles and your accomplishments with measurement-based care and engage in a discussion now. So these are just some questions that we came up with. So if your institution is utilizing measurement-based care, what methods have you used to successfully engage clinicians, staff, and patients? I know that's at the top of my priority list as the IS lead for psychiatry at UCSD. Getting people engaged and wanting to do it is like 85% of the battle. And how have your institutions overcome regulatory barriers? I know Eric mentioned the PHQ-9 ninth question being particularly bothersome to compliance. We definitely struggled with that at UCSD with rolling out measurement-based care. If your institution were to implement measurement-based care today, do you have any ideas for creative workflows to make this appeal to clinicians? What would you want to see happen in order for you to utilize measures in your visits with patients? How would things be easier for you if you engaged in that right now? What challenges have you come across with utilizing scales to make or change clinical decisions? Are there mistakes that we can learn from? Can we engage in a discussion of pitfalls and mistakes made? And if you're not currently utilizing measurement-based care, what reservations do you have? What issues do you have with implementing these measures in a day-to-day basis? Do you have any reservations on that? So yeah, if people have things to say, please go to the microphone so they can be recorded because this is recorded. »» So hi, everyone. And thanks so much for this. My name is Dave Kroll and I'm based in Boston. And actually my interest in this is largely outpatient because outpatient is the world where I'm really trying to kind of implement this the most. I apologize I missed the very beginning of the first talk so this might have been mentioned at some point. But a lot of us when we think about like what are the rules or sort of like kind of basic tenets of measurement-based care, a lot of us go back to that like 2017 Fortney article that kind of lists like here are the best practices. One of them that I kind of keep going back to is this idea that it shouldn't impose a barrier or shouldn't impose a burden on clinicians to administer. And I'm increasingly wondering if maybe we should actually walk back that tenet. Because effectively what I think that means in the real world is that clinicians should be able to rely on the system to generate this information for us. Meaning that when patients check in for their virtual appointment, the scale is given, the patient does it and there it is. And if it's not there at the time of the visit, well, you know, we're not going to do anything about it. You know, in my practice, the one area where I actually really, really reliably get a PHQ-9 every single time is when I'm embedded in primary care. In primary care we have MAs whose job is to get vital signs in every patient, they get screening on every patient. And so when I moved in there, the administration said, hey, can you just start doing PHQ-9s on every patient if there's a BH visit? And they're like, sure. So it's done on pencil and paper, they get it every time, it gets entered into the system by the MA after they do the rooming and that works great. But it requires a human being to kind of sit there with the patient and go through it and it takes time. Whereas in the absence of an MA, and by the way, one of the things that's different about my clinic is that mine's walk-in based, so people aren't showing up for a specific appointment. So there's no, none of that sort of tension of, do I start the appointment late to allow them to do this? It's just kind of whenever they're done, they can see me. But in the absence of that kind of structure, what I'm finding is, I feel like if, I'm starting to increasingly tell my clinicians, if you don't have that information, I actually think that you should spend some time in the visit actually getting that PHQ-9 filled out or whatever the screening skill it is. And there's resistance to this because then it's like, well, then that kind of makes more work for me if I'm having to kind of do this evaluation. So it's probably a little bit easier to make that argument if it's a time-based service. So for example, if it's a 50-minute therapy visit, then it's less of a burden to say we're going to spend five minutes doing this, as opposed to it's a 20 or 30-minute psychopharm visit where it may or may not actually be time-based. But I'm starting to increasingly think that the more we actually make this sort of central to the provider's responsibility to actually get this during the visit, even if it means taking time away from some other part of the visit, and I don't want to say punish the patient for not doing it, but really sort of demonstrate to them that if they don't get this done ahead of time, they have to do it, you know, in the visit. Just like when I go to the airport, I'd much rather check in online and fill out all the forms and sort of check boxes so I don't have to go to the airport and wait in line at the kiosk to actually do it while I'm there, right? But I think it is something that I'm increasingly sort of wondering if we should actually be asking our providers to say, it's actually worth putting a little bit of a burden on you to get this done so that we're not relying on the system, and I wonder how folks would feel about that. That's a really good question. I mean, personally, I find it very useful when I go through a PHQ-9 questionnaire with patients when I see them directly in a private practice that I have right now. You can talk about symptoms, we can talk about what the scale means. I find it very useful, personally. I think providers feel burdened by a lot of other requirements, and it can be hard to have mandates on providers. And I think you're balancing very aptly pragmatism versus idealism. There's this idea that this data could be acquired outside of the visit time, and that would allow us more time for shared decision-making and collaboration and treatment planning and discussion about that, because we've already preloaded the visit with an understanding of where their symptom burden is. And that's ideal, but sometimes just not pragmatic. And I think if you really believe in measurement-based care, and you've done it, and you've done the exercise, that's what's convinced me that it's very useful in the course of visits, but it's only through my experience that I've come to understand the utility of it and feel rather naked if I don't have it in my practice. I think one thing to keep in mind is the idea that the physicians shouldn't have to put in the effort to do it is kind of multilevel. Most providers would be willing to put in the intellectual effort to gather the data. We want to know what our patient's symptoms are and how they're experiencing life right now. We want to know if they're improving, if they're not improving. We want to know these things. It's really a matter of, and you'll hear from physicians saying, like, oh, it's another thing I have to do in the EHR. It's another, like, it's more clicks I have to then enter the data. And so I think from an IS perspective, we try to look for ways to engage clinicians in participating in data collection, but doing it in a way that doesn't require kind of manual entry, doesn't require, you know, more clicks, more time in the EHR so that you can engage with your patient more meaningfully. And I don't know if you do this, Dave, but I actually just write in my HPI, PHQ-9 symptoms reviewed and confirmed, and then I put in the score. And I don't document, like, their sleep and their appetite and all this other stuff. I just, like, say that. So to me, and I might be cheating a little bit from a billing perspective or something, but it lowers my documentation burden. And I think that could be an attack that you could use. Hi. My name's Jen Stanley. I do community mental health in Michigan. It's probably been a decade since I saw someone with pure major depression. They adjust R0 in our patient population. So I kind of have a twofold question, comment. Does anyone know of a reasonably short scale for psychosis or for borderline personality disorder? And how do you utilize measurement-based care in a population that often, A, does not want to be there at all? So anything that increases their burden means they are less likely to show up. And B, that often has very little insight. Yes, absolutely. There are some folks who have psychotic symptoms and are very aware of those symptoms, and it's very bothersome to them. And there are other folks that you're talking to in your visit who are overtly psychotic, actively responding. And they're telling you, nope, not hearing anything, not seeing anything. They're telling you about how worried they are about everything, and everyone's attacking them. And then you ask them, do you feel like anyone is trying to harm you? Nope. And you're like, oh, OK. So how do you integrate management-based care into that kind of practice, recognizing that we can't fire any patients. We have to keep everyone. There is no way to say, well, if you don't do this, I won't see you anymore. Actually, they'd be happy if I did that. Yeah. I don't have an answer. I will not monopolize the entirety of this panel. Yet, I do have many thoughts on this. You should just give everybody a PHQ-9. That is what sometimes the state asks us to do. Yeah, if they don't complete it well the first time, give it back to them and have them do it again. There's actually a surprising number of measures in the marketplace. There's more and more measures coming out. I will fully admit the PHQ-9 does not accurately portray everybody's exact burden of symptomatology. It runs the risk of over-indexing on symptoms rather than function. And there are no readily available patient-reported outcome measures for psychosis. So when you develop one next week, please let me know. We will take it and run with it. I think there is a challenge. And just like bringing it back to the five-hours discussion, the measure needs to be brief. Patient-reported measures are better in general when we can do them than clinician-reported measures. But there's some elements of patient care where they don't display insight to their true symptomatology where a clinician-reported outcome may be better. And I don't think we have a perfect answer for what the hell we're doing in psychiatry. Most of what we do is syndromic-based stuff off of the DSM-5 that's got its inherent flaws. But we're getting better, and someday we'll have better measures to replace the measures we have today—biomarkers, other things like the rest of medicine has—but we're just not there yet. So I'm a little zen about it. And I will also say that in ambulatory practice, the combo of a GAD-7 plus a PHQ-9 is just very interesting, even with patients with borderline personality disorder. And I don't have a great measure that I can recommend to you for borderline, but I think there's probably one out there somewhere. And there's probably a good one. And we have a framework coming out with this white paper from the workgroup on measurement-based care and implementation that tells you how to think about which measure to select and offers some guidance, at least, when you consider the different measures that you could pick from, because I'll tell you, there's a whole sea of them out there. And it's probably, rather than the exact measure, the process of doing something consistently to even get going is probably what we need to get used to as a field, more than which measure we want to select. And I'm a little agnostic to the measure. I'm more pro about just the process of doing something. Because right now, our gut just doesn't work. And that's not fair to patients. I think we really need to look at the whole process and try to operationalize it and specific to your clinic in that sense. And, I mean, the Swiss way is clinician-administered, a part is, and the way they try to be more objective about it is, as you said, like that it's a team approach, it's not just one person rating the patient. Yeah, but really, I mean, I share your concerns, because in the model that we are doing, that's where we produce a lot of dropouts, when patients just can't do it. I mean, not only psychotic patients, also the elderly in some dementia, so they just can't do the 53-item survey. So that produces dropouts, yes. But still, I'm with you, Eric. We have that other data, and that's really good. So it's an evolving process, I think. Thank you all. It's also always a really interesting discussion when their PHQ-9 is always 27, because then it's a discussion. No, no. It's always here. Right. That, too. Any time it doesn't comport with what I think is going on, it's always an interesting and rich clinical discussion. Yeah. And one of the things that I've been trying to stress to our health system is the importance of looking beyond a single number and looking at trends, which, for some reason, IAS really struggles with the fact that we need it graphed, as opposed to just popping the number up. Like, OK, it's 28 today. What was it last time? And making that as few clicks as possible to try to get to has been my huge... Anybody that knows me at UCSD is like, oh, yeah, she's still talking about this. Thank you for your talk. I'm coming from a little bit different population. I'm active duty military. And I guess, in some ways, we have a lot of resources thrown at behavioral health, as you can understand, probably, from a lot of the content that comes up on the media related to suicide and other things. But we have something called BHDP. It's the Behavioral Health Data Portal, which is a web-based series of scales, PHQ-9, GAD-7, a therapeutic scale, you know, PCL-5. It's... Everything's in there. And the patient comes to clinic. They're given a tablet. Imagine that. A tablet. They fill everything out before the clinic visit, ideally. And then, you know, you have these beautiful graphs that show you over time. So it's beautiful in that sense, if it works, because it doesn't always work because it's technology. But my question, or kind of my thought, is we moved to this because of something called the Defense Health Agency that's taking over and is pushing efficiency and reducing cost. And so they're using a lot of these outcome measures more to say, look at this hospital. You aren't able to get that dose response within six months. Or you know, this treatment facility is not as great as this one. So it becomes more of a marker of, you know, are your clinicians able to do their jobs effectively, rather than how are we helping patients and looking at this data? Which to me is concerning. And this has always been presented as, well, the civilian community is doing this. So we're doing this. Which to me, you know, after hearing this presentation, also, doesn't sound true. So I don't know what your thoughts are on that. Because I... You know, ultimately, as a clinician, it's great to see the data. It's great to have conversations about the data. But there's also, I guess, the yin and yang of this is that you can go too far into the data where then it almost becomes a barrier in some ways. So happy to hear your thoughts. That's definitely a risk. I have a question for another panelist on that. Because Rahul was saying that you're reluctant to look at these measures to judge hospital systems. Why? Pretty much the concerns that you're mentioning right there. Because it is an administrative sort of, yeah, put-on measure out of every... So everybody has to do it and they have to take part in that. And of course, people were scared, right, of those aspects. How do I... I showed you the slides that for every hospital, you actually get a benchmark. So I think we just don't have that culture yet of benchmarking our services. I mean, other specialties have that for a long time. If you look into surgery and their results, they've had that. But I think in psychiatry, we don't have that culture of dealing with that. I think one of the things that we've noticed as we've been rolling it out at UCSD is that, first, they were measuring... Or first, administration was looking at whether we were screening everyone. So you get the checkmark for screening everybody. And so that approached 100%, right? We can do this when tele-psychiatry is... You're doing your e-check-in. You can do all of those things. So now, it approaches 100%. Now what do we do? We need to continue to get better. So now, they're moving to improvement or remission. We had to push back from the psychiatry side on the term remission and what that means and what the expectation of remission is. So I think in some level... And I know that this is hard in a government, like a federal service. You have to have champions that understand what the purpose of measurement-based care is and the usefulness of it from a clinical side versus from an administrative side. And you have to kind of almost protect, like put some walls up and protect against that. Because what we don't want is the expectation to be set so high that nobody reaches it. And so that's the good thing about... One of the good things about this whole process is that it's fluid. It changes. So as we continue to improve, that target can change, you know. But it's a very important point if we discuss barriers, because I think we have like the rational aspect that we kind of all agree measurement-based care is good and it helps us versus like the emotional aspect. Are they going to be controlling me? Administration, are they going to be looking at my... Is that going to impact my salary or something like that? And we need to enter that discussion. Well, that's exactly what I want to talk about for a second. Because there is this nasty third rail, I'll call it the third, you know, the third rail of the subway where like if you step on it, you die. And people never want to step on the third rail. The nasty third rail underbelly of the idea, the concept of measurement-based care is pay for performance. Or, and there's three, I think of three main concerns that fall under this idea of getting measured on my outcomes. The first is that it's an existential threat to my quality of life. Like I could have my pay cut or I could lose my job because I'm not perceived as achieving similar outcomes to somebody else. I went through medical school. I had to do the cutthroat thing of trying to get an A and being a pre-med person. And I'm sick and tired of competition. I've made it now. I'm good to go. I want to sit back on my laurels and just see patients and help them feel better. That's all I wanted to do. And then go play golf. And I'm scared that if this starts to happen, I'm going to get compared to some of my peers and I could lose money. Or it may entice me, point number two, to cherry pick my own patient population. And then I want to just see those who are going to get better. Oh, you have borderline personality? I don't think I can see you. My panel's full right now of major depressives who are going to respond to Lexapro. And so I don't really want to take this. And then as a system, and consequently, even as a payer, for instance, the payers won't go to the third rail of pay for performance because they're afraid of the network implications, people cherry picking and adversely selecting against people who really need it, who aren't going to improve on a PHQ-9 score. So everybody's afraid of the third rail, except for apparently the Department of Defense. And in some ways, I kind of laud them for it, because it's something that we've got to be able to get to. We have to have a discussion around performance. And that brings me to my third point. I'm sorry. But my third point is that we're not surgery. When surgery happens, we are directly responsible for putting our hands in somebody's gut and taking out their gallbladder. And that's different. We have a lot more control over that, even though there's a lot of outcomes that would predict gallbladder ectomy and improvement after gallbladder that are probably related to the social determinants of health and other things. And somebody's obese, maybe, that increases the risk of needing to have a new gallbladder ectomy. But a gallbladder ectomy is a thing. We can measure that. There's a direct thing, a direct consequence, boom, we're done. And we know post-op infection rates, it's very black and white. But with depression, it's not. And like I was saying earlier, our phenomenology with mental health is poor. Our treatments, relatively poor. We have a poor effect size. We understand intuitively that these are chronic illnesses and that as clinicians, we have little power to actually change a PHQ-9 score. We're inept. We can say, you should try Lexapro and we should engage in therapy, but you and I both know that the outcome is not as directly correlated because we're not doing surgery. And so when we compare it to surgeons, it's a lot harder. And there's a lot more pushback because we have less control. And so we're even less enticed to say, yeah, sure, go ahead and grade me on that because of all these factors, in my opinion. And so it's a very complex situation and I understand your concern. And I'll finally just say, we've got to move forward somehow. And a PHQ-9 is a hell of a lot better than doing nothing at all. And we have to have very real conversations about adverse selection and what it could mean to our practice and our desire to practice. And I think psychiatry needs to step up to the plate. You know, like we need to step up and speak for ourselves when it comes to administration kind of imposing these things and understand that, you know, like it's a dialogue. It's not a top-down, I mean, government. It's not top-down. Like we can come in and say, no, we're not gonna do it this way. And we need to use our voices for that purpose. And I would also add most of the commercial payers, if you wanna fight back against the Department of Defense and they're using the commercial market as an example, most of the commercial payers are just asking that providers show that they're measuring something. They don't even care what. They just want to see that you're actually going through the process of obtaining a symptom-based measure. Thank you, that's very helpful. Hi, I'm Rob Gorney. I'm a psychiatrist who used to do clinical work. And now I work for what used to be Anthem doing utilization review. And it's obviously very different things that we look at. But I have a couple questions. The first is that when folks use a PHQ-9, what I have found over time is that there's a lot of variability in how the questions are asked and thus how the score is, what the score ends up being. And how do we account for kind of inter-provider reliability? The other issue that I've come across, which I'm sure you guys come across too, is the use of these tools to determine a broad range of a patient getting better or not that oftentimes is determined in part by the relationship with the provider and whether a patient wants to please a provider or wants to get better versus somebody who needs to stay in care for some reason or has some sort of either conscious or unconscious secondary gain for staying in care. And how those are kind of teased apart in the context that we don't just deal like we were talking about earlier with major depression, but also folks with all kinds of comorbid illnesses. The other piece is that I think that I have a close friend who works in AI stuff and does work with CHAT GPT. And I think that we're not actually that far off from CHAT GPT being used to monitor or look at what we say and how we say it during video visits and being able to pick up whether or not we asked obviously PHQ-9 and those types of things stuck to the script of a PHQ-9 versus using it in a discussion and how that might affect outcome and also being able to understand the tenor of the question whether or not it was perceived or perceivable as somebody just going down a list of questions versus somebody asking questions with some amount of empathy or care that even somehow that this algorithm is able to pick up those kinds of things relatively accurately and probably will be increasingly so. And what your guys' thoughts are on those things? Well, I will say as somebody who has done some pilots with CHAT GPT involved in the EHR, I don't think we're as close as we want to be. So I do think that the AI by virtual being AI is meant to change, is meant to evolve, is meant to improve the more you use it. And I think the only way to do it clinically would be to have it fully aware of all of the ways that those questions are being asked before implementing. Otherwise, and it can learn as it goes, but you have to have that baseline within AI if you're gonna use it for that purpose in order to make it relevant across the board. You guys wanna talk about the other questions? I think we've already covered the fact that PHQ-9 has limitations. So it has limitations and it could be interpreted in many different ways. It's all subjective. So there's that. And I think, yeah, so part of it is getting to that. And then the other part is getting to when you have a patient reported outcome, we have not really talked about perverse incentives for people to over or under-report their improvement in symptom burden based off of what they perceive as an outcome from their over or under-reporting. That outcome could be continued engagement in care and services, continued authorization for services or unauthorization for services. Yesterday in a discussion, yesterday afternoon, we learned that insurance companies were contracting with TMS and esketamine providers. And if the person had not, and I think this is not unjustifiable, but if the person, based off of their PHQ-9 response, had not had a 50% remission or response in their symptomatology, they would not continue to authorize TMS or esketamine post 30 some odd TMS sessions. And the TMS people were being coached by the TMS vendor to remind the insurance company that oftentimes there's a delay in effect size until they hit to 40 or 50 sessions. And this is just the pushback and back and forth. I don't think either side is necessarily right, but there are potential perverse incentives for subjective patient reported outcomes. And we need to acknowledge that and continue to work towards better, more objective biomarkers for illness. Thank you. This is a super important topic and I'm glad you guys are doing it. My name is Katie and I am the Chief Medical Officer of a health tech startup, similar. So I really, really appreciated, you know, you were part of the presentation, but we do collaborative care fully remotely. So I've just really enjoyed this discussion. But collaborative care was validated. Most of our contact is via telephone. And so we have a requirement to measure almost at every visit, if not definitely monthly. So it's all telephone and it's all virtual. And we have patients who don't want to do it. We have patients who don't see the point, but we use it really effectively and then our outcomes are excellent. So like regular collaborative care, you know, we get 50% decrease in 12 weeks in scores. And so as the Chief Medical Officer who set the program up, I'm in charge of quality, I'm in charge of education. So I have a couple of comments. We are looking to roll out gamification soon. Thus far it hasn't been an operational priority, both for clinicians, our clinicians, as well as our patients. So to the woman who has the serious mental illness population who don't want to be there, I think gamification would be a really great thing for you guys to look at. Absolutely. Can you define or give an example of gamification? Incentives, basically. So does anybody do Duolingo, the Duolingo language app? They do it really, really well. So it's a series of different types of rewards. So the best way to do it is you get, use psychology principles. You get a group of points that are yours and then loss aversion is more motivating than gaining more things. We're more motivated to avoid losing what we have than we are to work to get something. And so you lose points if you don't do certain behaviors and then if you do do certain behaviors, you gain points and then you get to compete with others to move up and get badges or stickers. And in the virtual environment, they're just like, ding, it pops up on your phone, congratulations, you made the Pearl League. But let me tell you, it has totally modified my behavior. I have like 183 streak on Duolingo right now, 183 days doing this every day. So we're looking to do that, which I think will definitely change. Another thing I wanna comment on is that we got a lot of pushback from different members of the team who aren't used to doing this. And I'm like, look, you guys, it takes three minutes to do a PHQ-9 and they didn't believe me. And so I was like, okay, let's do one right now. You're the patient. You can be as difficult as you want. And it takes me three minutes. And I had residents with me this semester and they couldn't do it in three minutes. And I'm like, okay, go practice. When you're not with me, you have to do it like 10 times this week, go practice. And we talk about different ways to administer the questions. I show them how I do it. So I repeat, because I'm on the phone with people, I can't even show them. A lot of our people are elderly. So I say in the last two weeks, do you zero, one, two, three and I repeat almost every single time until they get the hang of it. So you can get better. You know, another thing I wanna say too, I've really enjoyed the point that we need to step up as healthcare providers and that this is important. It absolutely changes the way I interact with patients for the better. And you and I have gotten better over time. And the way that we talk to people changes our outcomes. So motivational interviewing data shows this. The nocebo effect data shows this. And we can get better as providers. And so I think if we use this from an administrative perspective to try and identify the high performers and then get them to teach everybody, all of our performance can go up. So, you know, it's a struggle. But I think another thing that really helps us is the structure of the treatment team where we're forced on a population level every week to look at our measures, to look at patient improvement. Because then it focuses us on using the PHQ-9 data that we have collected. So those are just some of our experiences. I don't know that I have any questions, but I just have really, really loved the discussion. Thank you. I do have a follow-up question for you actually. It's gotta be a lot of work to get confetti stars to pop up and to display to the user their points and to decide what points will be denied if they don't do the activity and what points you grant them. And how did you convince not just the clinical team but everybody else to build that as an investment? Okay, so we haven't built it yet because there have been operational priorities first. But it will probably roll out in the next year. So it's coming. The whole executive team is on board with it. We've been talking about it since day one. I don't think it's that complicated to set up. There's literature from the exercise data. There's a group in, so I've done some looking into this. There's a group in, I think, Philadelphia who has a tool, a plug-and-play tool that you can use based on their own stuff. But the principles are really simple. How did you get the buy-in for it being on the roadmap? My boss and I think alike. I mean, social science principles suggest that incentives modify behavior. And we want to modify behavior to get great outcomes. And that's what our team is completely focused around. And so probably the biggest challenge we have is poor behavior or non-optimal behavior from our doctors, from our referring physicians who send us the patients. Because it's hard to change physician behavior. And so we might actually roll out gamification for them first, because operationally that would be a higher priority. Our patient outcomes are good enough. Could they be better? Very possibly. But if we could get better, higher quality referrals, if we could get the doctors to start talking about behavior change, because we do collaborative care mixed with lifestyle medicine. So if we could incentivize the cardiologists to start talking about diet at every visit, then we'd get higher quality referrals and people would be more likely to improve. So I don't want to monopolize the time. Katie, you're using an app-based product, right? We don't, it's not yet. So we have an app. It just came out patient-facing. Our provider-facing app is coming out soon. Okay. We actually looked at gamification in residency training. And I think it's a great instrument to use. And I think it's very crucial. If we want measurement-based care to evolve, we need to be creative and we need to try things out. So it's so exciting to hear that. I'm curious how it's gonna be. Yeah, it would be a cool presentation topic in future years when we get it going. We have a paper that went out last year. Right, exactly. On gamification. Hi, my name is Sarah Nagel-Yang. I'm in the Department of Psychiatry at University of Colorado. And we've been doing a lot of work around measurement-based care recently over the past year. And I'll say that adding it to the e-check-in with Epic has been a big strategy in terms of just standardizing things and making it as easy as possible so that we're not asking providers to do a lot of change. And we've focused a lot on the culture change around using the data. So now that essentially we're building a system where you're gonna have the data, we wanna see it being used in every visit and discussed with the patient. And I think we've been focusing on things like when we had our rollout of the change, having a big cake at the department meeting, giving a lot of these lanyards. That's gamification incarnate, isn't it? It's like, ooh, here's cake. Yes, cake is on the agenda for various steps in the process. And then we have swag, but then we're also, when people are having really high rates of the metrics that we're looking for, of utilization, not outcomes, we're commending teams of people like, the depression center had a rate of 95% of this and calling it out because I think getting a little competition between different parts of the department has been helpful. And I think we're very much still in the process. And one of the questions I have is thinking about how measurement-based care can help with another huge quality issue in psychiatry, which is access to care. And I think that it gets a little into the slippery slope of some of the things that we're talking about, but from the perspective of being in an academic department that serves a whole campus of patients, I definitely feel uncomfortable if I have a patient that I'm seeing that's coming to me every three to six months to refill their Lexapro, because I know that a primary care doctor could manage the maintenance treatment of their depression very well. And we have a lot of people that are needing to be seen and are waiting. And I do think that measurement-based care has the opportunity to show us when patients are improved and can move to whatever is the right maintenance treatment setting for them. And I also wonder, I think this is a little bit more complicated and I don't have a formed necessarily plan or idea, but I wonder about how, when we have situations where people are on waiting lists, how the use of measures before we begin care can help us find those patients that really need to be seen next week and they're not waiting three months along with other people who, again, shouldn't be waiting three months, but maybe can. Yeah, I think, so one of the, I fully 100% agree. One of the things that we've been trying to do is to, well, so first of all, it starts in primary care. It all starts in primary care, because that's where our patients come from, for the most part. That's where our patients will go back to, for the most part. And so operationalizing it on the primary care level where they look at trends, they look at PHQ-9s, which I know is a lot to ask of primary care, but collaborating with them in the sense of, like we have, what we have is a smart set that they, with an Epic, I know, that they can fill out based on the patient's PHQ-9 score, which I'm put, again, back and forth with them about the individual score versus the trend, but they, based on the PHQ-9 score, they have options. So if it's lower, if it's on the lower end, they can, you know, they have medications they can start, they, you know, a conversation they can have with the patient about what they wanna do, therapy, therapy referral. If it's kind of in the mid-range, they can do an e-consult, they can do, you know, like through all of those things. We try to support them as best we can, and then if they're in the higher range, obviously we want them to refer to psychiatry. They also have the option of referring to interventional psychiatry if the patient has tried multiple things. So I think, you know, it has to be a collaboration between primary care and psychiatry in that we're willing to take their patients and they're willing to take them back. I don't know that we're fully there yet as a culture, at least in our system, but ideally that would be what we would be looking for. This is probably the last question. Yes, last question. Very good. Thank you so much. Ed Norris, Lehigh Valley Health Network, Pennsylvania. I'm surprised that there are actually not more systems in here because the Joint Commission requires us to assess and monitor patients and then act on that, and they come every three years. What we did was we used the review of outcomes scales. It's only three questions, how you're doing internally, how your relationships are, and how your tasks functioning are, and we screen every single patient for every single visit, and it's simple because it's three questions and it can be done very easily within Epic and Entract. Final question. That's just a statement. The other thing about the company that you were talking about, the CEO, it's NeuroFlow. They've already created the whole thing that you want to do and gamified it because we're looking at them. The other thing that they do for collaborative care is they risk-stratify their patients so that you walk into their meeting and you already know who are the people in most, the highest risk to address. So how do you assess the fact that the PHQ-9, I hate to beat on this, you said it's not perfect, but it's not really meant for a monitoring tool longitudinally, and so how do you get around that argument? The PHQ-9 has limitations. I mean, it's just true, and yet it is a simple tool that's readily available. So I mean, I think it's clear we need better outcome measures, and we will continue to try and advance them, and if you have an alternative, that's great. I would like to make a quick comment. When we measure stuff, it's not just a matter of, when we measure stuff and we can't do anything about it clinically, it's just a recipe for burnout. A lot of gestalt quality of life measures and other things don't tell me what I need to do clinically for this individual at that point in time. So they can maybe inform me how they're doing, but it doesn't tell me, do they need more Lexapro, or do I need to add something different, or what's my algorithm I'm following for depression right now, and where does that, these gestalt outcome measures are great and should be looked at, but for a clinician intervention can lead to burnout because then they're just checking off a bunch of checkboxes and don't necessarily know what they're supposed to do with the data or how it's functionally useful for them as a clinician managing this person with the evidence-based skillset that they've been taught and know. And so we have to be careful about the measure selection that we pick and that we're not measuring things that have no value for clinicians or patients. Thank you everybody for coming. We really appreciate you. Good questions, thank you.

measurement-based care

psychiatry

treatment outcomes

patient satisfaction

electronic health records

Switzerland

clinical settings

ZoomCare

data integrity

PHQ-9

GAD-7

patient engagement

psychiatric care quality