I'm Bob Trestman, and had the honor and pleasure this past year of chairing the Presidential Work Group on the Future of Psychiatry. I want to welcome you here this morning because of, you know, some of the really wonderful and transformational challenges that we are facing as a field and as a nation. One of the key opportunities we have is to look at the future for psychiatry in specific health care in general when it comes to making decisions, guiding policy, shaping care delivery. And the way we do that is through data. And what we have learned over these years as we struggle is that so much of the data we have are process variables. How many people are seen? How frequently? With what codes? We don't actually have data on things that actually matter, like outcomes. Are people getting better? Which people in which conditions, with which medications, dealing with which comorbidities? All of these really complex, challenging issues that can be solved, clearly, if we have the data to do so. So this morning, I'm delighted to be able to share with you an opportunity to think about data and its potential uses. As most of you know, I hope, over the last decade, the APA has invested in growing PsychPro, our own developing data registry, which would allow us meaningful access on a national level to psychiatric data to allow us to start looking at management, decision making, care delivery, improving everything from diagnosis to treatment, looking at eventually mediators and moderators of care. But we're still a long way off. We can have a glimpse into the future, though, by expanding the scope of what we look at by focusing on some work that's been done already in primary care and looking at primary care as path leaders and path breakers for us. So today, we're going to have the opportunity to review some of the really amazing work that's been done in primary care with the prime registry. And that information is going to be shared to us by one of the leaders who helped to develop and nurture and grow it nationally, Dr. Bob Phillips, who is on the American Board of Family Medicine. And so he will be sharing with us some of the lead-in to the development of prime and over the last decade, what has evolved into a very, very large, substantial, and growing database that's helping to support decision making for primary care. We'll then be hearing from Dr. Brian Batson, who is the CEO and historically CMO of the Hattiesburg Clinic, who recently published a modest study looking at care delivery in teams, looking at the ways in which care can be delivered in primary care, looking at the distinctions between care delivered primarily by physicians and by nurse practitioners. And while it was intended originally to be a relatively straightforward study, they found things that they didn't originally anticipate. And it's blown up, quite literally, in the national press because of some of the really fascinating, almost counterintuitive findings that are of enormous relevance to us to help shape the future of our own thinking as it relates to the potential for building teams for care delivery models. And then Dr. Michelle Reed, who sits on our board, the APA Board of Trustees, will then bring it all together to help summarize, reflect on what she has heard and how that has impact on the potential future for psychiatry. Then we'll be opening it up to questions and discussion from all of you. So without taking up any more time, let me turn it over to Dr. Bob Phillips. Thank you, everyone. Bob, thank you very much. And I apologize for interrupting Switzerland. I was very much enjoying that slide show, too. So I want to give a little bit of context first. Why is the American Board of Family Medicine interested in policy? Why is it interested in a pretty big data play? So we are interested in how do we help our diplomats, our 102,000 family physicians across the country? How do we reshape the clinical environment so that providing good care and being good health professionals is the easy option? You know, I often talk about this in the core nudge, Cass Sunstein's book, about how do you create choice architectures that make this possible? And that means changing our understanding, it means changing the policies that govern the built environment that we're working in. Because right now, there's a lot of leveraging of our professionalism to do what is right for patients, and that's leading to a tremendous amount of burnout. So in 2013, we convinced the Institute of Medicine, now the National Academy of Medicine, to work with the National Academy of Engineering to come together and talk about what kind of engineering principles might help improve health care. And it's a bad day when engineers tell you you're messed up. So what engineers said is, the problem is not with the individuals working in the health care enterprise, but with the design and operations of the multiple systems in health care. As currently designed, these systems depend on the heroism of clinicians to ensure patient safety and promote care quality. At the same time, they add unnecessary burdens to clinic workflows, silo care activities, and divert focus from patient needs and goals. That is not a good diagnosis by engineers. And I don't think it's improved much at all in the last decade since this report came out. So this is what we've been trying to work at for the last decade at ABFM, is how do we change that environment? One of the ways we do that is in the data that we routinely collect from our diplomates, which is a burden. We understand. It takes about 12 minutes to fill out our recertification survey. But we actually collect surveys from every single one of our graduating residents as they leave training. We want to know, what are they preparing to do? Where are they going to go practice? What is their debt burden? What is their burnout level as they leave training? Actually turns out about 38% of our graduates leave training burned out. It's unconscionable. Then three years later, we survey them again. It's not required, but we still get about a 70% response rate. We want to know, we know what you said you're going to do. What are you actually doing in practice now, and how prepared do you think you were by your training? And we create feedback loops to our training programs from those data. And then we survey them at the recertification, which is currently seven to 10 years out. ABMS says now it's going to be five. But we kind of do a reassessment of what they're going through. And then every three years when they come in, we do a light survey. It only takes about two minutes just to update, what does your practice setting look like now? But it creates this data array over time that we have a really good longitudinal look at the workforce and what their experiences and lives are like. Oh, I'm sorry. It's walking me through each of these. So one of the products of that recently is... I've got a problem. What's that? Oh, I'm sorry. That's all right. We're good now. Is the Family Medicine Factbook. So we kind of swept up these data sets and said, what is the workforce look like currently? And one of our health policy fellows led that effort. But just gives a landscape of what's happening in family medicine. So that takes the data from each of those elements that we collect and runs them through a series of reports that we produce for the profession about what's happening at certification, the National Graduate Survey at recertification. What's happening at each stage of these people's lives, and then sweep those together into a data book about what's happening. And that's given us great leverage in Congress, even, to talk about what is happening in family medicine currently. Gives us simple things like, what is the family medicine to population ratio by state? And we break that down much more, but just giving a landscape for each state to understand. So it's very basic stuff. As I said, feedback loops to our training programs. Every training program gets a report. Here's what your graduates are telling us about their experience, their preparation for practice. And here are opportunities for you to learn from your colleagues or teach your colleagues about how you're doing what you're doing. It actually meets an ACGME requirement for our training programs to survey their graduates, which always gets about a 30% response rate. We get a much better response rate, and we take the burden away from them. So again, we get these early reports about, do you currently provide any direct patient care? Are you practicing in the outpatient setting? We ask them, again, about their debt burden and how they're servicing it, what their income is, which can perturb some people, but it gives us a better sense of how much people are able to pay down their loans and how it affects their choices. One of the things that we published recently is looking at National Health Service Corps versus Public Service Loan Forgiveness programs. National Health Service Corps is capped, and about 5% of our graduates have year-on taken on the scholarship or the loan repayment program. And they are much more likely to be in rural practice, much more likely to be in health professional shortage areas, even after their loans are paid. They're much more likely to work in medically underserved areas. The Public Service Loan Forgiveness has tripled in uptake by our graduates over time. It is almost like it's a policy conflict. We know that they were trying to reduce people's loan burdens, but it has actually started to work against their intentions of expanding health care access in the country. We also collect data on race and ethnicity of our family physicians, which did cause some problems. We do it because we want to understand, are our certification exams or our certification tools, do they have bias in them? And to do that, we have to know the race and ethnicity of our physicians and look for questions that might behave differently depending on people's race or ethnicity or even their rurality of where they grew up. And then we take those questions, we look at them quite seriously and take them out. But it gives us another window on what's happening in the diversity of the workforce that you can't get from any other source. So we know from other data sources about certain relationships, but one thing that our data have enabled people like Larry Casalino to look at is the relationship between physician burnout and the quality and cost of care. It turns out it is not an easy relationship, but it is there. The ability to talk about burnout and how it relates to outcomes, this is the only study definitively to be able to make that correlation. The two studies that came out before it have both been discounted because of the data sources, but we want to be able to understand how is burnout affecting not just quality of care but the lives of our family physicians. We tried for a long time to work with the Office of the National Coordinator for HIT about EHR functionality, satisfaction, and interoperability. We had actually structured these surveys that I talked about to collect a little bit of information when the HITECH Act came out. We wanted to know how fast were our family medicine practices taking up EHRs. We were able to show how it paralleled the National Inventory and Medical Care Survey, but it gave us a much better picture of what was happening with EHRs. Over the years we've gone back to ONC several times because they do the National EHR Survey. We said you're getting less than a 40% response right now. I don't think you're getting the adequate data to actually inform the policies that you have put in place about interoperability. And therefore it's threatening quality of care and it's threatening what our physicians' burdens are in terms of understanding what's happening to their patients outside of their practice. So we actually built into our surveys as a test case more about EHR satisfaction, about data exchange and blocking, about burden, about burnout, so that we could start to look at how do these things relate to each other. It not only gave us better data to improve and drive policy, but it gave us a seat at the table. So right now we are working with ONC's leadership to redesign these questions, pare them down, shorten the time it takes to fill them out, and increase the utility of them for the practices. One of the things we showed them is the National EHR Survey suggests that 8% of physicians are spending four or more hours a day of pajama time. They're working on their EHR outside of clinic. We actually showed, at least for family physicians, it's twice that high. It's 16% are spending four or more hours. And we think that that's not just about family physicians being a focus of our survey. We think it's about response bias. We think when you get down below 40% response rate, you're not getting good data on what's actually happening. This is a really complex figure, but we just published in the Journal of General Internal Medicine. But it looks at each EHR, so it looks at Epic, it's Cerner, it's Athena. What's the satisfaction with this? And what is it attributable to? Is it attributable to the EHR itself? Is it about the physician and their age or the kind of practice they have? Is it about the setting they're in, the ACO, or is it an independent practice? And what we found is most of the satisfaction is the TEAL. It's about the platform. And for Epic, there's much more satisfaction, and a lot of it's attributable to the actual functionality of the EHR. With Cerner, it's the opposite direction. The majority of people really hate this EHR, and they hate it because of how it functions. So we've been able to give them a much more granular, much more reliable data about how EHRs are working. And one of the things we've walked away from the study is understanding is current EHR policy by ONC, by CMS for certification of EHRs has had no impact on satisfaction or interoperability over the last 10 years. The only thing that has moved the needle is monopolization of the EHR market by Epic. It's the only thing that has moved the needle so far. It's stunned them. So that's our survey data of our diplomates and how we're using it for policy and trying to improve it. The prime registry, we walked into this space in 2016 because there was no primary care qualified clinical data registry in existence. So the ABFM funded it. We put about $7 million into developing and running it over the last eight years. And it is the largest clinical registry for primary care physicians. It does meet their needs and reduce their burdens for reporting out for MIPS or for their insurance companies. But more than that, it's become a laboratory for us to develop new measures and test them. And then move them out into a broader environment for endorsement. It's allowed us to start to study important questions about what's happening in primary care that no other data set will help us do. It's let us build population health data tools that help us, you know, identify neighborhood level risk for a patient of having social needs. And how do you translate that into payment policy so that it's supporting those patients when those needs are identified in practice. One of the things we've done is work with CDC. We're in our third year, actually, at about a million and a half dollars a year of doing research for them to understand, in their view, what happened in COVID and what happened with long COVID and treatment. What were the experiences of patients? And now looking at what happened with sexually transmitted diseases over that period of time? What happened with disability and what people with disabilities more likely to have COVID or be affected by it? What happened by race and ethnicity by neighborhood deprivation? And it's been fascinating for them. We've done it with Stanford as a partner because they're able to clean the data and do, you know, bring much more robust methodologies than we would be able to bring to them. And it's really started to tell the primary care story. It's built sustainability for our relationship with CDC and with our practices in the registry. It's created evidence for increasing understanding at HHS about what happened and how they should do it differently next time. Like, half of our practices had no vaccine for COVID through all of 2021, yet they delivered vaccine that whole time for every other disease that we vaccinate for because primary care is where most vaccinations happen. But they got no COVID vaccine. And then, more importantly, they got no data about COVID vaccine. So the Federal Registry for Immunization did not actually produce reliable flow of data to practices so that they could know, is this patient in front of me immunized? How about my patients who aren't coming in? Are they immunized? So it made them very unable to be an agent of getting people vaccinated when they are the most trusted person that people turn to for questions about immunizations. CDC was so, they thought that was such an important message, they pulled their authors off of it, and it's going to come out in health affairs soon. But it means we need to fix the next pandemic response. But the way we have built the registry and the way we've partnered with it is bringing in a lot of investors. So the National Library of Medicine, NIH, FDA last week gave us a million dollar grant over the next two years to look at disparities in treatment of COVID. But it's letting us, it's bringing in investors to answer questions that we think are important. More importantly, it develops relationships so that we can continue to investigate things with them over time. And it's bringing in, we're investing in artificial intelligence capacity. We're working with Stanford to bring that capacity in. But we know that there are not great real world data sets out there to train AI tools on for primary care. And it's dangerous if you bring an AI tool into my setting from a hospital based data set. So we're actually trying to work out how do we harness this real world data set and charge people to develop their AI tools. So again, back to the reasons we do this is that the professional behaviors are profoundly influenced by the organization, the environmental context of contemporary medical practice. We are affected by the settings that we're working in. And that these external forces need to be harnessed to support, not inhibit, professionalism in practice. We really need to change the environment to make being professionals easier. And we need form that supports function. The Bauhaus whole school of design is about form follows function. And we need to do that in healthcare now. That engineering diagnosis of what's happening is the reason we need to do this. The other thing, I talked about our registry being a laboratory. So we have a measures that matter in primary care program. We are developing measures and then testing them in the registry and then moving them out. Because we know that's part of changing the environment. My own health system is now putting a measure for us about number of new measures. I mean, I'm sorry, number of new patients seen every month. That's about throughput. That's about volume. It is not about relationships. In fact, it's going to shred one of the strongest functions of primary care and now measures of primary care called continuity. Sorry, your slides aren't advancing at all. Oh my gosh. So we need. Wow, I've been talking this whole time and they're not advancing. I apologize. They're advancing on here. Yeah, exactly. I'm going to turn that around so everyone can see. Yeah, of course. Apologies, folks. I think that the slides are just great. Thank you, the slides are awesome actually, yeah, yeah. Do you want me to keep talking while we, I know we're going to run out of time. So we have a whole measures that matter platform where we're developing measures of continuity, comprehensiveness, person centered primary care, which is a patient reported outcome measure. We actually have just launched one on trust that we published in Millbank quarterly, how we got to the trust measure that we're going to be fielding. Because we think these are going to be much more high value measures of primary care, and more importantly, they will drive the form that we function in. Instead of being measured on how many new patients I see a month, I'd much rather be measured on how many relationships I'm able to sustain in my practice. Because we know they're therapeutic. They not only improve cost and utilization, they improve mortality. There have been 20 studies now that show continuity affects mortality. So we're really trying to move at those measures, and the registry gives us the capacity to do that. We've moved the patient reported outcome measure through all the endorsement process. We've got continuity now on the core quality measure set that all payers can use. We're waiting for CMS endorsement. We couldn't have developed the data we needed to make the case for them without the registry. One of the other things is that it's not just about the form or the measurement. It's about how does how we're being measured align with our intrinsic values? How does aligning how we're valued with our values make us less likely to be burned out? So that you're always working at the thing that you're being measured on being the thing that you know delivers the best outcomes. See that's moving. No. But it doesn't move up there still. Okay. I'll keep going while you work on it. Yeah. Call somebody. So we're trying to move it through the policy process on the payment side. We're trying to move it into certification practice improvement. So if you're being told that your continuity is not good, you need a tool or a solution, how do I improve continuity in my practice? So we're trying to build those on the certification side as well. And we're actually trying to partner with CMS, not just move things through their endorsement process, but to help them understand the value. On the measure side, we're struggling, but with the Innovation Center, CMMI, they get it. And so I'm hoping the next primary care payment model that's supposed to come out in the next six weeks has continuity at its heart. The Gordon and Betty Moore Foundation, just down the road in Palo Alto, has been focused on diagnostic excellence. In our conversations with them, what they came to is continuity is an important intermediate predictor of diagnostic excellence. It took us a while to get there, but having a relationship, knowing this patient's experiences, knowing their values, knowing what's happened in their family can help you arrive at a diagnosis faster and it can help you convince them of what the next steps are, either for diagnosis or for treatment. They thought that was so interesting, they gave us $500,000 to help develop that continuity measure. And they've actually now just given us another $200,000 to help not just move it through the endorsement process, but to get it into general use, to get it into payers, to get it into health systems, to really start to influence the environment. And that was a really good sign. Social drivers. I talked a little bit about how we're connecting, you know, taking patient address in the registry and connecting it to their neighborhood, and then using deprivation indices to identify patients who are at greater risk of having social need. But we also worked over the last year with payers, with health systems, with patient groups, with CMS, to talk about how do we move from just adjusting payments based on clinical risk to adjusting payments based on social risk. And we published those in Health Affairs on both sides of that workshop series. And it influenced CMMI to launch a model in Maryland that adjusts payments for social risk. It reinforced Massachusetts' efforts over the last five years to do the same thing. We're hoping it turns into a general model. But the idea is that if I identify a patient who has social needs, I need resources to actually get a roof over their head, or to give them food, or to get them into social services. And so it's how do we rechannel money to people who take care of high-risk populations. The other thing that the managed care organizations told us, well, if you do this, it actually creates incentives for us to move into underserved markets. So it has a push-pull effect on being able to address social risk. This is my boss, so he will administer. So while they're working, again, it's taking data to drive policies, and in between that, to have really high-stakes conversations that can move that policy further faster. So it, and again, it's all back to how do we change the environment that our clinicians are working in so that they have the resources they need at the point of care. Which one was it? That one. It's that one right there. Fabulous. Let's see if I can. It should work. You could probably use this, too. Can I scroll down? I'm almost finished. And you all have been very tolerant. It's a remarkable pro to do it a cappella. Thank you. Well, it stopped again. Well, this thing. Can I do it there? You should be able to. Nope. It just froze again. Well, the last thing I wanted to point to is we actually got the Robert Wood Johnson Foundation to fund us. And you guys, you can have the slides. There are live links in these to everything I've been talking about. But we convinced Robert Wood Johnson Foundation that our work on social determinants and social drivers of health was of such high importance. And now that it was moving into the policy arena, that we needed to build a better one. We needed to be able to bring together data sets that would give us an even better capacity to say, yes, actually this patient is high risk. Even if they're not sitting in front of me. Because the highest risk patients don't come in. This patient sitting in front of me or that patient out there in my community who I am responsible for, how do I identify them at being at risk? And how do I put in place the solutions to solving that? And so we've actually moved into the census. So Census Bureau brought us in. We're bringing our data set. We're bringing an all payer claims data set. We're bringing IRS data, identified Medicare and Medicaid data. So we're bringing together very sensitive data sets. Scroll down. It really likes certain slides. I'm going to leave it there because it's not moving again. But how do we bring together these highly sensitive data sets so that we can build those better indices and have more reliability for how they should be introduced into policy? So we've harnessed Stanford. And I say that with all respect. But we have harnessed a big chunk of Stanford. We've harnessed several AI entities. We've harnessed the Census Bureau, the IRS, CMS to bring us the data sets because they all see the value of the effort of working together and changing the way we pay for health care or pay for social services so that our clinicians have a better chance of actually solving the problem that's affecting this person's health. So I'm going to stop there because that is kind of how we're trying to harness these data arrays and bring in partners to reshape the clinical environment. Thanks. So our next speaker is acapella or otherwise going to be addressing us about a more specific study focused on team-based care in primary care, nurse practitioner, physician delivery of care, and the outcomes of that. Again, looking at the use of data in this. So if I may, let me turn it over to Dr. Brian Batson, who may be able to use his slides to. Let's try something. Sure. Now we don't want to update. Now we want to just go to slideshow. And so it starts off. We'll see if it keeps going. And we'll see if it keeps going. We'll cross our fingers. So what we might do. Okay. It may be a little clunky. But I may close down and start after every slide if we need to. Thanks for having me here. Dr. Brindel, and for the invitation, my name is Brian Batson, as mentioned above. A couple of disclaimers from me. First, our experience that we're going to be sharing today was not a randomized double blind clinical trial. First disclaimer. Second disclaimer is if you're familiar with our experience and the work that was done in the publication, if you came here expecting me to bash nurse practitioners, physician assistants, you're in the wrong place. I'm not here to do that. In fact, you may find the opposite. I think what we learned throughout this process is there's some things that we need to do differently as well as the physician body. So with that, we'll give it another pause. Okay. Who we are. Hattiesburg Clinic. We're 60 years old, established by 10 physicians. This is important, I think, for our story, for you to understand a little bit about who we are. We're right at 500 physicians and APPs at this point, 17 counties, 73 locations, 50 specialties roughly, about 2,500 employees. The facts on the right give you a little bit more context of who we are as an organization. We're about 50% primary care, 50% specialty. We believe in data in Hattiesburg Clinic. Interesting correlation to the study that the information that was just presented. We're HEMS Stage 7 and EPIC Gold Stars 10, which puts us in the top 4% in the world for our use of data in driving decisions and population health throughout the country. Last year, we were also named the best place to work in Mississippi in 2022. So maybe there's some corollary here. Geography is also important to this story. We are in Mississippi. Many of you may know a little bit about Mississippi. The geography is important to this story as well. We are located roughly equidistant from Jackson, New Orleans, Mobile, Biloxi, but we're in the southern part of the state, not as rural as some of the other parts of the state, but nonetheless still fairly rural. You may be familiar with a few faces from Mississippi. Many faces you see here you may recognize, but what you probably aren't familiar with are many physicians from Mississippi. For the physicians in the room, you may have heard of Dr. Arthur Guyton, father of medical physiology. I had the distinct honor of having him as a professor throughout my training. Interestingly enough, before I went to medical school, I was an engineer. And so Dr. Guyton was quite frustrated with the engineering mind and the way of thinking and solving problems. So Dr. Guyton, as we looked at our predicament in primary care, some of Dr. Guyton's teachings were relevant to the way we approached this. For those of you who are not familiar with this information, HPSA data, healthcare professional primary care in 2022, the dark blue, the whole county is in a shortage area. So short of four counties throughout the entire state, Mississippi is in a shortage area. In fact, so much that Mississippi ranks number one in the country for the lowest physician to patient ratio throughout the United States. So shortage is very real in our area of the world. And it drove some of the decisions that our board of directors made as we were looking toward the future of Hattiesburg Clinic in primary care. In 2020, Medical Economics published the 20 states most desperate for primary care physicians. We were number four in the country with a ratio of only 109 primary care physicians per 100,000 patients. In 2000, the Hattiesburg Clinic Board of Directors, as we were looking toward the future and growth, made a very conscious decision that facing an area of the country that has the lowest physician to patient ratio, we needed to approach the problem differently. And very deliberately, the board organization made a decision at that time is we would expand our provider base with advanced practice providers, nurse practitioners and APPs. And in that decision-making process, knowing that our largest shortages were in the primary care area, made the very deliberate and conscious decision to allow APPs, nurse practitioners and physician assistants, to function in the PCP role, to serve as the primary care provider. You see the growth of the organization over the 20 years that followed. Roughly 10% growth year over year, but a large part of that was in the APP population. So this is our story of how did it get us to where we started looking at this data in 2020 to figure out what did those decisions mean. Now, layer onto that the decisions that the organization was making about expanding care delivery through APPs, layer that on top of the legislative action that was happening in 2010 with the establishment of the Medicare ACO program and a lot of conversation about furthering value-based healthcare delivery. So as we're seeing this growth happen in the organization, we also have this concept of how are we going to move into a value-based care arena in these strategies. I'm sure everyone is familiar with the ACO in the room, but in the establishment of the ACO program, this was a quote that resonated with us as an organization, that Medicare shared savings program will reward ACOs that lower their growth in healthcare costs while meeting performance standards on quality of care and putting patients first. A great, a great ideal, a great way of envisioning the future of healthcare delivery that we as physicians and other members of the healthcare team would be rewarded for providing the type of care that we went to medical school to begin with, to provide good care, not just volume of care, but value of care. Well, Hattiesburg Clinic in 2012 got our first glimpse at how we would do in value-based healthcare delivery. We, your organization may have participated in this GPRO group reporting quality and cost data. So if you haven't seen these grids to orient you, quality is on the x-axis. So you want to be further to the right. Cost is on the y-axis. You want to be further to the top. So you want to, in a value-based quadrant, you want to have higher quality and lower cost. And because I don't have animation, I cannot reveal it exactly, but what we found in 2012 was not a pretty picture. It was pretty grim. We, as Hattiesburg Clinic, were in a low-value quadrant. We were low-quality and high-cost care. But over the course of the next five years, through some very deliberate actions in the expansion of our care teams, the implementation of EPIC and using data to help drive those decisions, we moved in rural Mississippi from a low-value organization to a high-value organization, where we were providing high-quality, low-cost care to the patients that we served. The quadruple aim drove some of those decisions, of course. High quality, lower cost. Improvement of the patient experience. But I love the information being presented by the family practice organizations that it also, we have to keep in mind, this isn't just the triple aim. It is the quadruple aim where the provider experience must be positive as well. These were some of our guiding principles as we continued to make decisions. So in 2016, we entered into our first ACO arrangement with CMS. We had about 15,000, short of 15,000 patients. Distributed, each dot is a patient in our ACO. As you know, those are assigned to us. We don't pick those patients. In a very rural part of the country, we were now charged with delivering high-quality, low-cost care to this subset of 15,000 patients. It's this subset of patients that was the basis of our analysis that we're going to talk a little bit more about and that has been referenced earlier. But one thing to look at, and a point of pride for us as an organization, for the first two years in our Medicare ACO experience, we were the number one ACO in the United States for quality. In rural southern Mississippi, the number one ACO in the United States two years in a row for our quality efforts. So we were figuring things out as we were moving along. The other thing that happened along the way is we were delivering high-quality care. We were also saving Medicare a good amount of money. In the first three years of the program, you see here, $12 million saved to Medicare in these efforts of providing higher-value care to those patients that are in our ACO. One of the great things about being in a Medicare ACO is for the first time we were able to see raw claims data. We were able to get a glimpse into where are the dollars for our patients being spent. Were they being spent in our organization? Were they being spent in hospitals? Post-acute facilities? Medications? You get the raw claims data. You get every penny that those Medicare beneficiaries spend that are in your ACO. You know where their dollars are being spent. We knew their quality internally because of our epic dashboards, et cetera. But we didn't have a glimpse until now into what the cost of our patients looked like. Being the physician-run organization that we were, what's that curiosity that happened first? When we got the cost of the data, we looked around the room and said, well, I wonder which of us is the costliest doctor in the room. So I blinded the names. What we found, glimpsing back to those decisions that the organization made back in 2000 in the growth, the average Medicare patient in our population costs about $10,000 per year. When we looked at our top five primary care providers, four of those five were nurse practitioners. We make the top ten list eight of the ten. So even though nurse practitioners were providing a small subset and serving as primary care providers in our organization, their cost patterns were significantly different than those patients who had a physician as their PCP. So you see the numbers here. The number two physician, it made sense to see her there. She was a former hospitalist who was hired out of the hospital to run what we called our comprehensive care clinic, taking care of our highest risk patients. So she was assigned the sickest of the sick based on risk score analysis, the top ten percent, the sickest patients in our system. So it was no surprise to see her there. The other four in the top five, kind of eye-opening. This was our look in the mirror moment. Those decisions we made in 2000 about growing the organization and providing access but doing in a very different delivery model where we were allowing advanced practice providers to serve in the PCP role. We had to take a look at ourselves and say okay here we are and what do we do differently. So the information that has been studied was first published in the Mississippi State Medical Journal as really an observational analysis. That's what it was. It was our study on over 33,000 unique Medicare beneficiaries, non-dialysis, non-institutionalized. So over 33,000 patients and over 200,000 patient survey responses over the course of the years 2017 to 2019. So a very robust data set. As has been mentioned the findings were pretty clear in our data set. You see here the quality metrics that we analyzed. There were ten of them. These were the Medicare ACO quality metrics at the time. Eight out of ten quality metrics physicians outperformed APPs. And how did we define those two subsets? If your PCP was a physician, you're in this group. If your PCP was a nurse practitioner or PA, you're in this group. And then we had some who didn't have a PCP listed in our system. What was really interesting about this set of information, because we can say it was intuitive, we would have expected this, although no one's really reported this before, there were double-digit differences in what we would call process measures. Things that didn't necessarily require a physician licensure to order. But things like influenza vaccination and pneumococcal vaccination, patients were far more likely to receive those vaccinations if their PCP was a physician. When we look at cost, again going back to that top ten list, what we found is that on average patients who had an APP, a nurse practitioner or PA, as their PCP, their cost was $43 per member per month higher than those who had a physician as their PCP. Now when we risk adjusted that, that difference became $119 per member per month. When you extrapolate that over a population of 20,000 ACO patients just in our little ACO, we're talking millions of dollars a year that would be the difference if all these patients were followed by a nurse practitioner versus a physician. Medicare, CMS, that's millions of dollars that Medicare is spending. So what drove some of that cost? Some of the logical things that we would think about. More tests being ordered, more referrals being placed, but also utilization. Utilization of the emergency room. Patients who had a physician as their PCP were much less likely to use the emergency room than those who had a nurse practitioner. What was staggering though, and I don't have an explanation for this, is patients who had no PCP at all were less likely to go to the emergency room than those who had a nurse practitioner as their PCP. I can't explain that. And when you risk adjusted that became even greater. One could argue that those were healthier patients and didn't need to go to the emergency room. That is one possible explanation. Another driver of the cost was referrals. We saw that patients who had a nurse practitioner and or a physician assistant as their PCP were much more likely to be referred to a specialist for diagnoses that if the physician were the PCP those patients were not being referred. All these things driving the cost of care. The patient experience, because one of the things that has been reported throughout literature through time from the nurse practitioner and physician assistant bodies is that patients enjoy their experience with non-physician providers more than they do their physician providers. We did not find that to be the case. We found it to be very similar across the board where patients had equally positive interactions and rated both their physician and their APP very highly in our system. So we got this huge analysis now 33,000 patients, 200,000 patient surveys and we've made 20 years of decisions as an organization on how we're going to staff our care teams and it became this moment. Maya Angelou quote on do the best you can do until you know better than when you know better do better. That's where we were. It didn't change the fact that we're still in Mississippi with the lowest physician to patient ratio in the country but how do we do it differently? But more importantly as we looked at this and shared our story recognizing that we're not the only state in this situation. The HPSA data throughout the country shows that there are many states very similar to ours who have dark blue throughout most of their states and as they make their decisions on how they staff their care teams how do they do it maybe a differently like we're doing. So what did we do? A few things. With a year and a half lead time we changed the way we staffed our primary care such that no patient would be allowed to have a nurse practitioner or a PA as their primary care provider in Hattiesburg Clinic. You must have a physician as your PCP in Hattiesburg Clinic as of January 1st 2021. We had to message that we had to work around it and we had to you know navigate the politics sometimes with that were internal to us but it was a year and a half of conversations around this data and and what drove our decision to provide better care to our patients. So we shrunk panel sizes you see on the right I blinded these names as well. We had some physicians who had risk adjusted panel sizes of nearly 4,000 patients. Absolutely impossible to take good care of 4,000 patients as a PCP. So this is continuing to be an evolution for us as we work toward redesigning our care team. Some other things that we did were to no longer allow a nurse practitioner PA to refer to another specialist without the collaborating physician to review the case first. We began implementing alternating visits. So what our story was recognizing that physicians aren't just falling out of the sky especially not to come practice in Mississippi was that we needed to expand our care teams which meant leveraging the skills very talented people nurse practitioners and PAs in our system to work in more of a team-based model such that we didn't have physician PCPs and nurse practitioner PCPs with different panels. They had one combined panel where the physician was the head of the team and the patients then had alternating visits between them. So our general rule of thumb is that we prefer alternating one-to-one but one-to-two with one physician two APP visits is very appropriate and then that walk-in visits would be shared between the two as well. Those are some of the changes we made. We've continued to make more. This initial journal article from the Mississippi State Medical Association got some traction as has been mentioned. The AMA picked it up and shared our story. Naturally that story was twisted and maybe not understood exactly. Hopefully you'll have a better understanding of what led us to look at ourselves in the mirror and what led us to make some of those decisions back in 2020. At the end of the day we're still on this journey of value-based health care. In total in the first six years of our experience we've saved Medicare 49 million dollars and we have shared in a portion of that. But more important than the dollars is what we've done from quality. We have improved our quality scores tremendously since we began on this journey in 2012 with that first glimpse at the GPRO data when it was a very ugly story of the quality that we were providing to our patients and we've seen continued improvement through this and the retrain the transformation of our care teams has been a part of that. We feel like we're still getting started and one of the beauties of being a part of this meeting and others is that we get to share our experiences and challenge each other to look at our own data. If anything else nothing else comes from this is that that you might go back and look at your systems and your data and help your organizations make decisions about driving health care forward in a positive collaborative fashion. So thank you for letting me be here today to share our story. Thank you and now Dr. Reed and let's see is there a, excellent, okay pretty pictures back. Thank you so much. I think that you're challenging us here to think about how we practice medicine and I think I'm so envious of the prime registry and the information that's coming about. I'm a psychiatrist and I work in a community mental health center or a federally qualified with that has two on-site federally qualified health centers and we're a certified community of health or health clinic and part of our federal funding talks about how we use evidence-based practices and look at quality and this is just light years ahead of where I would say that we are in terms of working with Medicaid meaningful use and the Medicare merit-based incentive systems. I'm just laughing we talked before the session starts about what it means in community mental health and what quality is for us. We have a series of competing organizations that assign quality to us. We happen in Michigan to work in three different counties and each one of the prepaid inpatient health plans has their own idea about these dreaded heat is measures and how that impacts our overall quality of care. We certainly have numerous evidence-based practices that we have to go by and we do have something of a pay for performance so they actually send people out who look at what we're doing in the fidelity assessment scores and if you high score high enough your rate of pay for a sort of community treatment or dialectical behavioral therapy the rates of payment are higher but nothing as sophisticated as this. So I really think just looking at the Board of Trustees of American Psychiatric Association and the funding that we put behind our own psych pro registry I think this is something that we really need to be more about getting more and more practices involved in the registry to a place that we're able to get this type of granular data down to the practice level to inform what we're doing so I think it's just fascinating but I have one question. We were talking earlier and we talked about how this Hattiesburg article got made its way around. I heard it has something to do with death threats. Tell us, just tell us. I've died for you to share with the group what you shared with us earlier about the reaction to the study and how that impacted you guys there. Sure, yes it's been an interesting journey. I will tell you the first time we shared this data again remembering we did this as a quality improvement project but it's where we were and we shared it I and two members of our Board of Directors shared it with our Mississippi State Medical Association. It was the February of 2020 and obviously things changed the next month and it kind of got put on the back burner for a little while. State Medical kind of said there's something here we need to do more about this because we need to ask other organizations and to consider studying their data in a similar fashion. We were we were fortunate in that we had such a robust data set. We had the EPIC in place so that we could look at quality. We had now the raw claims data from CMS that could give us the other side of the equation, the cost information, but we were unique a little bit I would say in that. Then as it was published somehow it got picked up. I don't understand how it got picked up so so rapidly. AMA picked it up and that was a big piece of it. Within two weeks it was the single largest viewed article in the history of the AMA's website within two weeks and I think it really resonated so much with physicians feeling like our voices were being heard that medical school does matter and that this was the first time I believe that we felt a little validated that what we brought to the table on the health care team was important. The flip side of that always is that the other groups involved in our study, the nurse practitioners and the PAs, felt slighted and that was certainly not the intention of our study. It was to evaluate us as a quality improvement project, but in that feeling slighted based on our information there's a lot of backlash to us as an organization, to the AMA, and to me personally. In fact to the point of very personal threats to me and our organization and the integrity of all of the above. I had to put that in context and to understand where it was coming from. As I mentioned as a disclaimer in the beginning this was not a randomized double-blind control trial. We didn't have an intention to ever think about this until we got to that point of seeing our data to figure out how to do moving forward. What I will say though is despite the negative spin that this was put under in many places, it did challenge organizations to follow us and looking at their data. For those of you who are not familiar with the Stanford study that came out last November on the VA hospital, it almost replicated exactly what our design was, but it did it in a scientific randomized double-blind controlled trial and came up with almost exactly the same results that we came up with. Since that time I've had at least two other organizations around the country, large organizations, who have reached out to me to say thank you for putting this out there into the universe to begin the conversation in a different way. Two organizations have said we've looked at our stuff and guess what it looks just like yours, but we don't know what to do with it. Oh by the way we're not going to put it out there for the public to consume. A, because you know it's already out there so why throw our hat in the ring, but B, they don't necessarily have solutions. So until they start to craft and engineer some new solutions for their unique circumstances, I don't know that we will necessarily see others go down the road, the slippery slope that that we tread upon over the last couple years. So that's a little bit more of the backlash, if you will. Well and one thing I just wanted to ask, looking at the registry that you created, what did you do to get all these people to join? I know we're pulling teeth, we're working with PsychPro, how did you get that many practices to get involved in it? Well I think the largest cohort came in first because of MIPS, you know, the burden of reporting measures and the threat that that presented to them was a big part of it. But now only about half of our participants are actually reporting to MIPS, the other half are wanting access to the quality information that they can't get out of their own EHR system. So our ability to take their data and turn it into information they can use is why a growing number of people are coming. Audiences, how many of you all have to deal with the MIPS? Is that a big thing in other states? Are you all dealing with the quality metrics? I just wanted to know, is this something you think will be a solution, looking at maybe those dreaded HEDIS measures? I mean, what are your thoughts on that? In community mental health, I don't know others, do you all work with the HEDIS measures at all? Somebody raised their hand. A set of measures that was designed primarily for primary care that's being put into mental health services and I don't know, what are your experiences with HEDIS and is this an alternative to HEDIS 2.0? Well I do think so. I mean the most reported measure, because we're a qualified clinical data registry, we can have measures that are not generally available and continuity is one of them. We've had it for three years now. CMS approved it for MIPS for our registry. It is the most reported measure. It is the lowest burden for them, but it is also the thing that they value the most. So I think there are some folks who are coming in for that and we want to disrupt it. In fact, we wrote a health affairs blog just last week that challenged CMS on the universal foundation measures because they are almost exclusively HEDIS measures. They all fall to primary care to collect and improve and they're going to be across all programs and we think this is not good. This is not good for primary care and it certainly doesn't make sense to roll it into psychiatry for you all to manage blood pressure. I mean that just doesn't make sense. But yeah, no I think we want to disrupt this very strongly. I know some of you said you'd had some experience with HEDIS. What are your thoughts on that? Well I'm the chief medical officer for Los Angeles County Department of Mental Health and we serve over 250,000 lives a year. Our struggle is that our payer sources, primarily Medicaid or Medi-Cal in California, come through and do quality audits and I think what I struggle with is that individual providers think they're doing a good job, but then when you look collectively at things like measuring metabolic screening, psychotropic use in terms of monitoring for side effects, they tend to externalize blame a lot. It's not me. I'm doing my job, but you know the nurses aren't or these folks don't have resources to come in to get blood draws. So I think there for psychiatry it's hard to translate because there's a lot of difference in terms of how people perceive their health. It's about the relationship, less about some of the technical aspects. Like in primary care there are much more measurable things like blood pressure, A1C. I think for us it's really about more outcomes and quality of life and I think those things we're not quite as good at capturing. The reaction though is the same. People say it's not me and the problem with HEDIS measures that are at the practice level are not at the individual level and you had to take a deep dive on the individual to figure out where is the, where are the real problems and then because that's, until you do that you can't design the solution. So I think you're, I think you are entirely correct about the inappropriateness, but I think the reaction is the same and unless you can drill down you can't figure out what the solution is and that's part of the problem of the measure environment or the ecology that we're in right now. Right, I think what we found most successful is just doing a peer review that focuses on those measures and fortunately that's been well adapted because we were able to tie it in with board recertification so that if you participate in the peer review it serves as a quality outcome and now we've gotten much more many more of our physicians on board with that. Did you have a comment? Thank you for an excellent presentation by the way it's really useful to hear the insights from primary care because I think there's a lot to apply. I recently went into private practice my question is about kind of smaller practices and this is where kind of a vast majority of patients get their outpatient treatment is in not sort of bigger you know systems it's usually like smaller individuals in private practice and it's a lot harder to collect the data for those people so I feel like we're probably missing out on a lot of measures there. You mentioned EHRs and one of the things that really struck me having worked in academic hospital systems is how hard it is to get access. Epic for example they will choose you if they decide you are worthy and the option of even piggybacking onto a hospital system is not there as it used to be so what you're left with the systems are often inadequate for collecting those measures unless you really have the time to go into a deep dive and do it particularly for solo practitioner so my concern is that you know if CMS is really pushing for this because I think evidence-based care is very important however it could end up hurting a lot of practitioners in solo practices if they're not able to just get access to those numbers that they need to show that they're delivering the care that they are doing. And I think that's maybe you want to talk a little bit about psych pro and what our role is with individual practice that'd be a good thing you want to talk about. Sure. One of the opportunities that psych pro has offered many small and individual practices is having access to that data so in that participation it meets the standards for the individual performance status standards that MIPS that's being required by CMS and by insurance companies so that and many people who are participating in psych pro are smaller individual practices for exactly that reason so there are these opportunities our challenges continue to be reaching out and finding the leverage that makes it worthwhile for individuals to participate so that we really have access to this data in a meaningful way. Thank you. So any advice you would have for smaller organizations about where to start how to get involved I mean we've certainly been involved with our funding sources around certain each the three counties for us three different sets of HEDIS measures and it's a constant fight with why are you picking looking at opioid use disorders when it's not on our formulary and we don't prescribe opioids so they were looking at things like that so it's a constant talk tussle with them about what measures are meaningful to what we do and I'm thinking you know maybe leaning towards the APA creating some more psychiatric specific measures that would be more relevant to how we practice it's really like trying to fit us into a hole with these 200 some measures most of which have nothing to do with behavioral health so what suggestions you have for smaller practices solo practices or even the public mental health system I think the pressure is long at a grove I mean CMS has been pretty clear about it they want everyone in an alternative payment model and to do that as a solo practitioners very hard unless you have a resource unless you have someone who can actually turn your data into the what you need to report so I think I think so I think that's a first is for them to understand that psych pro can actually meet that need for them now the second phase is then how do we develop better measures and get them certified for our registry participants get you using them and get your feedback on it and then move those out into the broader environment where you're you're changing the metrics and a number of our ACO partners who have come into the registry recently have done it for that reason they want to change the negotiations with their payers yes we're giving you the measures you're demanding right now but look at this set of measures that we're committed to that are aligned with our values and what our patients need can we operate with these next and move them into agreement about the set of measures that you're going to give and if they're going to stick with the heatest measures we'll give this to but those really aren't related to value and we can show you that because here it is I would add that you know just with the evolution of the measures we've seen this play out when we were reporting for an ACO in 2016 we were reporting I believe on 43 measures I can't remember the exact number but it was it was around 43 when we look to next year it's gonna be 6 but we only got from 43 to 6 by participating in the process of reporting that data it is hard it is very hard but through time and I think the American Psychiatry Association has a very valuable role here in making that conversation happen to say okay we will start with a large denominator but let's get toward a solution where that denominator is 6 or 5 or whatever it ends up being but those that are remaining being really meaningful measures and then partnering with some entity that would facilitate the independent practitioners to be able to to collect and share that information in the ACO space not too far off in primary care there are several nationwide organizations that that help facilitate independent practitioners participating in the ACO space those are those are primary care based and some of them are quite successful because that lift the administrative lift of the participation is too heavy for an independent practitioner or small two to five doctor practice so identifying those resources that can help move the journey forward but also going into it recognizing that in the out at the outset there may be a lot of noise before we finally get to the signal of here are the six measures that matter thoughts from the audience who else has issues or concerns about quality or tell us about what's going on at your place that you're struggling with good morning I'm Ken Thompson from Pittsburgh and a community psychiatrist as well I there's so much to this that I think is both manifest and thinking about how the heck you get data and use it to drive change but the process that you're also discussing about what is it that compels people to supply data or to make it possible for them to supply data I think is really a huge infrastructure challenge that we face and I'm gonna I'm gonna maybe I'm gonna put Bob on the spot just a little bit because there's a this Bob right this Bob the one on the left my left because I think it's possible that we could get the idea that the way to drive this move in psychiatry to get into this to figure out how to systematize in a very unsystematic system the collection of data is the notion that it's going to be aligned because we're going to figure out how payments going to shape it and and I want to suggest that it's I don't I think that's really important I don't think that's the only thing and where I want to put Bob on the spot is it's my understanding that within family practice there has been a very significant move over the last almost two decades to redefine and redesign what family medicine and primary health services are so when they're doing their data collection it is based around the idea that if they didn't modernize how they practiced there was not going to be really good family practice opportunities or a profession so they've been motivated to look at how they practice and where they practice and what they do based on that so you heard Bob talk about the notions Bobby spoke about you know where are we in the world how are we providing services how are we making sure people are getting into areas where there's no access right now what are we doing to move in that direction there's a sort of mission driven path part of this process and I'm not sure that in psychiatry we figured out what our mission is to be able to motivate folks to actually provide the data to allow us to look at how the profession fits into the world right now so I maybe I've got the story wrong but I think that's been a really critical piece of this and has allowed you to get to the place where we really frankly I'm so envious I mean so what you were saying Michelle like what how do you do this how do we get there so I'll just stop there well I can thank you so part of it is that our diplomates are already acculturated to providing these data it doesn't mean they always like it but they know that it's gonna it's gonna be required of them so sustaining that is easier than starting it you'll get a lot more pushback in trying to start it but I think you probably needed to come at it and maybe we can help with this is helping them understand why this is important so why is it important what can we do with it to help you in your career and the second part maybe maybe being clear around the help for them is is is direct but the help for them can also be around how are we going to change your practice how are we going to change the training models how are we going to change the specialty to get to mission and having that that kind of governance about mission as the reason we are doing this I think can help communicate why are we giving you this new burden we're seeing our natural National Graduate Survey response rate decline because it is not required to do it from 67 to 71 percent now down in the 50s we're probably gonna make it required now because the the derivative benefit to the programs on improvement of training is so high that we think it's necessary we're gonna get pushback about that I'm hopeful that we can demonstrate to them that there's good reason for it I'll add and I'll contend that there's a lot more overlap maybe than might first be recognized when we when we put into a silo of family practice versus internal medicine versus psychiatry we know that in a value-based health care model nationally and we know this locally within our ACO that the total cost of care is heavily influenced by a patient's mental health we know that patients who have chronic conditions and one of those chronic conditions is a mental health diagnosis the cost of care for those patients is double those patients who do not have a mental health diagnosis so when we approach it from a viewpoint of rather than the silos that we often can follow fall into but that this is a collaborative effort to improve health care for our patients collect that that collective our patients that data points what whatever we're measuring total cost of care is something that I believe mental health and psychiatrists do have an important role in us figuring out how to do better. Go right ahead. Bill Arroyo from Southern California I have I'm a psychiatrist and I have worked with our state agencies for at least 20 years around the managed care plans I recently served as a member of the group that developed the Medicaid waiver for California five-year Medicaid waiver I'm also impaneled by law to advise the state on the children's Medicaid plan which includes physical mental dental or works and I've done that for about the last 10 years or so that being said I'm very familiar with data collection by our oversight state agencies California has a very complex health care oversight structure which includes at least two state agencies that are responsible for monitoring health plans and some of the health plans can choose which state agency they want to be monitored by okay then there is a State Department of Health Care Services which is the agency I primarily work with that establishes the contracts with the health plans regarding Medicaid that being said metrics and medicine is a very interesting industry HEDIS measures are merely one of and I'm looking here at our website at least 25 tools that are used to monitor Medicaid health plans in California that being said there are some very large health plans in California that cover millions of lives and many of them fail to reach the benchmarks that are established in contracts and in regulation however the state agencies also realize well they can penalize health plans and they do on occasion for I think last year our largest Medicaid plan was penalized 250 million you know that's really you know doesn't cut the margin a whole lot let's say I mean they're huge but the state agency is also faced with the option of discontinuing their contract but what that would mean is that millions of lives would not have health insurance and they're not about to take that step no matter how many failures that health plan has has made with regard to the benchmarks that are established by regulation and their contracts so that's my reality in California and I'm wondering if other states share the same reality anyone in the audience any thoughts on that what's going on in other states okay I think you want we'll have our pan to panelists we'll give you all some closing remarks as we wind this down but what are your thoughts how this applies to psychiatry the comments that you've heard from others give us your your parting thoughts and your hopeful advice about how we move forward well I want to say that we looked to psych pro early in the development of the prime registry as a as our own benchmark for engagement in research I mean that your registry was doing wonderful research with NIH and others that we really admired and aimed for where I think we have some opportunity to work together is then how do we leverage it not just for research but for policy and how do we turn your data into policy influence and we'd love to work with you on that I would say that our experience is on a very small level I gave you a story of a 500 doctor clinic in our 500 provider clinic in rural Mississippi I think it's remarkably similar to the psychiatry story nationally of we don't have enough psychiatrists in the United States of America so how do we take the skills the knowledge the expertise of the psychiatrist and use the information we have available to us and teams to leverage that skill set to the best that we can provide our patients so ours is a little little small little story but it translates I think remarkably well to the psychiatry profession and when we look at CMS models like collaborative care that's very progressive far more progressive than anything that's ever been done in family practice or internal medicine where we recognize there aren't enough psychiatrists or we recognize there are not enough internists but we then put a model together that can take care of more patients hopefully in a positive fashion it's still it's an experiment of sorts right now but it makes sense it makes sense that the psychiatrists will be operating at top of license as will the LCSW or the counselor or the psychologist or the family practice doctor who's the one actually writing the medications for those with one diagnosis it seems to for me be remarkably similar in what challenges we faced in Mississippi with primary care and what the nation faces the psychiatric care I am just delighted with this conversation because I think it highlights what our challenges and opportunities are for the next decade to start thinking about not only collecting the data but thinking about what data needs to be collected in what form to shape the conversations it also brings up some of the opportunities it was you know as Dr. Arroyo just mentioned from California organizations are too big to fail and so aren't being held accountable in ways that shape improvement we have challenges galore but we also have a lot of smart people who are really dedicated to improving the quality of care being delivered to those who need our care so some of the challenges are clearly improving the data collection process engaging people around the system and challenging what care is being delivered by whom and I think it's also almost a clarion call to the field of psychiatry to rethink what care are we delivering directly as psychiatrists and what's the difference in the years in med school the years in residency training and fellowship and how are we then spending our years in practice and are we delivering care that no one else can deliver so with that I think our time is up and I really want to thank everyone for their participation it's an exciting time it's an energizing time thank you and enjoy this conference be well oh and yes there is one other thing just for those who are really interested and related to the next step is the presidential workgroup presentation Monday morning at 8 a.m. where we're looking at the future of psychiatry and the support that the Board of Trustees has given the APA to help shape the future of the APA and the future of psychiatry thank you one and all

psychiatry

healthcare

data-driven decision-making

outcome-oriented data

American Psychiatric Association

PsychPro data registry

family medicine

National Academy of Medicine

physician burnout

team-based care

evidence-based practices

HEDIS measures