Okay, good morning. I think we're, this microphone is very definitely live. So we are gonna get started. Thank you for being here and perhaps beginning your morning with us, although I guess you don't know if you had earlier meetings or not, but I'm Jeff Linus and I'm joined here by Drs. Josefa Chong and Bob Boland from the ABPN and we're really pleased to be thinking with you, both those in the room and those streaming this or watching this on the recording, to think with you about lifelong learning and specialty board certification, or as you can see from the subtitle, how to not peak at the end of training. This is something that we've developed at the ABPN over the past year and have presented this live a couple of times. Actually, interested to know, anybody in the room a trainee currently, either a resident or a fellow? So it's great, really appreciate your being here. We did this most recently at the American Academy of Neurology with one of our neurology directors and we're joined by some trainees and also by program directors and program coordinators and other faculty, which is great and really appreciate that. A version of this is actually available on video on our website, kind of broken up into two short videos, which programs, or we put it there in order to have programs to be able to take that and use that as springboards for discussion at seminars, at programs. And perhaps what we talk about today may lead to your giving us feedback to how to improve what we offer you all in that regard and maybe things you want to bring back to your home institutions as well. So actually, just to say a little bit more about Josefa and Bob. So both of them are on our board of directors. So they're both psychiatry directors on the board. Our board, as you'll hear, is both psychiatrists and neurologists on the board of directors. Josefa serves as board secretary. Bob is also on the executive committee of the board currently. And we'll be kind of tag teaming and kind of dividing up the slides here. So basically, what we'd like to kind of cover, as you can see, is think a little bit with you about lifelong learning after completion of formal training, the why and at least a little bit about the how. Of course, there's no one recipe for that. And then move into telling you a little bit more about the ABPN in terms of our mission and who we are as people, the people involved with the organization. And then think with you some more yet about certification, including continuing certification. So this may seem fairly self-evident why we need to think about lifelong learning after the completion of formal training. But the reality is that the evidence base for what we do as psychiatrists is going to change after we finish training. And in part because of that, although perhaps not entirely because that practice is going to change. I think each of us can attest that since we completed training, most of the treatments and most of the things we do for patients on a regular basis now either weren't around or were around, but we didn't know how to use them the way that we do now because of the evidence base changing. And of course, that's mostly a good thing for our patients, but it means we've gotta pay attention to how the field evolves. And as an example of this, this paper now goes back some years, but this is one kind of sample study. They looked at one high-impact journal, particularly the New England Journal of Medicine, looked at 10 years of papers that tested and established medical practice. This is not just in one field, but across fields. And what they found was that 40% of the studies, or 363 articles that tested and established practice in the New England Journal in those 10 years, 40% of these studies found the opposite of what the established practice was, and only 38% reaffirmed the established practice. Now, of course, one study doesn't necessarily undo a whole body of work, but it does give us some sense about how likely things are to change. There's another angle about lifelong learning which most of us don't wanna think about other than perhaps in kind of cynical comments that sometimes we might say during training about peeking at the end of training, which is the idea that, in fact, experts in all fields, including physicians, including us as psychiatrists, our skills and expertise do naturally tend to decline or decay over time unless we pay attention to maintaining and continuing that expertise. That's not to say, well, inexorably declined. I don't mean to be that negative about it. Clearly, that's not the case. But what it does mean is that it doesn't just perpetuate automatically, right? We need to work at it to keep our expertise. And I also will say that there's lots of, in terms of thinking about lifelong learning, even when the evidence base is clear, even if it doesn't change, there's lots of evidence that there's wide variations in practice and that an awful lot of patients don't get evidence-based care. Now, some of that, of course, is patient factors or access factors, which are really critical, but some of that does relate to the expertise of the folks who are providing the care, and closing those gaps can take a long time. Even when new evidence kind of becomes fairly well accepted as this really needs to become the standard of practice, in most cases, new evidence takes years to find its way into routine clinical practice that most patients have access to. So all the more reason to think about what we're talking about. Another angle on this is scope of practice concerns, which is something that we hear a lot about from our colleagues in terms of people from other disciplines that we work closely with in many cases. But among the things that we bring to the table as physicians are not only our specific expertise as psychiatrists, and for those who are subspecialists, as subspecialists, but also our ability to critique the literature and to apply evidence in a sophisticated way, and that's part of what we bring to the teams as physicians and as psychiatrists. So, okay, so lifelong learning, I think we're all in the room here recognizing that it's important that we do that. What are the barriers? These are a few kind of broad categories of barriers. I mean, time and money in some sense are two sides of the same coin, right? It's, you know, thinking about in training, I think, I know I did, I think most of my friends did in residency, and I think it's still true for trainees in the years since, find that it's hard to find time to read, hard to find time to do things other than what's sort of required and expected of us in the training program, and have some idea that maybe when we're out of training and don't have that kind of structure in our time that we'll be able to spend more time reading. And then we get busy with our professional lives as well as the rest of our lives, and it doesn't necessarily get easier for most of us. And for many of us, time spent reading or attending conferences or other ways of kind of extending our expertise, ones that are taking time away from seeing patients or doing other things that are kind of revenue-generating, income-generating activities. So those are obviously major barriers for most of us. Another barrier that may seem a little less obvious but probably makes intuitive sense when you think about it is that we tend, as people, to go to the things that we're most familiar with, the things we're most interested in, right? So I know, I'm not counting since I've taken my role at the ABPN, I'm much more likely to go to a geriatric psychiatry meeting than some other meeting in psychiatry, even though I'm a subspecialist in geriatric psych. I know more about geriatric psych than I do about most of the other parts of psychiatry. And even at the AAGP, the annual meeting of the Geriatric Psychiatry Organization, I was much more likely to go to sessions on late-life depression, which was my research interest and a clinical interest, because I was the most interested in it, but it was also the stuff I knew the most about, and really needed to go to other sessions. Since I've been doing talks about this kind of topic, I think I feel enough guilt that I'm a little better than I used to be at least sometimes making myself go to things that I know I'm not as familiar with and really ought to be, but it's something we all have to be aware of. Another angle on this, what makes it challenging, is that literatures are complicated, and to really fully, I mean, there are general principles of EBM, evidence-based medicine, that we all learn to try to critique articles, but to really critique an article well, you know, the more expert we are at something, the more able we are to do that and put the article's findings in context, but of course, we're not equally expert at all the aspects of the field that are relevant to our patients, and so that's a challenge as well. The last thing, just get one bullet on the slide, but I'm gonna show you a few more things, is what's referred to as the Dunning-Kruger effect after a couple of authors, although lots and lots of literature has shown this, which is, how good are we at assessing ourselves at what we're good at and what we're less good at? Because in principle, we should guide our learning by what we know we're not as good at, right? But how good are we at doing that? And it turns out, unfortunately, as experts of any sort, as human beings, we're not very good at that. So kind of the classic kind of founding study of this looked at grammar skills, and what you can see is that self-perceived skills in grammar compared to objective measures and actual test score don't track very well, and they track in the most dangerous directions, right, in that the people who actually do the least well on these tests perceive themselves as being way better than they actually were, which, of course, for physicians and our patients, is quite worrisome. Now, you could say this is grammar, what's this got to do with medicine, but this has been replicated many times across many fields of expertise, including among physicians, so there was a study that looked at this in internist pediatricians and psychiatrists and looked at it in patient management and clinical assessment and other skills, and I'm gonna show you one slide for each, but all the specialists look the same, and you notice that the shape of these graphs looks exactly the same as what I just showed you in the grammar skills, right? In this case, it's how peers perceived us rather than an objective measure of competence, but still, peers' perceptions of people's competence, again, the folks who were perceived most poorly by their peers tended to overestimate their abilities, whether it was patient management, whether it was clinical assessment, whether it was communication skills, and that was true, again, for all three specialties across those three domains, so the shape of the curve should be scary for us, it should tell us that we can't go only on our own self-assessment. Now, obviously, there's some scatter in these curves that I'm not showing you, so there might be some individuals, probably most of us would like to think, well, I'm not that person, but you don't, that's the point, right, we don't know, and so we have to be mindful of that. So, with all that, then, just some general thoughts, and I'll be interested also in Joseph and Bob and folks in the room's reflections on what has been helpful for each of us in trying to do these things. So, continuing medical education, of course, is critically important in sustaining our expertise throughout our careers, and there's lots and lots of ways to get that, whether it be live conferences, whether it be online, whether it be live things that are synchronous learning, like some people are watching this live in the moment, but also asynchronous learning, there's lots of modules to read or videos or other kinds of media that are available to us in many, many forms. What I will say is that it's important to look, because of what we just talked about in terms of the limitations of self-critique, to look for things, not everything that we do for CME, but at least some things that we do ought to offer some way to providing us feedback, right? So, one example of that would be what's called self-assessment CME, which are basically CME programs that ask questions that allow us to take the questions ourself, learn our score, and see what we did compared to other people at the same conference or taking the same CME activity. The APA offers self-assessment CME, as do many of the major professional conferences and many other things available to us as well, and the idea is to give us feedback that we can use to see how we compare with our peers and what our strengths are. In terms of reading, you know, again, there's no one formula for this, but I think planning some kind of time in our, if not work week, at least in our work month, to try to, you know, what journals do we either tend to read fairly, at least read the table of contents fairly thoroughly. I actually subscribe to RSS feeds, so I get flags when new table of contents come out on journals that I like, both in psychiatry and broad medical journals, and at least scan the table of contents to try to decide what I want to set aside time to read in more detail. I will say that for those who participate in our continuing certification program at the ABPN and who choose to do the article-based continuing certification pathway, so ABCC is an abbreviation I'll be saying a bunch of times, and as will my colleagues here, so that's the article-based pathway. You know, the ABCC program offers us a bunch of curated articles in psychiatry and in the subspecialties. You know, the curated lists are up there for anybody to see. You don't have to be in the ABCC program to see what articles that we're using. And the nature of the ABCC program is that people take quizzes after each article that assess their understanding of what the article's trying to convey. So that's another example of providing feedback as well. Now, it's not to say it's the only articles that are worth reading. The articles are chosen for a variety of reasons, including that they lend themselves to asking questions, but they have been chosen by peers in the field to help guide our reading. I'm a little bit biased about the teaching part of this, but for most of us, there's opportunities to teach. Of course, we all teach our patients and the families that we work with, but beyond that, teaching peers, teaching folks from other disciplines, teaching residents and fellows and students and others. And, you know, I think most of us involved with teaching would say nothing really forces us to try to understand something than when you try to convey it to people who may be less familiar with it, or at least that's the presumption if you're trying to teach it. And so I would put in a plug for looking for opportunities for teaching as a way of lifelong learning. And then what I already said about pushing ourselves to the less familiar and the less comfortable, right? What conferences we go to, what sessions we go to, and so on. Taking a step back for a moment, you'll notice I'm using the word competence here, which will be very familiar to some of you, but for those who are new to thinking about these kinds of things, the word competence may seem fairly uninspiring, to put it mildly. And so, you know, there's lots of people who've thought about this in education broadly and in physician education in particular. One definition that I sort of return to in part because I'm at the same institution where Ron Epstein and Ed Hunter were at when they wrote this paper, defines competence as you can see here. I won't read it, but it's, you know, so it's a very broad and actually aspirational kind of definition, right, of what we mean by physician competence. And again, there's lots and lots of educational literature which I can only kind of skim the surface of in a couple of moments here. But so for example, there are many versions of what's been called the Dreyfus model, looking at levels of competence. And the idea here, if you look at this, I mean, the details in the boxes here are, you'll see different versions of this out there. But the idea certainly is that people who finish psychiatry residency training should be at least at the competent level across the major domains of what we expect for psychiatrists to be able to do. Many of us, as we graduate residency, will be more than that in some areas. We'll be proficient, perhaps expert at a few things. But of course, the idea is that we can keep learning, right, to gain and put as much of our practice as possible up to levels of proficiency or expertise in the Dreyfus model. I wish that there were shortcuts to doing this. One of the interesting things about expertise is that experts actually do take shortcuts. And we know this if we sort of study the ways that experts reach the conclusions that we do about the patients that we're working with and caring for. It turns out if you do functional neuroimaging studies, you can see that the novice working on a task has to involve a lot more brain systems than the expert does, right? Because we've found shortcuts, mental shortcuts, that obviously have brain function correlates. The question, you know, is how can we get to expertise more quickly? And we don't know how to do that, right? What we know is experts try to teach how we think about things, but folks have to kind of wrestle with this, develop the range of experiences that inform the development of our own individual shortcuts essentially in our thinking that get to true expertise. Another angle on thinking about competence and lifelong learning is that the idea that when we graduate, we might be sort of toward the top of the curve, but do we then kind of put our expertise on automatic, right, in which case at best we level off. This is a little optimistic in that it doesn't show declining, but at least at best level off, whereas if we keep paying attention to our learning, what we refer to as deliberate expertise, and there's a whole literature on this as well, we can in fact keep progressing. So if you then fold in about the notion that in fact skills will decline over time without attention paid, it doesn't level off, right? It kind of goes up and down, but if we keep pushing our expertise in a deliberate way as we think about lifelong learning, we can help keep ourselves above that threshold that we want to stay above throughout the course of our careers. And so I kind of think about this as the cycle of professional life, and I guess you can start this in any one of the three things here, right, but in principle what we should do is reflect on ourselves, including from external feedback for the reasons I mentioned, use that reflection to kind of guide our growth and our lifelong learning, then assess ourselves and do that in part based on external or extrinsic assessments to guide us again in our reflections and around and around this goes, right, throughout our professional lives. So with that, I'm then gonna, we're gonna change gears now from thinking about lifelong learning in general and talk a little bit more about the ABPN, and so I'll turn this over to Dr. Chong. Good morning. Thank you for all those who are joining us here in person as well as those on streaming. One of the key things about the ABPN and making us more familiar, I think, to diplomats as well as the public in general is talking about the mission and people. If anything, I think we're all familiar with this schema of what it takes to make a doctor in a basic way, medical school to residency to clinical practice, and at each of those transition points, there are part of the credentialing process. So for example, the medical degree, the verification of where you went to med school, that it was part of an LCME accredited school, which will then allow a graduate to enter into an ACGME accredited residency, and then following that, specialty licensure and clinical practice. With that, it serves the mission of the ABPN. ABPN is part of a constellation of organizations and governing bodies that basically do credentialing. We'll show another slide which has pretty much kind of where does the ABPN sit in this giant constellation, and the basic mission is actually the result. I would say that the most important point on this slide is this, that third one. What's the result of what the ABPN does? And it's to provide credible reassurance to the public, families, the patients of who is taking care of them, and it's done through the process of independent assessments and to demonstrate the expertise and abilities of an individual diplomat to provide high quality patient care and in an inclusive manner. Part of this has been re-looked at and also reformulated with the board to include the language of diversity, equity, and inclusion. With that, one of the key things about the ABPN, I know it sounds a little bit, I work for the government, I have to say, and I know that it's like, you know, I'm the government, I'm here to help, but sometimes it's more appropriate to say the government is us, you are part of us. And so ABPN is us, it's everybody who's in this room. And so, but it helps to break it down. We've got 14 exam writing committees which are populated by 300 plus physician volunteers. Many of those who you know through your, if you're a trainee, I got a good guess that somebody you know in your department probably works on one of our committees. And several in this room who have been experienced writers for that. We've got 18 exam writing committees for neurology, 14 for psychiatry, and the board itself, we have 17 directors. Currently we, it's usually 16, but we have an interesting kind of blip because we had transition points for certain directors. So there are actually nine neurology directors and eight psychiatry directors. I promised you this slide. This is the constellation. If you've ever wondered, where does the ABPN sit in the universe? And in full disclosure, I actually belong to two of these, the ABPN and then also the ACGME. I'm a member of their executive committee and so in case anything I say gets back to them, you know, you can tell them I was honest. And you can see that where the ABPN sits is right there towards the left with the American Board of Medical Specialties. The ABMS consists of, I think 26 now, or is it more? 24, 24 individual member boards of surgery, OBGYN. Think of the major disciplines. Each one has an accrediting board. And so as you can tell from all of this, this all goes towards that idea of the physician candidates, physicians, candidates, diplomates, looking at the assessment of individuals with the idea, again, to reassure the public, provide credible reassurance to the public that this individual that you have in front of you taking care of you, yourself, or your family members has demonstrated competency in an accredited manner, in a regular manner, and also documented. So what does it take to do initial certification? Pretty, a lot of stuff. You don't realize it till you've gone through it, although I'm sure the trainees here feel like it's too much. But graduating from an ACGME accredited training program, a clinical skills evaluations, unrestricted full medical licensure. Only after having these criteria, met these criteria, can one sit for the ABPN certification. The nuts and bolts of this information is actually on the website, which has been redesigned to be a little bit easier to navigate. I do like the colors because personally, again, another disclosure, they're in orange and blue. So I have to say that given I'm from Florida. And it's, this is the website when you click on that. It's abpn.org. Starting off at the very bottom here, the physician portal. It will take, if you don't have a portal now, let us know, we can help you get on. It takes less than five minutes, literally, to start to finish, to get on, get into a portal. One of the key things to note is at the very top is the become certified. And this will bring you to this page, which then walks through the different steps. I just went through this actually with my senior residents. We do a talk, we do a presentation on this right before Christmas, before people start disappearing into their individual electives or selectives or vacations. And then do it again before they graduate. And each time, I always ask them, have you started your, have you looked at the site? And I kind of dog them about it. They kind of laugh now. I think I've become the nagging mother. And it's only because it's good for you. So again, the criteria. Now we can sit through these. Specialty certification is recognized by most healthcare systems and payers. And that's one of the key facts to know. And then certification is not a one and done because, I'm gonna hand this off to my colleague, Dr. Bowman. Thanks. Right, so it's not one and done because of all the reasons that Jeff said before. A whole bit about skills decline over time. So just because you passed a test after residency to think that that makes you perfectly competent for the rest of your career, hopefully his point's made, and hopefully we intuitively know that that's just naive. And ultimately, it's important that, you know, that there be some sort of decent assessment. As he already explained, self-assessment is not enough. So there really needs to be an external assessment. And really, I mean, really to do that, you need independent assessments. I'm sure people even in residency now know that they're being assessed by people they work with, and that's great, but, you know, let's face it, there is some bias there. And that's fine because they're there to teach you as much as they are to evaluate you, so it's a good thing. All the same, it helps to have someone outside of that system to be able to do that as well, to really give you a less biased, maybe not unbiased, but less biased view of how you're doing. And face it, I mean, even if you're one of these, incredibly rare people, I must say, I'm not sure I'm one, who would just do it anyway and keep up with the literature and keep up your skills and get out there and practice and learn from others all the time to the point where you really didn't need a certifying board to make you do that, even if you're one of those people, well, you should still participate. And why? Because it's really our duty. I mean, think about it. The profession of medicine is one of the very few professions that's allowed to self-monitor. My brother-in-law is a pilot. They don't get to do that. And would you get on a plane if they did? If they could all decide amongst themselves who is good and who is bad? So yes, it's a great privilege we have, but that also means we have a great duty with that as well, right? That we need to do it well and it needs to be credible, because the fact is is that if we don't, someone's gonna take it away from us. And so that's where we come in, the ABPN. The ABPN is the certifying board. It is that independent board that we need for certification. Now, as you can guess, and certainly my experience on the board and to the point that you've been paying attention, it's not static. Things change over time. It's a flexible process there. We're always changing. We're always trying to make things better. And by better, I mean more flexible, more relevant to your practice, and less of a burden. An example, of course, and what you've already heard about and what I'll say a little bit about again is about the ABCC pathway, the article-based pathway as a alternative to taking an exam. And of course, we continue to change and we're always open to your feedback. You'll find that in everything you do at the ABPN, every process you have, every time you do an article, every time you take a test, there's always a part for feedback, and I do wanna let you know that we as chairs of committees and as the boss and everyone else, we take that feedback very seriously and do read it, and I can promise you that changes have been made based on that feedback. So if you have to, remember one slide, this would be it. No, you don't have to memorize it now. It's on the website. But this is kind of the whole process in a nutshell. And you can see it breaks into several different points. First, you have the activities. There's certain things you just have to do during a three-year period. So because certification after you take your test is broken into three-year blocks. So there's some stuff you have to do. I'm gonna say more about these. It's gonna get broken down on the next slide. Then you also have to be assessed, pay an annual fee, keep up a valid medical license, and voila, you're certified. As far as those activities, well, there's the things that you hopefully are doing, things like CME. So we're all here, so we're getting CME. Self-assessment, once again, for the reasons that Jeff said and I think you know, it can't just be, I mean, I'm sure the residents know better than everyone else. I mean, how much do you really learn from a lecture? Just sitting there passively, depends. Sometimes I learn, and sometimes I'm just kind of sitting there and, of course, looking at my phone and things and stuff. So there needs to be an iterative process as well, and that's why not only do we require CME, but self-assessment CME as well, and then, of course, there's also performance and practice activities that you have to do during that three-year period, which we can go into more if you wanna hear about, but it really boils down to different kind of QI activities and the short story is, if you're working for a health system or a group practice or any other kind of organization like that, you're probably doing those things. Then there's the assessment part, and once again, you have a choice. It's either two different things. Either the article-based pathway that's rather new or the old-fashioned 10-year recertification, which was the standard before this. Then also, you have to, once again, like I said, keep up a license and pay your fee. I remember the point of this all is that it's really trying to be flexible, and it should be, because we all learn in different ways and at different paces, and by flexible, I mean that you have different kind of choices of different ways to meet the requirements for certification. Now, my bias, and I think the bias of everyone here and stuff is that the ABCC is much better than taking a test for learning, and I do think it's a better just pathway for learning and assessing, but you do have recertification as an option if you do wanna take the test, and the whole point of this all is that it should be clinically relevant, so you should actually find it helpful to what you do, and it also should be a sense that you have some choice in this matter, that you actually get to pick and choose how you wanna learn, and you do. You get to choose your own CME, you get to choose what kind of self-assessment activities you do, you get to choose your own performance and practice activities, and you get to choose topics and articles in the ABCC, of which there's quite a few choices, actually. Just to kind of word about the ABCC, so once again, as we keep talking around it, this is the alternative to taking a test, where instead of doing that, you can read a bunch of articles in the three-year period. How many that is kind of depends on how many subspecialties you have and how many certifications and things, but it's around 30, usually, or 25, depending, and so you do that over three years, and then you answer these really brief, open-book test spot. They're not timed, they're open-book, and so really, it's just sort of testing, did you read the article? So far, the feedback about it's been good, so we put this up here to kind of, not just to brag, but to reassure. Once again, we do ask, after each exam, like each mini-test that people take and stuff, what their experience was. We do survey people all the time about it, and you can see that it's pretty positive. People find, number one, that the articles, they can get them, which is good. We don't provide the articles, because that would be too, it would make it a very expensive process, but it seems that people are always, we try to pick articles that are easy to access. The article, people do find that it's actually helpful. I mean, I participate as well, and I must say that it really does force me to learn parts about my field, which I think I'm an expert at, but it keeps changing. There's things happening that I'm not aware of. People do feel, actually, that the test is fair, and that's an assessment of their understanding. We worked hard on these tests. It's different, it's a challenge to write an open book test, versus the typical closed book one, because you've got it in front of you. So, and these days, I think, for me, the biggest challenge when I look at questions and stuff is, how do you write questions about an article where the answers aren't all in the abstract, or sometimes they're in the title? And actually, when I write articles, that's a good thing. You kinda wanna have it all up front, knowing that not everyone's gonna read every detail of your article. But all the same, we have to look through, so we have to do that, and yet not let it be just like silly trivia that's not very important. So it is a challenge, but at least the feedback so far is that we're doing a fairly good job. People find the questions well-written, and overall, the experience, you can see, 93% is satisfactory. If you're an instructor at your institution and stuff like that, and I can speak for myself, I don't get ratings like that. So that's a pretty good thing. Once again, they've already made reference, both Josefa and Jeff, to the fact that we have a website, and we keep all the information there. So when you get into this thing of, well, what do I actually have to do? This is fine and well here, but just tell me what I have to do. How many articles do I have to read? How many, when do I have to take a test if I have to take a test? When's this all due? When's my three years up? Well, simple enough. You just go to, oh, that's cute. You just go to the Physician Portal and log in there. If you've never logged in before, you'll have to register, I think, but once you do that, it's a very simple process. Then you're on there, and it tells you everything you need to know. Here's an example of what it's gonna look like. You can see, it tells you what you're enrolled for. So this anonymous person, is psychiatry and geriatric psychiatry. You can't imagine who this might be. And then it tells you how they're doing their assessments. That person's doing better than me. Well done. I still have about, I think about, I don't know, maybe 10 or so articles to read before December. Continuing certification activities tells you how you're doing with that. Those you self-enter. Sometimes, some organizations actually do send them in. Most of the time, we have to do them ourselves and put them in. And then it tells you what your status is. So simple enough, right? The most important thing is that, you know, the organization is trying to be helpful, and it's really trying to be available. My experience as a user, because once again, you know, everyone on the board is also a user, is that the board's very responsive. If you have a question, call, email. They're very good about getting back very quickly. You can also find out a lot of things you need to know is right there on the site. Everything I said is there. In fact, I think pretty much almost everything you've heard today, you can find on there. So the information is easily available. But if it's not clear to you, do do that. And since you're here, also come to the booth if you have questions. What's our booth number? 818, right. So, you know, we'd love for you to drop by and give questions, thoughts, suggestions, anything else like that. I'm gonna turn it back to you, Jeff. Thanks, I'll just add that in terms of coming to the exhibit hall, the folks at the booth have laptops and can look up your portal or get you registered if you're not registered. And so they can answer very individual questions that you might have about your own status or your own needs in ways that the three of us are not equipped to do. So that's both. But they can do that at the booth. And then, as Bob mentioned, email or call. I think a lot of people hesitate to call us. I think it's, I don't know, for some people it might sort of feel like I'm going to the principal's office if I have to ask them a question. That's really not how we view it. And our staff who person the phone and the email are wonderful. And we promise get to get back to you within two business days. It's often a lot faster than that, obviously depending on the demand. So with that, thank you very much for your attention. And we're happy to entertain questions. But thank you very much. I will ask, we were asked by the organizers to please use the microphone for questions to make sure that folks who are watching remotely or watching the recording can hear the question. And just one statement is that the slides that you see, they can be downloaded through the APA app. If you've got the meetings app, it's actually the handout attached to that. So you can download it straight to your phone. Keep it forever. At least until they change. So we do have the microphone set up for questions. Yeah, Randy, Dr. Weston. So I greatly appreciate the efforts you've made. I'm old enough that I came in, just missed out on the lifetime certification. You know, you pass the test and you're in forever. I just missed that. But I mean, the article based one is so much more likely to lead to constant kind of improvement. That is good, but it's still testing knowledge. Are there any thoughts about how we test skills, how we test interactions with patients? And I realize it's an incredibly complicated, difficult thing to do. I don't have a solution in mind. I'm just wondering, because we're kind of stuck to testing knowledge, which is important, but it's not the only thing about being a good provider or a good physician. Great question, something we think about a lot as a board, and you folks may want to join in as well. I will say, I think that our examinations, both the article assessments and the other examinations, test reasoning skills in addition to pure knowledge, that's application of knowledge. At least that's the goal. And I think, obviously, I'm biased about that, but I do think they get at that in fairly useful ways. But of course, you're right, there's lots of other skills as well. So for initial certification, that's why we depend, among other things, on ACGME accredited training programs and the CSEs that happen, among the many other things that happen during training. So that just because, I mean, anybody who studies enough, there are many people who might be able to pass their exam if they studied long enough without having done that, but that doesn't make them a psychiatrist. For continuing, sir, we wrestle with this, right? And how do we come up with something that, because we could do that, right? I think we could come up with a, what, 12 or 15 or 20 station OSCE, Objective Structure Clinical, you know, thing with standardized patients or vignettes and a whole bunch of, say we could do that to pull that off for our tens of thousands of diplomates and what that would, because we have to, you know, our fees have to cover our costs, we'd have to pass that along to everybody. So we continue to think about how technology might offer opportunities for that in the future to try to get at those domains better, but we don't think that anything is there yet that's ready for prime time, unfortunately, although that could be different next week, but at present, but you folks must know. Yeah, I mean, it's really just the same, but I'm on one of the committees for the American Board of Medical Specialties doing certification. You can imagine all the specialties are asking this question and have different approaches, different ones are trying to do it. A lot of them, but you know, as you know, most boards have moved away from like direct observation activities, which used to be the standard for that. And I think it's probably just as well, because, you know, I always, did you take, Randy, the oral boards? Yeah, and it's terrifying and just hard to believe that someone sitting with you for like, you know, one interview can decide if you're a good doctor or not, right? So, I mean, so we are looking, I mean, so we're also trying to learn from some of the other boards as well, who are, there are some pilots going on of different kinds of things of like, you know, more like tabletop simulations now, not computerized, but you know, who knows? Do you want to go down to here? I will add that competency-based medical education has been a real push from the ACGME, and there are a few specialties that are further along in that, a lot further along in that, than we are in psychiatry or in neurology. Further along, meaning they also, the ones that are now implementing this in residency training for pediatrics or family medicine or surgery, started this process over 10 years ago, and they're sort of just now adopting this in training. I mentioned that in relation to your question, because if the promise of CBME is lived out in reality, we can imagine that it might, I mean, and I will say ACGME and ABMS are both very much imagining that it might be incorporated into continuing certification in some way, but there's a lot of ifs along the way, right? Do we have valid assessments? Are they feasible, even in training? And then are they scalable for people in practice? But these are all things we're wondering about, so nothing like that's going to happen anytime in the next few months or even the next year or two, but we are investing in trying to advance CBME in training, and probably, at least initially, is perhaps as part of formative rather than summative assessments, so rather not pass-fail things, but lifelong learning might be a next step at some point down the road. I'm happy I'm following up on your question, because I had a very similar kind of thing. I'm going to lay out my bias right up front. I'm a technology-minded psychiatrist. There are some of us out there. I try to stay in the field. When I was working with Bob Spitzer on the DSM-III, we wrote the DSM-III on IBM's electric typewriters. We correct the race. And we drove the secretarial assistants nuts by doing that. We eventually gravitated to computers and word processes and made use of the technology. I very much like Dr. Boland's remark about airline pilots or whatever, but I could simplify that a little bit. If you go to take a license in any state, you have to show that you know how to drive a car. You could take a written multiple-choice exam, but until you get in a car with an instructor sitting with you and drive it, you're not going to get your license. In the old days, this is the way we used to do it. And I'm old enough to have been grandfathered into this, having gone through the frightening board process, then become a board examiner, then become a hired gun to help people pass the board examinations and things. And one of the bones that I have to pick is we've lost a little bit of something by not having that in the initial certification. Because as good as you could answer a multiple-choice question with your database, it doesn't show that you could do a 30-minute interview. And after the interview, did you know how to present the case? And following presenting a case, did you know how to respond to an examiner's pointed questions about things that came up in the case? That's different from answering a multiple-choice examination. The chatbots that we have out, even the cheap ones now, are good enough to pass any multiple-choice exam. There was a publication in JAMA Neurology just recently that spoke to that. It showed that it passed out of 75% to 80% of the multiple-choice questions. I haven't seen that replicated in psychiatry. When I wanted to go to replicate that, I couldn't get anyone's question set because they're all proprietary. It turns out later that the people that did neurology, they didn't get any permission to do this, by the way. They just took from one of the programs, I won't mention it, but they just took their whole questions and put it there without getting permission. And I know this because I talked to the people that made up the questions, and I asked them if they obtained permission, and they said no. And then I asked the senior author about whether or not you got permission, and he never got back to me. All right, so it's very hard to get these proprietary databases. My point being that in your mission of the boards, I think it's number six or so, it says make use of technology in advancing the field or whatever. And I think I disagree a little bit. I think the time is now. And in fact, there's never been a better time in terms of we're at the early phases of artificial intelligence. And just as one suggestion, you have 18 people making questions up that are good questions. You could replace those 18 people with one chatbot, and I would challenge you to show anyone which question set is better. And most of the Turing tests that have been done, they show they can't differentiate what's been done by a live person versus what's been done by a chatbot. So you could have wonderful questions all created with a chatbot if you wanted to go in that direction. So my point is I think now is the time to start. And you have some very talented people who have engineering backgrounds on the boards because I looked them up. To harness that ability and harness the technology and start doing some good simulations. You can actually have simulated. You could use video. You could do a lot different than just multiple choice questions. And with that, I'll leave it to other people to ask questions. Want to comment? I don't think there was a question. Yeah, thanks for sharing. I have to say, I play with chatbot. I don't think it actually does as well as us yet. At least the version 3.5. And not only that, but I'm not convinced that a lot of them aren't just copying from something. Depends on the database. Well, yeah, it's whatever they put into it. But ultimately, you're right. Can it do those kind of things? Sure. If not now, it will. So I get your point. And I will tell you that we, like the other AVMS member boards and most organizations, we are working on internal projects using computer-assisted technologies, including AI. When such things are ready for prime time, whether it be behind the scenes and helping us generate or classify questions, I personally think they're short of not needing human input. But they probably can help us. And so we are exploring all those things. And we'll be sharing that with you all as we think they're ready to adopt for us. Hi, all. I was very interested, Jeff, to hear your initial sort of layout of what we know about lifelong learning and expertise. And one of the things I've always been interested in as an educator is at what point does prior learning accelerate the process of new learning? And so, for example, one of the things I hear from child fellows, and that's primarily been my educational experience, is that people tend to teach them as if they don't know anything already. And they say, of course, we do. We've been through medical school. We've been through general psychiatry training. And although we may not know the details, we know how to approach these kinds of problems. We know how to think about them and that kind of thing. And it seems to me that competency-based education will need to take that into account somewhat, right? So, for example, somebody who maybe wanted to retrain and get new certification in a subspecialty is different after 20 years of general practice than they are coming out of medical school. So I just wondered what we knew about that and if there are thoughts about sort of integrating that into how we approach lifelong learning. Thank you. I mean, great, great questions. Known to many of you, but Sandra DeYoung, who's not only active with the EPA, but is also on our board, one of our directors as well. Yeah, I mean, great, great questions. I don't know that we know the answers to your question in a systematic way. You may know some literature that I'm not familiar with offhand, but what you're raising is really important. I am thinking about our clinical skills in psychiatry. People, they're known as the CSVs, what we call the clinical skills evaluations required. And they're supposed to be at the level of a practicing psychiatrist, which means that most residents who do this at the PG1 or 2 level should not pass most of them, right? But how many of you have, I don't have to say if it was you yourself, but how many of you have known somebody who took a CSV that failed, particularly earlier in training? Because we actually would expect, I mean, I'm hoping. Yeah, sure. So we're hoping. So at least in a very broad way, now, whether that happens as much as it should, given what the standard is supposed to be, but what does that mean at the level of a practicing psychiatrist in practice, right? It's a very broad, very oversimplified attempt to get at that in a way that has to, I think, given what the point of the CSVs is. So then if we think about it in lifelong learning context and people in practice, right? I mean, if it's an older patient with depression, what you would expect from actually any of us might be different than somebody whose work doesn't touch on that very much on a regular basis, right? So I think you're asking a really good question. I don't know the answer. And I don't know that there are systematic studies or data to try to answer it yet. Anybody else have any thoughts? I mean, we need all of our heads together to try to tackle these complicated, what some of my colleagues like to call wicked issues in science and medicine, right? So not in the sense of evil, right? We all know the term. Boston wicked. Yeah. Steve, just a microphone, please, for folks who are not in the room. Yeah, just to update on that, one of the difficulties we were talking last night, on the addiction exam committee, one of the ways we code the questions on the exam is according to its placement in the broad field of knowledge. But there's no coding in terms of difficulty. So there's no way at this point for us even to take all of our test bank and say, here are the easy questions, here are the tough questions. And that would be, and we were thinking last night, Jim Berry and I were trying to put together an idea as to, how would we do that? How would we even make that assessment as to whether a question is easy or hard? And it would be one person doing that. But that would be an approach, at least to start going in that direction. First off, I got to thank Stu for being on the addictions committee as well as on the item reclass. We just went through looking at thousands of these. And in a way of speaking, we do have, the ABPN does keep stats on every single question. And in particular, on two vectors, if I may say, the Pbis and the Pval, in terms of what percentage of all test takers get them correct. And then looking at the Pbiserial, which is a comparator, in terms of which gives us, in a way, the degree of difficulty of a question in terms of the folks who did the best on the exam overall. Did they get this question right? But more importantly, did they get the question wrong? Or did the people who scored the worst on the exam or had the lower scores, if they got the questions right, that folks who did well did poorly on, have to look at the question in terms of, was there a difficulty with the scoring? Or is the question invalid? So was it miskeyed or the question invalid itself? So it's a really interesting dynamic. And the person who can speak to that most, I wish is here, because he gives a fascinating lecture about this, is Linjun Sun, who is the director of exam development, the VP for exam development. But it's a fascinating thing. Next time you come up, we should definitely look at that. Because I've always been curious, could you skew the test in terms of, what's the hardest thing? What's the hardest test? And then we give it to people for fun, if anyone would take it. How well would the best people score on that? Because that gives you the Olympic judging degree of difficulty, in a way. But that's my take on it. But I think it's a great question. I have another question. And the answer might be no. And then I'll just go sit down. So I know you have enough work trying to ensure the competency of all the psychiatrists that are out there. But a lot of us are also responsible for competencies of other professionals. We might be working with nurse practitioners, with PAs, who are working under us and with us, or family medicine. Is there any conversation between you all and the groups that certify them to see that everyone in the mental health clinic is competent, not just we have really competent psychiatrists and then we just hope for the best with everyone else? Is there any conversation or interaction there? It is a really good question. The short answer is basically no. But it's been thought through. And in the end, the role of the specialty boards, it is the American Board of Medical Specialties, right? The focus is on physician skills. There are lots of reasons why we might think about such collaborations as you're intimating. There are also people with lots of concerns, right? Feeling in competition with folks from other disciplines for some of the same practices in their communities. And if they could pass our exam, well, that just shows that they're just the same as a psychiatrist, which obviously the exam is part of it, but not all of it, right? For reasons we've already been saying. There are a couple of other specialty boards among the ABMSM reports that do certify people who are not physicians, but they are not clinicians who have the same kinds of overlaps of scope of practice as the clinicians that you're alluding to. So medical physicists, for example, are certified by the American Board of Radiology. So PhD physicists, but not providing direct patient care and not competing with or overlapping with the skill sets of the physicians. So anyway, the answer is no. It's been a fair amount of thought about that. But in the end, we both at ABPN and the ABMS community have decided to stay in our lanes for a variety of sociopolitical, but I think not unreasonable, reasons. Having said that, our content outlines are public, right? So not the questions, of course, but the content outlines are public. And anybody is welcome to take a look at them and adapt them as they see fit for their field. Let me just follow up. The knowledge base is a moving target. It's not static. How many people know what the age of recommendation for a person to get a mammogram is? Is it 40, 45, or 50 where you get a mammogram every two years? What is the age? Does people know that? Because that could be a question. OK. So what is the answer to that? I just heard on the radio today, they just lowered it from 50 to 40. OK. So if you took the exam a month ago, the answer would have been 50. You take the exam today, the answer is 40, or maybe not, depending upon what's going to happen. So this is why it's so important to know that it's a moving target. And you have to have the latest, most up-to-date things, because what was the state of the art 10 years ago is no longer the state of the art. It's no longer the state of the art. And that's where, I guess, continuing and furthering your education comes in handy. Well, how do you best do that? Yes, you could read articles. But nobody could be so good to read all the best articles, even in their own field or whatever. The whole idea is you could use technology to get that information, the most up-to-date information. And the bonus attention I would have is the technology exists now that you could precisionally, for an individual patient, get the right information on multiple, multiple levels. So just getting back to a question on something like mammography and breast cancer, it's not a one-size-fits-all. Can I ask if you have a question for us? Excuse me? Do you have a question for us? The question is, using technology, you already have the means to run all of your exams through a chatbot and find out what the difficult questions are and what the easy questions are. You could just run them through a basic chatbot. We can do that already without the chatbot. Okay. Well, but the thing, the chatbot will do it in 30 seconds. You don't need live people. And if you just want to do a little thought experiment to that. Stephen, if I may interject. Sorry. No, if I may interject. Part of it is that putting anything that's copyrighted into the chatbot releases it into the world, which means that any time the security of the exam is compromised. And so feeding, and I'm just speaking on my own, with my own rudimentary idea of how computers work, the World Wide Web works, chat boxes, et cetera. And it's basically a furnace that you feed information to. And everything you give it, okay, is transferred into something else, into a different kind of energy that's out there. And there are people way smarter than me every second who are able to pick and choose what they want coming out of that stream of zeros and ones. And so we do already know, we do our own metrics. And again, because of issues of security and confidentiality, those questions have to stay secure. And you know, the New York Times has a major lawsuit going against OpenEye. Yeah, yeah, we know all too well. But I'll just say, I wanna go back to a point that you made because I think it's an important point. And it's why we do what we do is that things change and we need to stay up to date. And I think that is an important point. And you know, that's why we have committees and that's why we don't just rely on ourselves. We do have experts in the field, including some here who sit there and do that. And yes, that does happen that people can say like, you know, this isn't true anymore. And now with that, we've got that. We've also got feedback from you all when people take the exam. If they have a problem with the question, they tell us and we do take that seriously. And then the third thing is that, that Arbus you're talking about. Every now and then it seems like really smart people seem to be getting a question wrong. And then we actually take the time to say, well, why is that? Maybe they know something we don't. Every exam just to say, after there is a post exam review of every single item and the performance of the item and the performance of the candidates. It's this constant cycle and it's, I don't know how else to say it, that having multiple sets of eyes on every single thing that comes through. With the exams, have you taken any of the computerized exams recently? Recently, no. Okay. The last time that you took one, you would have seen that there's a box that says, feedback literally on every single question and we get feedback. Some like, oh, this was a great question. This was interesting or this is wrong, just as Bob was saying. And so in a way I agree, but it's a constant process. I mean, if anything, being involved now over eight years, actually 20 years of writing questions, I thought that by the time I got on the board, I had an idea like what a good question was. Turns out I had no idea how much time it takes to write a good question and then see it perform multiple times and then have to revise it. And it is this constant and continuous process. I have six or seven committees and I and a vice chair read through everything multiple times before that ever gets on a test. And then even after the test, when I think that's the last time I'm seeing that, several months later, it comes to me and said, here's your form review and I'll look through every single comment. How do you know the questions aren't already out there? Hopefully you haven't put them out there. No, I haven't put them out there. Just like the New York Times and OpenAI were saying everything is out there. In some of my own experiments, I found things are out there, but the chatbot in the program could deny it. I guess in the end we say we do our best. And we hope for the integrity of the diplomats and candidates that take the exam. That's actually a point. We have a basic trust in the professionalism and integrity of our candidates. We don't, you know, like we ask people, did you do this CME? And they say they did and we believe them. Yeah, yeah, occasionally we audit. But mostly we believe them. So, I mean, we have to believe in our own professionalism. Otherwise, we should not be allowed to certify ourselves. We'll say that some of the other member boards have, all the other member boards have continuing certification programs that are ongoing. Some, they don't all use articles. Some of them use timed questions that you have to answer, multiple choice questions that you have to answer a certain amount of time, then move on to the next one. And some of those boards have seen, not surprisingly, those questions circulate, because they're administered at, you know, distributed across the web. Our article exams are open book. We know that people could share the questions, but people need to attest every single time they take an article exam. And we encourage people to use journal clubs, talk about the articles with each other, but we say, do not discuss the questions, and people have to attest every single time they take an article exam that they've not, as you say, trusting in that. I would say just one other quick thought about what's actually a really interesting big topic, another part of what you raised. People using resources, web-based resources that we've all been using for years in our practices on a day-to-day basis, and now increasingly will be informed by AI tools, there are fields of medicine, including particularly some fields in neurology, where, you know, reading electroencephalograms is augmented by computer reads, for example. We're gonna need to, we are starting to think about how do we incorporate, how would we incorporate assessment of that into our assessments, given that that's part of clinical practice, right? And the articles is actually a step in that direction, right, can we read and understand an article and apply it to a clinical scenario? It's a small step, but it's an important step in that direction. Can you access it? That's right, that's right, but we're gonna need to wrestle with that even more in the coming years, as part of being, because being a physician involves using these tools, right? So, Sandra. I just wanted to follow up on Randy's question, because I think it is a really important question, and I think one of the things I certainly see is that psychiatrists go through a very rigorous process of training and initial certification and continuing certification, and when they get into the workforce, they see that other mental health professionals don't necessarily go through that kind of rigor, and they have feelings about that, unsurprisingly. I guess one of the things I just wanted to communicate is that, first of all, ABPN isn't the only means of trying to build skills and certify those skills, and one of the things that we've piloted at one of our affiliated mental health clinics is actually one of our graduates has started a program for the non-MD mental health clinicians to give them a sort of a tiered approach to what's informally sort of certification. So, in other words, when somebody comes straight out of, an allied health professional comes straight out of a training program, they aren't given the full range of responsibilities. They can earn them by going through various kinds of educational and assessment processes, and they did that all on their own, and it's their way of trying to assure that the people that they're working with, they're comfortable working with, and I think there's a whole range of things that we could do as psychiatrists, really, to partner with allied health professionals to help them develop skills and these kinds of programs, but probably not through something of the Board of Medical Specialties. Yes, oh, can you use the mic, though? Sorry, we're being live streamed, apparently. Good morning, I unfortunately arrived a little late, but I was really looking forward to learning as much as possible as I just started my medical journey. I have a question regarding, I'll just give an example of, because it keeps haunting me. My aunt was, you know, was one of the first neurologists in Kuwait back in the 80s, and when she was in medical school, H. pylori was not known to be the reason to cause ulcers. I remember. Yeah. And she became a neurologist, though. She didn't become a gastroenterologist, and till this day, I'm thinking, I do not wanna be a neurologist that does not know that H. pylori does not cause ulcers, and we can take that into, nowadays, well, when it comes to other fields, not just psychiatry, so definitely, I want to be certified in the American Board of Psychiatry and Neurology. I wanna be up-to-date with what's happening in this psychiatry, but my question is, how do you manage to keep up with the H. pyloris of the other specialities? Sign up right here. All right. Thank you for a wonderful. Thanks for a wonderful question. That is actually part of the process of being ABPN certified, is continuing certification through the process of the article-based program, and so, once you reach initial certification, you're enrolled in continuing certification, which is, you have a choice. You can take another big exam, like you always do in med school, or you can enroll in the article-based continuing cert program, which allows you to select, and these articles are selected very carefully by a committee for each program, which, and we even have an, we did this for geriatric psychiatry. We fast-tracked an article that we thought was so significant, rather than waiting for it to go into the next three years, we actually fast-tracked it to go live now, because it's on the, on Leucanomap, and we've gotten such a great feedback, like, wow, I was, this is what I was looking for. Walking up and down, in terms of this meeting, having folks tell me that, it's kind of amazing. I just read about this, and two weeks later, it came into my clinic, and that's exactly why the article-based continuing cert, it's almost like having your own private reader, research librarian saying, hey, this might be of interest to you. Does the work for you. Yeah, that's, that's what we do. Thank you so much. I just say the premise, though, is important when you're saying, and I think you're kind of implying the answers as you say it, I mean, when you become a doctor, you're signing up to be a lifelong learner. Exactly. Right, exactly. That's a good thing. Sometimes it's a bad thing. It's a good thing. Thank God our field changes, right? Yeah, I think. And look, you're here at the APA, so obviously you're embracing it. Yeah, thank you. If, I don't know if this is what you asked, what also we're asking, but I'll take, I'm gonna answer a question that I heard implied in what you're saying, too. What about something like H. pylori, right? So our article pathway would have something in it if it was the psychiatric or neurologic implications of H. pylori infection. But what about something more fundamental, right? H. pylori and gastric ulcers, which would not be in our article pathway, but I will say that in terms of continuing certification, we don't, unlike some of our boards and other specialties, we don't tell you what CME to do, right? Or what conferences to go to, or self-assessment CME, or what you do for performance in practice, as long as it's something relevant to what you do. So we actually, so the articles, we curate and try to keep it related to the fields in question. But the rest of one's lifelong learning, it feeds into continuing, so we have the freedom to choose whatever we want. In practice, you may not have been here when I mentioned earlier, but so I use RSS feeds to help scan table of contents for journals. And JAMA and New England Journal and BMJ are on that list, right? And I subscribe to the medical letter, maybe anybody here know the medical letter. I still subscribe after all these decades to the medical letter, which is drugs across all of medicine, right? So we each have our own, should need to develop our own methods of what are we gonna read, what are we gonna try to keep up with or stay connected to? And then certainly you can use them for, as part of CME or other activity requirements for continuing certification with our board. Thank you so much. Thank you. I think it's a great question. The excitement of psychiatry is that it's where medicine was 50 years ago. And the thrill of it is to figure out what these diseases really are because DSM approaches it from, these are the behaviors people have when they have the disease rather than this is what causes the disease. So the excitement about it is really what the future holds. And I can't tell you how many times we go through questions in the question bank that were written five years ago that we throw away because we look at them and we say, well, this isn't right anymore. Thanks. Thanks. Hi. So I have a quick question, a burning question about the PIP requirement in the continuous certification process. So just for context, I took the board exam in September. I passed, thank God. And now I don't wanna take the exam again in 10 years. So I just wanna make sure I understand the CC process. Thank you. So the activity, so there's two pieces of it, right? There's what, that's in that diagram that, actually, maybe I should call it up. Maybe you should call it up, yeah. Because, yeah, bad news, you gotta do it anyway no matter what you pick. Let's see if I can test my vision here if I can call it up. Did I get it? Okay, great. Yes, I don't. Not scrolling it either. So, I mean, obviously, the print may be a little fine if you're not up close to the screen, but this is on our website. This is the basics of the program, right? So the three-year wheel of activity requirements starts the year after you pass initial certification in psychiatry or neurology or any other primary specialty, right? So, and it covers everything. If you were to get a subspecialty certificate, it's still one three-year cycle for CME, including self-assessment CME and the PIP. And you have to do one PIP-type activity every three years. The assessments, the article, or the 10-year recertification exam is a separate piece, right? And that obviously is for psychiatry. And if you have a subspecialty, there'd be a separate assessment article or exam for that, right? So it begins the year after you pass the exam. And you were all nodding congratulations, so congratulations and welcome to the community in that way. And then for PIP, it can be a kind of classic PIP, performance and practice quality improvement product, but there are many other ways to satisfy the PIP requirement, which we won't dwell on the details now, but there are on the website and also available if you wanna talk to us either by email, phone, or coming down to the booth in the exhibit hall, booth 818, have I got the number down, Patty? It's a, please feel free to ask us questions. But I will tell you that among the most popular ways that people satisfy the PIP requirement is feedback modules. If you teach residents or fellows, and if they evaluate you, that counts, right? If you get evaluated by your peers, formal peer supervision, for example, if you have a supervisor who evaluates you, we have forms available on our website they can use to evaluate you that incorporate the core competencies and so on that count as PIPs. So there are many different ways, and you only have to do one of them, right? Any given three-year cycle. So there's actually quite a lot of choice. So anything's not clear as you look at our descriptions, just please ask us. So thanks. If I may follow up with a question. Where do you work? Do you work in a hospital system, private practice, or a VA? I'm actually at LA County. So I'm at the Department of Mental Health outpatient. Okay, so there are gonna be requirements for being involved in a peer review process where you'll be reviewing sets of charts on peers. Your charts will be a certain number, like 10 every four months, because that's gonna be part of the requirement just for the hospital itself. And so that counts. And so that's an example of, there's probably something you do, but please ask or come to the booth. Be glad to go through that. I wanna understand, because I think other boards, like, for example, I heard, I think Family Medicine, they have a similar activity, and they have to, they can make their own little project and then just kind of run it in their clinic and then just report the results to the board. So I thought it was something like that. That's their performance, yeah, their QI process performance improvement. So different terms, and there is that avenue through, but you just have to contact us and we can look through that. Okay, so it's flexible. Yeah. Yeah, so if you, in fact, are doing something that doesn't seem to fit any of their boxes but does seem to meet the spirit of what we're after with PIP, just please ask us and it may well make sense. We just approve it as an approved PIP activity. It's pretty easy to do that. For most people, there's things that we're already doing for our employers, and it's probably the hardest for people in kind of solo or kind of fully independent small group practices who maybe may have less of these going on, depending how large they are. But people who work in health systems of any sort, there's probably things you're already doing or required to do or doing because you wanna do as Bob said earlier, the principle is we'd like to give people as much credit as we can for things that we're already doing that meet the spirit of these things. Oh, yeah, and one last plug for the APA. I'm on the program committee. There is, you can take a self-assessment exam just at no extra cost as part of your registration here, and that fulfills eight credits of the self-assessment. So that's a good way to do it too. Does the eight credits for the self-assessment count within like the total APA program, or is that like a separate? It's, I think they give you CME for it too. Yeah, so it's all part of it. I think if you attend everything that you can, you can get up to, I think, 24 CME credits. Is, I think it's 24, 26, but it's, you get a lot. If you attend everything that you can possibly do by attending the sessions here, out of 528 sessions, you can get up, you don't have to attend all 528. I'm just saying that, you know, if you pick, there were only two things that you attend that don't count. I think the convocation doesn't give you credit, and I don't think the opening gives you credit, but everything else gives you credit. I know we have another question, but I also will just say that we kind of touched on continuing certification in broad, fairly broad strokes in this presentation, given the focus on lifelong learning and people who are in training or just finishing training, but for those who want to hear more about, even more details about CC, we're actually doing, the three of us are doing a session on Tuesday, Tuesday afternoon, I believe, and we'll be talking about continuing certification in some more depth, so please. Hi, I have a little bit of a bigger picture question, and I know you talked a little bit about sort of reasons for lifelong learning, but as a fellowship director, I find it increasingly difficult to inspire our trainees, to embrace the idea of self-assessment and lifelong learning. I think that for many, this idea, like the sort of culture of wellness, and my wellness versus spending more time reading and learning, I think that the two of them often don't always align in the minds of some of my junior colleagues and trainees, so I have seen your videos, Jeff, and sort of given presentations to my fellows about some of those topics, but I'm just wondering if you all have more global advice about how to just inspire that as a culture on your clinical service or in your training programs? Certainly, it's a good point. I mean, obviously, people are balancing many things in our lives, and this is certainly on the minds of our trainees. I know that, I don't know how much this is happening in one-year fellowship programs, but certainly in residency and at least some fellowship programs, engaging folks in performance, quality improvement activities as part of the training, right? And maybe they just see that it's something else to do and another burden, but if it actually produces tangible, to them, improvements in what they're doing or what care the patients are receiving, one hopes that that would give them some sense of what we're aiming for with some of these kinds of things. But I'd be interested whether others have experiences with how to engage folks around, because in the abstract, it all sounds fine, but in practice, for some people, it's gonna be a challenge. The VA actually provides a salary bump if you are certified and in continuing cert. So, full disclosure, I'm with the VA, too. So, I'm stumping for everything I represent here. So, part of the ways is that it is being able to be certified in itself is a credential, but it's also, it's a qualification, it's a sign of quality. And it's something that hospital systems and local societies actually recognize. I think it's part of also being that lifelong learning and being able to be certified I think it's part of also being that lifelong learning. I understand that there are pressures. I mean, I feel it, I'm sweating about making sure I get through all my articles by the end of the year. At the same time, to me, I tell my residents it's a sign of pride for me that I am certified and that I am actively trying to make myself better for my patients, because ultimately, that's what it's for. It's really for my patients. It's not for me. I'll just add, I do think it's really challenging when you only have your trainees for a year. I have them for two years, but they're also at the end of their medical school and residency, and they're tired, and they, she's nodding, yeah. I mean, it is definitely, I think, more challenging at our end of the training spectrum. I guess one of the things that I have found interesting in the whole wellness movement is this sort of work-life dichotomy that has been set up, and as a psychiatrist, I think of work as an integral part of a gratifying life, along with love and play in the Freudian sense, and so as a teacher, one of the things I try to do is to imbue lifelong learning into everything we do, so I actually, in my teaching, if there's something we don't know in a seminar, we pause and we look it up, and similarly, I have didactic sessions where everybody goes and learns something and comes back and does peer learning, and I actually think ongoing learning is a key gratification piece to what we do, but the trainees don't see it that way yet. I think they can get there just in the same way that I think teaching is one of the things that is just endlessly gratifying as one goes through one's career, so I would love to see a sort of shift away from the notion that any extra work is inherently burdensome and negative, and consider the possibility that a little extra work can actually be an enormous amount of gratification. Yeah, you're reminding me of literature supporting what you're saying, right, that it's people who are more well in their work and lives as physicians are not necessarily working less hours, right? They may sometimes be working more hours, but they're more engaged with what they're doing, and the variety and human engagement and content engagement parts of what we're talking about are the things that we, and I'm also thinking about role models, right? Our trainees encounter folks who've been in practice for a number of years or decades and they're still very much engaged with their work and they're people that they still respect. Well, how did that happen? It didn't happen by accident, right? It happened through some reasonably planful approach to being that, and I guess what hopes, and they don't have examples, but perhaps they have examples of people who've been in practice a long time who are not at the top of their game anymore, and so helping them, so what's the difference in career path here? Where do you want to be? Well, yeah, we really appreciate your questions and thoughts and engagement, as we were just saying, and thanks for being here, everybody. Thank you.

lifelong learning

specialty board certification

psychiatry

neurology

Jeff Linus

Dr. Josefa Chong

Dr. Bob Boland

American Board of Psychiatry and Neurology

medical knowledge

Continuing Certification program

self-assessment

professional development