Welcome. We are in that down phase right after food and early afternoon session, so I know everybody is sort of kind of tired. Maybe you've had your coffee to wake you up, get you through the rest of the afternoon. But we are going to go ahead and get started. And I'm going to, I have the pleasure of introducing our other folks today. I'm Shambhavi Chandraya. I'm the Program Director at East Tennessee State University. To my right is Dr. Jessica Kovacs. She's the Chair of Psychiatry at Temple University. And to her right is Dr. Daniel Gee. He's the Program Director at the University of Nebraska Medical Center. And to my far right is John Spohn. He's the Associate Dean for Northwest Arkansas, University of Arkansas for Medical Sciences. We have other titles, but those are the ones that we're sharing with you at this point in time. And as the title says, we are going to talk about how to get into a, how to make a successful entry into psychiatry residency, helping folks who are involved at a different process level than advising their students in terms of entering psychiatry residency. I'm going to turn it over to Dr. Kovacs. Thank you. Okay, so our objectives are here. We're going to identify different needs as stakeholders. We're assuming that all of you in here have some sort of stake in the recruitment process. We're going to describe and assess the various tools and resources that are currently out there and talk about how to effectively use a great tool called the Roadmap to Psychiatric Residency. We're going to go through the supplemental ERAS application. And also, one of our learning objectives is to collaborate with other stakeholders to innovate areas in residency recruitment and medical student advising residency selection process as targets for potential improvements in this very, very complicated, or what has become more complicated, I think, process. And certainly, it's become more stressful for a lot of the different stakeholders involved. Okay, so first, I want to see who the stakeholders are that we have in the room today. Who here considers themselves a residency applicant? Okay, all right. Student advisor? Medical education dean or other administrator? Residency program directors? Okay, so we have a couple of those. All right, good. Okay, so I want to talk through just a little bit about what are the different priorities of the different stakeholders and how they view this process because I think that we are in a process where the different stakeholder groups sometimes have opposing needs and are working in sometimes different directions. So the residency applicants, I think the first, and many of you in the audience will know this, but the first, their first priority is getting a spot. Okay, making sure they land a spot. Second priority is let's hope it's the right spot and a place that you want to be and a program that you want to be in. And I think that one of the other priorities in this process is how does an applicant obtain really reliable information about the process and about programs? The student advisors can really be anyone in a medical faculty. Usually, the advisors are concerned more with the overall trajectory of the student than a particular match or fill rate, and they're interested in helping the student truly find hopefully a good fit. There's two primary challenges that most advisors face. The first is keeping current because the recruitment field is really changing very quickly, and the second is time. Most medical school faculty really have increase in clinical and educational demands, and they just don't have the time to read up and keep current or to always give the students as much time as they want. And I will say there's probably some schools in the country where their faculty have a lot of extra time for this, but they are really in the minority and they are lucky. The dean's office. So the medical education dean is typically more concerned with the big picture for the school. The dean is very invested in the percentage of students who match pre and post SOAP, as well as this overall list of prestigious institutions to which their students match, and the image that this might portray to their school, to the board, to future applicants. They want their students, therefore, they want their students to match, right? They want their MSPEs to look as good as possible, their letters of recommendations to look as good as possible, and they want to do anything they can do to get their students to match, including sometimes reaching out to residency programs directly to try to pull for a certain student that they might be worried about. So the number of applications students submit is also seen as a way to increase the chances of matching. Deans also may encourage students to put in more than what is necessary in an effort to sort of over make sure everybody matches. So, you know, the reputation of the school is certainly very important to them. If this were their primary concern, they probably wouldn't be reaching out to push for individual students who might have weaknesses to residency program directors, but it really, I don't think, is necessarily their primary concern. Their primary concern is that match and fill rate. And then we've got the residency program directors, and so I am a chair now, but I was a residency program director for, between my APD years and over a decade, and that's how we all met, because we've all been in that role. The residency program directors, their first priority is that they want to fill with great applicants, okay? So these days they aren't as worried about filling. There's that little anxiety in the, you know, pit of your stomach that I hope my program kind of fills, but there are so many applicants going into psychiatry that for the most part they don't have to worry about whether or not they're going to fill, but they want to find the best applicants, and they are really struggling to find the best applicants. So most programs have well over a thousand applications to screen, and then sometimes hundreds of applicants to interview, but the program director and the faculty time are usually really limited. And so they're trying to find ways. They all want to do a holistic review and look at every aspect of those applications, but they're trying to find ways to honestly to do it quickly and efficiently, because they have to. And they're trying to find ways to figure out which students are really interested in their program. You know, they don't want to waste those valuable interview slots on students who really don't want to even come to their state, you know? And I think a lot of people still think geography is very important. It's probably less important in today's match for applicants than it was previously. Yeah, so yeah, at the end of the day they're usually limited in both time and resources, but they're trying to do it reliably. So we've got a bunch of different groups with sort of different interests in this match. Let's talk a little bit, well, John, you want to talk a little bit about what's happened in recent years with the numbers. Sure. So I was expecting this to be a little bit more of the advisors and program chairs and things like that. So most of you are applicants and relatively new to this. So psychiatry, if you've looked at this, is kind of popular these days and even kind of competitive these days, and it did not used to be that. I've been doing medical student education since 1999. For years, it was one of the easiest things to get into. It was down there with pediatrics and family medicine. People would do psychiatry as a backup plan. If they couldn't get into something else, they'd put psychiatry because you could probably get into that. But unfortunately for applicants, that is no longer the case anymore. It has gotten crazy for people doing advising and for people applying. So yeah, back in 2012, it went down to 3.9 percent. So you can see it kind of went up and down with somewhere in the four to five percent being kind of average. So that year, there was a bunch of stuff in like the Psych News and Psych Times talking about nobody going into psychiatry. And so a bunch of people got interested in doing research, trying to figure out what factors predicted people who would go into psychiatry. Institutional factors, what kind of curriculum makes a difference? Individual student factors, what kind of people, what kind of background are the people that go into psychiatry? So we could try to target people to get more people applying. I actually started going around giving talks to colleges around UAMS to talk to people who were non-science majors because statistically, you're more likely to go into psychiatry if you're a non-science major. And at this point, no one's trying to recruit anymore because we just have so many people. The whole district review part is getting really very difficult to do. So I don't know when this is going to stop, but this past year was the highest ever, 7.3 percent. So that puts us, I think, number three in the specialties in all of medicine. So P's in family medicine, four. Internal medicine, family medicine, and P's are above psychiatry and the percentage of MD graduates that go into it, but not by much because family medicine and pediatrics have been going down the last few years, and psychiatry keeps going up. So at some point if this keeps going, psychiatry will be the number two specialty in medicine for the number of people with an MD degree go into it, which from my history is just crazy. We never thought it was going to be like this. And I'm not sure anybody really knows for sure why. That would be a good project to do. So yeah, back in the days when we were advising students for the match, things were pretty predictable. So the top line there is the total number of positions, so somewhere around 1,100. Most of them filled, and you see the green line is MD applications. So for a long time there were somewhere around 700 people putting in for the psychiatry match. It looked like things changed there around 2014, but not really. That was the all-in when the match went that everybody had to put all of their positions in the match. So before then some programs would leave a few positions out of the match, and so it looks like numbers jumped up, but not really. It was about the same. But starting about 2015, things just started going up, and they haven't stopped. So it's more than doubled since 2014. It's a lot of people going into psychiatry, and that's just MD applicants. So obviously the DO programs are opening up all around the country, and there's a good number of DO students that are going into psychiatry as well. You could say that it's actually pushed out some international students because of the number of US students going into it. There's also the unmatched rate, so this used to be something we really didn't worry about unless you were a really kind of borderline student. But now we really have to make sure that your practice for your interviews and that your dean's letter and everything looks as good as possible because it's much more likely now that you're not going to get into psychiatry than it used to be. So this is the total all MD unmatched rate. So you can see it used to be kind of around 10 percent, and then it doubled in 2018. That was the psych is the new derm year when everybody started because actually that year psychiatry had the highest unmatched rate in the entire match. And this is the metric the NRMP uses for what is considered the quote-unquote most competitive specialties. So this is when it says MD only, it means you only applied to that program. So they have a percentage of people who went into ENT and ortho, neurosurgery, dermatology, who only applied to that specialty. That would be students who are probably better off in the match because they don't need a backup plan. So you could see psychiatry in 2018 was 13.7 percent. It was number one that year. So more than orthopedics, more than dermatology. That's a lot of people who were playing that didn't get in. And of course students picked up on that and started putting in more applications. So when I applied in 1994, I put 11 applications in and I got 10 interviews. And the 11th just said I sent it in too late. They had already given up all their spots. I was not that great of a candidate, but back then you could get interviews all the time. And I had a person who was in the top 10 percent of the class a couple of years. She applied to 15 places and got four interviews. Top 10 percent of the class. Like a 250 on step one. But she applied to very special programs because of a couples match issue. But it's definitely gotten more difficult and people have responded by applying to more programs. You can see the numbers have gone up 142 percent just in the last eight years. And then for poor program directors, luckily I've actually never had to be a program director. I was mostly a medical student education guy. But for program directors this has gotten really difficult to do. In the old days you might have a couple hundred if you counted all applications, but you could really go through those and kind of pick. There's no way we can go through a thousand applications. Nobody in psychiatry used to care about step scores and now I hear people are weeding people out based on your step score, which sounds ludicrous. That's not the way we used to be. But yeah, you can see a 250 percent increase since 2014. So your job has gotten a lot more complicated. So I guess with mostly applicants this might be a little bit different because this may not have really changed. So it may be that for some of you this is, I don't know, is this new news for anybody or does everybody know things have gotten this much different? So I guess for the applicants you were kind of wondering how are you approaching the match with the way it is now. And for those of you that are program directors or medical student advisors, what are you doing differently? How have things changed at your school and how is this changing your role in the process with the match? So we'll take five or ten minutes to discuss this. All right. So our advising and PD group here. Any overall things that you've, or changes that you've seen locally in applying to residency and how have you guys adapted? What was some of the key concepts of your discussion? I think absolutely for program directors that's really been an issue. And so the MSPEs are being scrubbed of any comments that say like, oh, here's your next area for working. So you feel like your hands kind of get tied, right? Because the letters of recommendation all say everybody's in the top ten percent. The MSPEs get scrubbed. Ninety percent of people get honors. The numbers are gone. And so you're left kind of trying to evaluate who is this person and what are their strengths and weaknesses and how will that fit with my program? And if you have like a lot of feeder schools that you might get ten applicants from, they might be written. It's a great, I remember from some of the other ones in Philadelphia, I would know the different writers and then you can put that together. It's tough. Yeah. So that's I think some of the biggest struggles that the program directors are facing. All right. You want to talk about them, Daniel? Yeah. All right. So thanks, everyone. We weren't totally expecting to have so many students here or applicants, but I hope that this will be very helpful. So my role for this presentation is to try to go with, go through some of these resources that we do think that you guys should take advantage of. And we'll try to kind of tell you about the pros and cons of each resource. So we're going to talk a little bit about data driven resources for advising. And all right. So the first thing that many of us, certainly if you have a role in undergraduate medical education or UME, these are things that oftentimes the dean's office will use. So the NRMP interactive data tool, I think, is actually one of the funner things if you have any sense of what fun might be for me. But in any sense, but this is, for those of you who don't know what this, the NRMP actually surveys program directors roughly every two years. And they literally ask them, like, what is your top ten of items that you look for, for both screening and then for ranking? Really simple, easy to use. They put out these nice, very lovely bar graphs, which I'll show you in the next slide. And it helps, I think, students and advisors figure out, like, what areas do we need to focus on? So I mentioned in the group that I was in that if your advisor is not kind of, you know, up to date, so to speak, on what the current trends, the things that Dr. Spallon talked about, they're going to be out of touch and they may give you, frankly speaking, bad advice. The bad is that for this tool, which again I'll show you, is that they use the data from the 2020 survey, and the data from the 2022 survey is actually out now. And you'll have to, unfortunately, kind of search for it on the NRMP website. But this prints out a very lovely, colorful graph. My thing that's always fun to play with is that if you change the specialty to, like, say, surgery or orthopedic surgery, you'll find that psychiatry is very different for what we look at as program directors versus staff. So the biggest thing that you'll see, they hadn't built out necessarily the same dashboard for every year of the program survey that came out, but you'll see that the personal statement basically is in the top one or three literally every year for, like, 14 years, I think, in a row. Only very recently did step one kind of pop up in the top three. But nonetheless, we care about personal statements. Beware of chat GPT, for those of you who are All right. AMC Residency Explorer. So this is a free tool that all of you have. I'll talk a little bit about Texas Star. But this is source verified data. So, you know, as you're looking at resources to try to figure out your strategy or game plan for how you're going to apply or screen for the program directors, this is an important thing that you may also want to look at. And it actually has a nice comparison tool, which looks something like this. So basically, you can input your data on your personal situation, board scores, you know, what settings you want, and it kind of gives you a little bit kind of a heat map. So if you think of, like, Tinder for, like, applicants, this is kind of it, right? Because then you can look through different things. You can see how well you match up with, at least based on the data that they received, and you can see, ooh, I'm super hot. I got three out of five dots or something like that for this program. So this is a really good, can be a really good tool. Again, it's open to anyone that has an AAMC login, which all of you should have now at this point, or soon will have. So the good about this is that you get personalized statistics as an applicant about where you stand, what programs have done historically, relatively easy to use. It can actually help program directors, which is something that we really love, hopefully try to help decrease over-applications to particular programs. So if you find that there's probably no chance that you're going to be maybe getting an interview at a program, it hopefully would discourage you from applying to that program. The bad, as I said, is it does require the AAMC. Unfortunately, unlike the previous slides that I showed you where you get this nice colorful graph and you can figure out exactly where you're at, it exports it out into an ugly text-only Excel spreadsheet, which no one likes to look at. And the ugly, I would say, for this is that it really disadvantages a lot of newer programs. So we have a relatively new program in Omaha. We split off from an existing program, and so we are actually not located on there. So you can't figure out if you're a good match for our program or not if you're an applicant. All right. Texas Star. Out of curiosity, who in the audience actually knows what Texas Star is? All right. I think a couple years ago when this presentation was given, like, no one actually knew. It was like a secret. So Texas Star stands for Texas Seeking Transparency in Applications to Residency. And this is administered by UT Southwestern. So they actually run and hold kind of the data set. And you can go to the website and kind of look at it. So this is a very busy slide, but basically it has the same idea that, you know, this is information that the deans send to Texas Star. Again, you put in your kind of credentials. You can kind of see where you're at. And it generates kind of like a heat map, right, of how competitive or not you might be for those particular programs. And there are, you know, it has a lot of programs, but it doesn't have all of them. So the good is that you get, again, personalized statistics for applicants looking to see if they're a good fit. And again, advisors can use this tool. Hopefully it will decrease over-application by students. And arguably it probably has some of the more well-defined kind of criteria that other databases do not have. It does require a login. It is self-report. So think about where you get your sources of information. And it has a limited sample size because not every medical school dean has actually logged in and volunteered to participate in the system. So it's not exhaustive for all 330-plus residency programs now, but it does have a good number of those programs there. And again, if you are a relatively small program or if your dean in medical school hasn't provided information, it can, you know, in some ways hurt you. So, all right. Who's heard of Doximity? All right. So you will hear more about it because wherever you go for residency, the chair of that department will tell you to fill out their Doximity survey because they want to boost everyone's rankings. So Doximity is kind of one of those social media platforms that are primarily geared towards physicians. They created something called Residency Navigator now, and you can search through it based on reputation, how much, how many publications the faculty will put out, board certification rates, you know, did 100% of their class pass or not, how many people go into subspecialties or fellowships, et cetera. And this is modeled after the U.S. News and World Report rankings. So it kind of follows a very similar thing, and they say, and this is very important, that they say that they use peer nominations, ratings, and handwritten reviews. So I don't know what that last thing means because no one can actually read the handwriting of any doctor anyways. So if you log on the website, you'll see that it gives you kind of a nice listing of who's kind of the top dogs in the field. And not unexpectedly, a lot of the programs that you probably think are on the top are on the top. So the good is that it's relatively easy to use. It does give you information on board certification and subspecialty rates, which is sometimes a proxy for how competitive these programs are or what maybe the academic focuses are for that program. The bad is that it does require a login, doesn't really have any personalization options that I know of right now. It is not data-driven. This is all opinion. And the ugly is that, again, we're not quite sure whether methodology comes from. And I think I looked at my program the other day, and we were like the top program in Nebraska, but like there's only two programs in Nebraska, so I felt really good about it. But then you don't know what it actually means, right? So all right. All right. Program websites. So this is something that we've added from a prior iteration of this talk. So for the program directors, in fact, we're going to actually have a publication in Academic Psychiatry about this. I was participating in a national survey of applicants over the last four years, and this is now a summary of what the abstract will be in the eventual article that will come out. But basically program websites are really probably one of your best sources of information. Why am I saying this? In the pandemic, nearly every psychiatry residence program updated their websites. And if you use like the Wayback Machine or archive.org, you can actually see some old webpages, and they were horrible. Most program websites were horrible. They had outdated photos. They may have had residents who were like last there 10 years ago. They had inaccurate listings of who you can contact, et cetera. But students and applicants over the last couple of years now have really found that the program websites have not only been updated, but they're really influential in terms of where they decide to apply and could be kind of in some sense a tiebreaker for where they actually rank. The bad is that it's somewhat inconsistent. As many of you know, websites are not always designed by the residency team, so there's a lot of inconsistency in standardized information. The ugly is that, of course, they put the most shiniest, happiest-looking resident on their webpages, right? All of you have probably done that. Oh, not the sad ones, yes. So it's often self-serving, but what I would also say too for the applicants here in the room, as much as we've tried to work with our internal design team, they're really kind of anal about how many tabs you can have, how many pictures you can use, et cetera. And so oftentimes there's a lot of constriction around kind of what we are allowed to put on the website. Like if it doesn't harmoniously work within the same color scheme, like we can't use it, we can't use separate logos, et cetera. And this is not a problem unique to Nebraska, I'm sure. For the program directors, there's probably some design dictator and if you don't meet he or she's needs, it's like you can't put anything on the website, right? The other thing I would say too, just as an antidote for our program, there's like a three-month delay when we submit changes and the time it actually shows up, which may not work particularly well for recruitment here. All right. Social media. How many of you are on social media of any sort? Probably all of you, and those lurkers are probably also on social media, but you refuse to raise your hand. All right. So many of you know that I use Twitter fairly regularly, but a lot of programs have often have now moved to Twitter and Instagram, and also as a form of social media, I've also started using, you know, participating in virtual fairs, sometimes using Discord and other kind of, I would say novel, but kind of gaming-centric platforms. So the good is that a lot of programs, again, because we could not contact or have live visitors or applicants in our spaces, we spent a lot of time and effort working on social media. So the great thing, of course, for the program directors here is that it didn't cost you anything, at least monetarily. However, you know, it does cost a lot of time and effort to really think about some useful content. I think for most of us that have thought about social media a lot, it's probably very helpful in generating interest, but probably not super helpful in defining what your rank list will be at the end of the day. So the bad thing is that a lot of social media accounts are usually run by residents, which is not entirely or inherently bad, but sometimes, you know, different residents as they go through the different years will have different foci or time or they have different pacing of their output and posts. So it sometimes is very publicized and very active, and sometimes it's like a dying, like, they just put out, they pump out, like, one post a month, which is probably not helpful for all of you. So the thing with virtual fairs that I wanted to point out is that nearly every program now participates in virtual fairs. I remember over the last two years now that a lot of our virtual fairs apparently interfere and conflict with other virtual fairs happening at the same time, and students and applicants are left wondering, like, how do you go to two virtual fairs that are being hosted at the same time? So one of our advices has been to try to participate in either regional or national event where there's, like, multiple programs, and you can kind of skip around different rooms, and although not virtual this year, thankfully, there will be a live residency fair tomorrow, so I hope many of you will participate and get to see that. But nonetheless, as far as the social media goes, a lot of content is really highly dependent and, again, also dependent on who is running it. All right. Who uses Reddit here in the room? Ooh. All right. Mostly ugly, by the way. Yeah. All right. So Reddit, for those of you who didn't raise your hands, Reddit is kind of a group slash news aggregation tool, so you can kind of type random things in and you can find out what conversations are happening around it. So Reddit has been, I think, really popular recently for a lot of self-reported quote-unquote data from applicants. So applicants apparently are sharing their rank list, their, at least in past years, their actual visit to various programs. Some have actually put in, as listed here, their actual rank list, which is kind of scary at times. There's a lot of end-of-one advice. I don't know why there's so many experts on Reddit, especially if you're an applicant, because you're, like, managing yourself and not presumably a lot of other people. But I think it's been extraordinarily interesting from just a psychiatrist standpoint that people are putting their interview highlights to the lowlights, too, as well as interview questions that they get asked, et cetera. And so this was taken from our presentation a couple years ago, but I'm sure some of you may be starting to get familiar with these. There's kind of an annual, and you have to search for it, kind of a spreadsheet that a lot of applicants participate in. I'm just going to hold my thought about that. And then similarly, at least in the years that we did live interviews, there was also kind of a comment section about actual interviews. So I won't go into the specific details on this slide here, but a couple of the applicants wrote that, you know, we had awesome tacos. I think the PD director's house was a bomb. And then they also, like, this program's kind of creepy. So the comments ranged from, like, you know, very positive to, like, oh, my gosh, I don't think anyone should go to this program. So good, pretty idiot-proof to use. You can log on to Reddit. You don't even need an account. You can find most things. Sometimes it helps sensitize students and applicants about what's kind of being discussed or talked about about particular programs. The bad is that this is all self-report. So if any of you have used Student Doctor Network, right, SDN, right? It sounds like STD, but SDN, you'll know that it's a lot of speculation. And at least when I was at Michigan, I occasionally would read SDN. I was like, this is not even remotely true. And I'm, like, having been a resident fellow and a faculty member at Michigan, I'm like, these things are not even true. So the ugly is that people might be lying. And there's really no verification or accuracy of the data that's being presented. So one of the things that we this is you. Sorry. One of the things that we've worked on collectively is something that I'm going to hand over to Dr. Kovach, and she's going to talk a little bit about the roadmap. So we actually started the roadmap before COVID. And as a field, those of us who are more interested in residency recruitment into psychiatry and into the trends and what was going on, we as a group kind of started discussing some ideas and how we can be more helpful to the applicants, to the advisors, to everybody involved in the process. So what we did, we brought together APA, AAP, Academic Psychiatry, ADAMSEP, the Program Directors Organization, ADPERT, I'm sorry, ADAMSEP, the Medical Student Educators Organization, ADPERT, the Program Directors, and then PsychSign for the student interest groups. So we had leaders in the area from all of those groups come together to write something that would hopefully be helpful for the applicants and for those who advise them. And we had noticed that applicants, the advice they were getting was, all advice was not created equal. There were medical schools out there that didn't have psychiatry residencies. There's medical schools out there who don't have their own psychiatry departments. You know, so the degree, we wanted to sort of level the playing field a little bit around the degree of advice that was out there. So we go through all sorts of different areas in there, questions about the MS4 year, how to choose programs, for instance, determining how many to apply to, letters of application, personal statements, et cetera. There's also some special topics about at-risk applicants. This is what it looks like. The good is that it is easy to use. It's free. It's, I think, thorough and expert advice, but I was involved in the first iteration of it, not the sort of updates. We've been updating it as well. It includes a lot of different resources, and it levels the playing field to some degree. The bad is that, of course, you can't get individualized data out of this. It's maybe a 20- to 25-page-type document. It's linked from all of the parent groups' webpages. APA, I think, is usually the first one to kind of get the updates onto it. I'm going to look at theirs first. This is just the table of contents and some of the things that are in it. So some of the specialty topics for faculty advisors, for applicants from different types of medical schools, military, et cetera. And then we found that we really need to do annual updates. So this is the annual update that was put out last year in July of 2022. So it talks about program signals, geographic preference, past experiences, sort of all of those things that change each year. So we are anticipating a update for July of 2023 that will talk about the updates to the signals, secondary applications, and other things that are new and different. So I'd really advise you to take a look at that once it comes out, both for the advisors as well as for the medical students. And I think going straight from APA's website might be the best route to get there. All right. So for our next group activity, we want to talk a little bit about what's been your experience utilizing these resources? What other resources are you using that we might not have heard of or included today? And anything that you might do differently around the utilization of these resources for the program directors this year versus prior years, sort of trends that you're seeing. Okay. So we'll keep our kind of grouping of the faculty advising and program directing group and break off into groups of four or five, again, of students. Maybe try to mix up or applicants, maybe try to mix up with some applicants that you weren't with last time. Okay. Well, we're worried about time and want to make sure we get to the supplemental. So before we do a breakout. Yep. Go ahead. What we wanted to do is because one of the new pieces that's going on, as you know, is that we're dealing with the supplemental application that we've had one year data on. So what we thought was we'd go ahead and present some information from that and then tell you what the new changes are going to be with that. So I'll give you most of the data on the 22-23, the first year that we've had it. It was optional for both the programs and applicants and there was no extra fee. Students had a six-week window to apply from August to mid-September. And then the program directors got it at the end of September with the rest of the ERS. The important thing was it was separate from the main ERS. So program directors really had to go get that separately. So it was kind of a hassle there. Most of the programs participated, about 90% participated. And I want to go through the specifics of it. So there's several sections that was in there. So one of the sections was experiences. So students had an opportunity to share something more meaningful about themselves. There was five options that they could do as far as meaningful experiences. And the suggestion was that you don't put in there what you've already talked about in personal statements or what's covered under noteworthy characteristics in the dean's letter, the MSBE portion. We're limited to 300 characters and about 20% of applicants chose not to do anything in meaningful experiences. Now within the meaningful experiences, you have several areas that you could talk about. You had to clarify the type. So that could be anything from education, military, volunteering, work, teaching, professional organizations you were involved in. The duration of frequency, that could be were you involved in it once a year, once a week, once a month, or did you do it every day for say in the summer for one or two months. You could talk about the setting. So were you in a rural area which was defined as less than 2,500 people, urban being more than 50,000 people. What was the name of the organization and what was your role or title within it? And then there was a couple of other areas. So what was it that was a focus area of what you did? Did you work in healthcare? Did you work in the arts? Were you involved in a wellness? Were you involved in a QI or technology? And then within the key characters you could talk about what exactly about you with that particular activity did it apply to. So did it apply to communication skills or problem solving or empathy or teamwork and leadership? What was it that you thought it was relevant to? Then there was the option of doing one additional in other impactful experience. And this was not intended for everybody to do it. It was only if you happened to have something very unique that you thought was impactful. And you could write a lot longer, about 750 characteristics, things like, was there something major that happened in the family, maybe a death in the family? Or did you come from a very difficult financial background that made it difficult for you in terms of your arrival where you were as a medical student? Or was there something, so were you in a violent neighborhood that influenced perhaps your education or other aspects of things? So something very unique and not one that necessarily a lot of people would have experienced. So the other area was the geographic preference area. This was based on the nine US census areas. And within each area, it showed what states were included within that. You could pick up to three preference areas. Or you could say you don't have a preference. And whatever you decided, you could then choose to explain why or why you did or did not have the preferences. Now, if you chose a geographic area, only then did it show up for that particular division that you identified. If you said no preference or didn't respond at all, all the programs got to check it out. And then within the geographic, there was also the division of rural and urban. And it was on a five-point Likert scale, very much like rural, little bit like rural, no preference, little bit like urban, and very much like urban. And what you'll see later on is that there was very little, less than 5%, that chose the rural. Most people had no preference. And about a quarter did somewhat urban. About 20% did very much urban. And then the big one was, of course, the program signals. We had five that you could signal for programs. And this was only visible to the programs that you chose to signal. And there was lack of clarity last year about should you signal your own program. And then there was lack of clarity about if you did an odd audition rotation or an away rotation, should you signal them? Because there's been this thinking that your own program is probably going to consider you for themselves. And also, they know you well. So why waste a signal on that? Or if you've already auditioned somewhere else or done an elective rotation, they already know you. Maybe you don't have to. But it was lack of clarity. So it was all up in the air. Some people did, and some people didn't. OK. So within the actual data, about 92% of people actually did the meaningful experiences. It averaged about 4% to 5% most people did. Where did that fall out? So about a quarter of people talked about volunteer service or advocacy areas. About 16% talked about work education and a lot less research and extracurricular. The focus areas seem to be much more in the medical education and community involvement area, which was higher than, say, clinical science. And then about 10% did access to health care. As far as characteristic, what particularly was most significantly focused on was teamwork and leadership, about 20%. Next was empathy and problem solving, just more than communication. Only about 56% of people filled out the impactful experience, which makes sense. It wasn't intended for everybody else. What is important to know is that 65% came from underrepresented in medicine, which, again, I think it speaks to it needed to be something impactful. And many of those groups do have something unique there. As well, about 72% of international graduates did fill out the impactful experience aspect. What did they talk about when they talked about the impactful experience? The resilience for a quarter of them. Another quarter talked about personal family illness. And then there was about 20% that talked about getting into medical school being impactful for them. Now, for the geographic setting, about 90% of applicants used it. About 77% actually said they had one or more preferences, and the rest didn't have any preference. Now, within the US DO or MD candidates, what is really important, we talked about maybe geography wasn't as important, but it actually is. What this is showing is that 90% of the geographic thing aligned with either their permanent address or the address of the medical school. So people are choosing to stay within that when they're applying. And so that's where they applied was 70% of them sent it to the programs that was in that division. And the setting, as I said, it was mostly either no preference or in the urban areas. As far as signaling, so again, about 50% of them sent it to geographic divisions in their same permanent address, which was about 50%, and then 45% to the medical school area. And here's what's important to know. 90% of signals and geographic division aligned. So people are going to signal in the division that they're interested in, or the geographic division they're interested in. And it is important to know that there was an average of around 64 signals most programs got, which was about 7% if they're getting about 1,000 applications to them. But the range was everything from 60 to 40, and about 10% of the programs got about 22% of the signals. These were generally the Ivy Leagues. They got a whole bunch of them. And clearly, they didn't interview everybody that they got signals from. The university programs did have more signals, 85 signals, and the community-based ones got about 45 signals. So they also surveyed advisors. And here, they were looking at what did the advisors say. So as far as past experiences, they thought it would help. And again, slightly more than half people thought the impactful experiences helped, as did the meaningful experiences. They didn't think the key characteristics and primary focus area was necessarily going to be helpful. For geographic preferences, again, about 2 thirds, let's say, told the students to put what they preferred regardless of competitiveness. And some said, if you don't have a preference, put that down. As far as signals, again, this was the most important part. More than 70% thought it did help the student get noticed by programs, and they did advise them to signal based on their true preferences, although some did say, also signal based on how competitive you are. It took about an hour for them to advise each student, and they did utilize the instructions with their AAMC. The program directors felt that the experiences did help them get a better picture. They ran 60% to 70% for all of the areas, and they thought that what people described in it was appropriate material. And about 50% used it for the interview selection, versus about 20% who used it as a tiebreaker to consider interviews. And a small percentage, about 20%, continued to use it for rank order listing time frame, which the AAMC said you really shouldn't, because you might be interested when you apply for interviews, or ask for interviews, but after you've interviewed, you may not have the same preference, so it really isn't supposed to be used for rank order. As far as impactful experiences, about 65% thought that the signaling helped, and again, 40% used it for the interview selection, but here, a majority used it for tiebreaking, for asking for interviews. For geographic preference, two-thirds thought it really did help them identify applicants who would have been overlooked. So, for example, if someone's applying from California, or you're on the East Coast, you know, you'd normally, because we didn't have COVID, and we just did not think why are they interested, this did allow them to know if they signaled, yep, they really are interested in me. 80% used it to screen, and about a third of program directors basically interviewed everybody who signaled, okay? What was clear here with the program directors is that they gave more weight to the signals than they did to geographic preferences, and there's been a survey that was sent out, and program directors overall decided it's still going to stay at five. All right. What do the applicants think? The applicants thought, and most of them used it, they thought it did help them get recognized, or to showcase themselves, and here's the crunch. The top reasons for the program selection, 64% of them said it was the geographic location of the program, versus 50% thought it was the alignment of the program strengths with the career interests, which was about equivalent to the closeness to the family or friends. The top reasons for the geographic preference was 87% was proximity to family and friends, which was, and next was about 75%, again, the location of the desired programs. About 50% said lifestyle or having previous ties to the area. So for signaling, almost everybody thought that it was important, and 85% of them signaled only in one specialty. Remember, you could signal in different specialties depending on how many signals each specialty allowed. And most of them, 80% thought that they actually did signal where the true interest, and some said they would do it where they wanted to get more noticed. And again, drives back to geography. The top reasons for signaling, 62% said it was the geographic location of the program, more so than the perceived fit of the program to them, or even the closeness of family and friends. And they said they spent about an hour with the advisors, to prepare it, and then some time to think about it more. So what are we looking at comparing the 2022 and 2023? So we still have five, but the supplemental application now is integrated into the main ERAS. The program desk will get it all at the same time. You have up to 10 experiences that you can share with more depth, and within that, you can have three more of those three that you can describe even more as being more meaningful experiences. You still have one option if you want to use it for an impactful experience. Geographic preferences are still there, as is urban and rural. So in summary, the past experiences and impactful experiences do help enhance your application. Geographic preferences help if they're concordant with the permanent address or the medical school address. Signaling definitely helps being noticed, and program directors pay a lot of attention to it. And the geographic and the signaling tend to become concordant. So that's it, I think. Now we can break out for our groups that we have. Thank you. All right, thanks, everyone, for participating in your groups. I know there's a lot of questions out there. I think maybe to use the best time that we have in the remaining time, which I believe is about 10 minutes or so, we're gonna try to maybe, if people are okay with this, we're gonna go to a panel. So for those of you who may have some outstanding questions that weren't totally discussed, I think we have a couple of microphones in the middle of the room, but if people want to pose some questions to our panel, we'd be happy to try to help things out. So tell us your name, too, and kind of where you're coming from as far as your stakeholder status. Yeah, my name is Manath. I'm a fourth year medical student at University of Virginia. I just had a question. We were talking about, is it possible for you to change your 10 experiences and basically change your whole ERAS application based on what school you're applying to? So can you change your application to be more catered to your geographic region, depending on? That's a good question. I know you can customize your personal statement per school. I'm not sure if ERAS will allow you to do what used to be a part of the supplemental app, because it'll be consistent to every program. So I would probably guess no, unless. I've never seen anybody do that. Yeah. I think so, but I'm not 100% certain. It's also a lot of like, it would be mind boggling, like really confusing to you, because you're like, I don't remember what I put for this 50th program, right? So. Right. I guess it's like if you have an experience at a school and you want to include it for that school, like that type of thing. Yeah, I would say probably the best advice we could give is that you should try to customize the personal statement to the program and have it be more than, and by the way, you know, East Tennessee State University is my favorite medical school and training program, because we're going to pretty much catch up on that. And I was saying to the group that was in earlier, you know, we want to see, I think most program directors would align and maybe Dr. Kobach would say, we want to see alignment between your personal statement, your activities and your letters of rec. And if there's non-alignment, we either will be suspicious. So I mentioned as a, you know, I think a lot of applicants anecdotally have said, you know, we want to be, I'm making this up, social justice warriors and DEI is like my most favorite thing in the world to talk about. But if I see nothing on your application, it makes me very suspicious that that was just kind of a generically added statement. So does that make sense? Thank you. Yeah. I joined ADPERT and the rest of the panel would know this. So when we started a new program in Nebraska, I not jokingly reached out on the listserv and contacted former colleagues of mine and asked them, can we steal any of your policies? Because as you will find out, program direct programs are run on policies and you need to have a certain language. And if you don't know what that language is, your program will not be accredited. Yeah. Yeah. And we, several of us participate on the recruitment committee. So we are particularly interested in how the, hopefully the right candidates come to your program. So that would be the committee, even within ADPERT to really kind of talk to. And there's also a new program caucus too. So other folks, myself included, that have built new programs here in the last four or five years, but yeah. And there's also the new training directors section within ADPERT as well. So at your first meeting, they'll give you a more intensive experience with it. And then since you're looking at curriculum and all those kinds of things, the virtual training area within ADPERT gives you materials that you can look at as well. There's also a curriculum committee that's putting in materials there. So there's a lot of stuff that you can, once you join ADPERT, you can have access to on their website. I think it was made up of work. I'm sorry? I just decided it's difficult by doing things without the people in there. Don't reinvent the wheel. Yes. Any program director who's been a program director longer than two years will tell you, do not reinvent the wheel. Yeah. And we love, we love to share the wheel with you. Yeah. All the wheels we've created. All right. It's a very generative group. All right, question from the audience here. Hey, my name's Seth. I'm a medical fourth year at Baylor College of Medicine in Houston. One of the statistics that really jumped out is for the program directors, they said 76% used that impactful experience essay as a differentiator for who they gave interviews to. And it was compared to like 20 something for the three meaningful experiences. And so it makes me think like, wow, having that impactful maybe past experience is something that we should really think about. Any recommendations for how we should think about that section? Yeah, Dr. And then a sub question is in relation to past experience with either family members or personal struggles with mental illness, how to talk about that in a healthy way. I've wondered often, is it seen as like a liability if you have a family history of something or a personal experience with something versus something that actually gave you resilience? So any recommendations on how to frame past experiences in a productive way? Dr. Chen, do you wanna take that? You might use the microphone there. So let me take particularly the first one. There it is. Oh yeah, okay. So the impactful really is not meant for everybody. That is one thing that is specified. And if you think you've had an experience that's really out of the ordinary, that really is important, then it makes sense. What they, in fact, WMC was trying to suggest this time is people that talked about it, about 20% did that as, you know, how difficult it was getting medical school. It's difficult to get into medical school, period, right? And for some, perhaps more so than others, but that was not typically what it was intended for unless there was something very unusual. So for example, if there was, you know, a parent that was majorly ill or unavailable and you had to be their caretaker and all, so something very unique had to be in there that you would want to use the impactful experiences. And I think from a pre-D perspective, in looking at that, what you're looking at is to really look at how did that impact you. And so you may have utilized some of that, although they're suggesting not to do it in the personal statement as well, but some of that might come through somewhere else as well. It may have shaped how you, you know, because in 300 or in 750 characters, you can't say a whole lot, realistically. And as PDs, we're already overloaded with so much information, you want to use your time, your space very effectively. So let me ask that. And the other question was more around- Disclosure. Disclosure. Yeah. That one, why don't one of you take that? Okay. I apparently have 30 seconds, as Dr. Spalding is warning us here to get done. So I guess my perspective on disclosure is that for most of you in here in the audience, probably some sort of medical event or crisis probably inspired you to enter medicine. You know, we do some mock interviews for our fourth year medical students. And I would say just as an observation, if you use disclosure, so maybe relaying that a partner or a loved one had mental illness, if it makes you like re-traumatized or uncomfortable to talk about, but you've actually put it in your application, I would not put it in your application if you're not ready to be able to expound on it. Does that kind of make sense? So I'm not suggesting that it's not important to you as an individual, but if it's something that can't be conveyed in a positive manner through your interview, then I would think it's not maybe a good thing to put it. There's also a similar issue around personal disclosures around mental illness too, whether you have ADHD or OCD, and that would be kind of similar advice I would have for that as well, Dr. Kovach. We will conclude the session, but if folks wanted to come up, please come on up and ask us a question. But thank you for coming. If you want to stay, you're welcome to stay. Thank you.

psychiatry residency

Shambhavi Chandraya

application strategies

NRMP tool

AAMC Residency Explorer

supplemental applications

ERAS

program competitiveness

application tailoring