I think this will be an exciting talk on covering different digital approaches around diagnosis and treatment of substance use disorders. So we have three excellent speakers, or really two excellent speakers, and John Torres will have here. So this is the order that we'll be speaking in. So Dr. Bold is at Yale. We'll be speaking about some very interesting substance abuse work with apps and wearables around smoking cessation. As I said, Dr. Lisa Marsh likely needs no introduction. She's been the leader in using technology in mental health forever, at least as long as I've been in the field, and really making Dartmouth into the leading center of it. And then I just tag along and used to have more hair in that photo. So our agenda is, here as I said, we'll go over these ones. We'll try to save about 20 to 30 minutes for your questions. We were told that this is going to be one of those on-demand sessions, so they want us to ask questions into the microphone, so if people are listening on demand, they can hear it. And I just want to quickly set the stage with some facts from SAMHSA that I think all of us intuitively know, but the numbers, they're pretty striking, right? So 46.3 million people, age 12 or older, would meet the DSM-5 definition of having a substance use disorder, and then 29.5 million would be alcohol use, 24 million would be a drug use disorder. Those are big numbers by any metric. And then really the youth, right, 13.5% of youth age 18 to 25 had a substance use disorder in the past year. And I think then the number that really motivates this talk and to think about is, again the numbers are old because they're public, but 94% of people age 12 or older received no treatment, zero. So that means 6% of people got something, and we can talk into questions, is that 6% of treatment good? Is it evidence-based? Is it useful? But we definitely need new approaches, right? And I think that all of you are probably in this room because technology is going to help us in this area. And we've all heard, but you can't train enough people to see that many people, right? Even if we doubled the number of psychology, psychiatry, counseling, nurse practitioner spots, social workers, this is going to be a hard number. So I think what, in listening to the three talks and the questions, it's really how do we kind of address that 94%? What do we begin to do? And how can we do this the right way? I think you'll also hear in these talks that technology is not a panacea. It doesn't work perfectly either. We can't just tell everyone, you have an app, our job is done, we can go home. I'll sort of say technology is not going to take anyone's job. I can guarantee you of that. If you are, it's just not going to happen. But I think we can think about what are the real pros and benefits to risks of it. So with that, I'll hand it off to Dr. Bull. Thank you so much. All right, let me see if I can get this started. Wonderful. Well, thank you all so much for being here. I'm really excited to kick off this series of wonderful presentations, and I'll be talking to you about digital approaches to assessing and treating tobacco use. I'll start and just note that I have no conflicts to declare, and I'd like to thank and acknowledge the sources of funding for this and other related work. So tobacco use is the leading cause of preventable death, and this is an area that's really a passion of mine because of the huge public health impact. So this graph here shows the annual death rates in the U.S., and you can see tobacco-related deaths in that top bar, which far exceed even the total of all these other substances combined. And so this is really despite huge advances that we've made in reducing tobacco use through education, through treatment, through public policy, and yet tobacco use is still the leading cause of preventable death, and so a huge area and public health opportunity. Again, this is within the U.S., but worldwide this picture looks very similar as well, and specifically the tobacco-related costs in the U.S. So one in five deaths each year are due to a smoking-related cause. Sixteen million Americans live with a smoking-related disease, and smoking-related death and disease cost over 300 billion dollars each year, so huge opportunities for addressing and reducing the overall costs here. In terms of tobacco use epidemiology, so what are we talking about? Really, cigarettes are still the most commonly used tobacco product among adults. This is over 28 million adults, and many people say, I don't see people smoking, I don't know that many people smoking, and more and more it's often in marginalized populations, so people who might not have as much access to our other existing forms of treatment, and so really thinking about who most needs our care and tobacco treatment. And then there are other tobacco products, so the next most common is e-cigarettes, followed by cigars, smokeless tobacco, and pipes. There are three million adults who are using more than one tobacco product, and the most common co-occurrence is with cigarettes and e-cigarettes, which is still really problematic, because we know that any amount of cigarette use has huge health care costs. Even smoking one cigarette a day has the same cardiovascular risk as smoking, you know, many, many more than that. So really without complete cigarette cessation, this group is still at high risk, and each year more and more youth are starting tobacco products as well, so there's always a new generation of people coming up, so I always like to set the stage that tobacco use is not something that we've solved. We have huge areas of need here, and there are a number of health benefits of quitting smoking. So quitting smoking is really the single most important thing that anyone can do, any health behavior change, because it impacts every part of the body, and there's really no point at which it's too far gone. So quitting at any point improves health, overall life expectancy, lowers risk of cancer, cardiovascular disease, pulmonary disease, and a number of other things. So the good news is most adults who smoke want to quit. So on average, 68% of adults who smoke want to quit, which is over 22 million adults in the US each and every year, and about half of them actually make a concerted effort to try to quit in a given year. The bad news is very few are successful, so only about 7% of people who try to quit are successful quitting in any given year. And as we've already kind of alluded to, there are a number of challenges with treatment. So we have some good medications, and we have brief counseling that can be done even in just a few minutes in the clinical patient encounters, and these are really effective, but they're very low use rates overall. Less than 31% of people are using these standard, gold standard levels of treatment care. Most people try to quit smoking entirely on their own without any kind of support, and there are a number of limitations to interventions, including access, provider availability and training, cost, insurance coverage, and really in-the-moment need. So being able to access that intervention and that support in that moment that you need is something that we have not been able to solve with traditional medicine. So identifying new approaches to promote smoking cessation is really critical for improving public health. So enter digital health opportunities. You all probably saw that coming, right? That's why we're here. So digital health and mobile applications can offer potential opportunities for smoking cessation care. So we know that almost all US adults own a cell phone and the vast majority own a smartphone. They always have it with them anywhere they're going, and over 80% of adults who smoke and are motivated to quit own a smartphone. So this is really our target population. This is the group who has readily available potential treatment opportunities at their fingertips. So providing smoking cessation care through a smartphone could be accessible, scalable, cost-effective, and importantly provided in the real-world environment in those times of most need. In thinking about this and in kind of preparing for this, we realize there's really limited knowledge on the usability and the scientific basis for smoking cessation apps that are available. So in collaboration with Dr. Torres and his team, we said, what's in a smoking cessation app? Let's dive down into that. So we did a search of the Google Play stores and the iOS app stores in 2021 using four really common search terms that any average consumer might use if they were to open up their app store and say, I want help quitting smoking. The apps were visually screened for relevance, and then we had two trained coders who rated all the apps using the APA app evaluation model. And I'll talk a little bit more in detail on the next slide about what that covers. And we characterized the apps that were available in July 2021, and then again an update in November 2022, just to see what apps were still available. The citation for this, if you want to read further and dig into it, is available here as well. So the app evaluation framework is based on the APA app evaluation model, which has 105 objective questions assessed across six domains. So you can read more specifically at the mindapps.org or at this publication print here. So here are the six objective domains assessing things like functionality and accessibility. What is the cost of the app? Is it available in Spanish language version? What are the inputs and outputs like notifications and what kind of data are going into it? What's the privacy and security of these apps, which is important when we're talking about health care and health care information that's being obtained? What are the data sharing policies? What's the evidence and clinical foundation? Are these apps actually helping people quit? Is there any evidence that they're successful and effective? And what are the features and engagement style? This is probably often what we think about with the apps, right? What's the gamification? What does it look like? And what are those key skills that are actually being transmitted through the app? The benefit of this really objective app evaluation model is that it is standardized and can be used to assess across conditions. So here are two other related publications, one assessing apps for bipolar disorder, one assessing apps for suicide prevention and depression, just as other examples of how this app evaluation framework can be used. So what were the results here? So first we identified 389 apps, which may be no surprise to anyone. We had hundreds and hundreds of apps to weed through. We pretty quickly excluded 41% of these. So most were totally irrelevant to quitting smoking. Many were not at all functioning. So they were still in the App Store, but you clicked on it and absolutely nothing happened. So there's a lot of just defunct technology out there. And so what were these irrelevant apps? Most of these were like a cigarette simulator or a map showing places where you could smoke and kind of get away from smoke-free laws or places to show where you could get the cheapest rate on your product that you were looking for. So none of these would be helpful were you actually looking to quit smoking, right? If anything, you know, they're just kind of perpetuating the continued tobacco use. So what we did rate was the 228 apps that remained, and here's the availability across different platforms. And interestingly, when we went to revisit this list in 2022, 31% of these apps were just no longer functioning. So there's really not a lot of longevity, not a lot of maintenance of these apps for ongoing support. So what do these apps actually look like? So I'll walk through some of the features. So in terms of cost, 83% were free to download, but 61% had some cost embedded into it once you downloaded it. So an in-app purchase or subscription cost, so not really free to the user. In terms of data privacies, I always find this so interesting. So 37% collect and share some level of PHI, and they say that, that they're going to collect and share your date of birth or your mental health information that you provide. But there's also very few data security measures. So only 39% had any explicit data security measures. Only 9% allowed users to opt out of data collection, and 1% reported that they were HIPAA compliant. In terms of evidence and clinical foundation, so every single app said, we are here to help and we're going to help you quit smoking, right? They all had these great clinical claims. 6% had supporting clinical studies. So those were studies of the feasibility, the usability, the efficacy. So very few of these actually had any evidence-based support. So in terms of clinical features, so self-monitoring is one tool that we know is really helpful in behavior change generally, and especially in quitting smoking. So kind of tracking when and where someone smokes. And so this was a common feature in many of these apps. So about 64% had some kind of goal setting or habit tracking. So, you know, might be a valuable tool in that respect. 51% allowed for other tracking, tracking of symptoms, but fewer allowed for tracking of things like mood or medication, which might be other areas that could be improved upon. In terms of other clinical features and intervention skills, so there are also a number of evidence-based interventions that we know are helpful. Things like psycho education, mindfulness, CBT. And these are present at very low rates, really across the board in these apps. So kind of less than a third, or, you know, as little as 2%, having some kind of evidence-based foundation. If you're really looking for a CBT app to help you quit smoking, you're going to be digging through over 300 apps to find those four that are available to you. So this really raises some gaps and opportunities for further development. It's not surprising here. It's very challenging to find relevant apps with empirical support or with evidence-based skill training for smoking cessation, which really emphasizes the need for a holistic clinical tool. How can we better harness our resources? And so mindapps.org is an incredible resource. It's an app library that you can really search through all of the available apps that have been rated based on this comprehensive evaluation. And so here's just a snapshot of what this might look like. It's kind of small, but you can see there are filters here, so you can filter by cost, you can filter by a supported condition, you can filter by specific treatment approach. And then there's an array of apps, so you can see, is this compatible with my iOS or Android device? Is this free or is there a cost to me? And so it's really a great tool both for average consumers as well as clinical guidance and clinical tools. If your patient says, I want to use an app, right, this might help eliminate the needle in a haystack search process. And there's also a huge need for more empirically supported and skill-based apps, right? So cataloging all these apps is incredible and there's still a lot of growth, an area that we need to advance on. And we really need to examine the best ways to utilize this technology alongside patient care. For many people, an app is not going to be enough. It's not going to be a standalone support that's going to get them all the way to their goal. So how can we use this alongside medication or alongside other technology? And that's where I think there are some unique opportunities also within wearable tech. So a bit about wearable tech, if you're not familiar with this space, although many of you may be, which is why you're here and interested. So 33 to 45 percent of adults already own a smart watch. Any raise of hands? Everybody who's got, yeah, many people have them on. And 92 percent, almost everybody who has one, is already using this to track and monitor some facet of their health on the watch. It almost makes it just part and parcel to putting it on. It's so intuitive and can provide real-time feedback to specific health status. And the growth in the wearable market can provide unique opportunities for real-time assessment and potentially real-time treatment and intervention. And specifically within the wearable devices, there's gesture detection that can identify smoking patterns. So here's a visual representation of this. So there's a SmokeBeat system, which basically uses the embedded sensors in a smart watch and a smart band and can, with machine learning, really draw out these patterns of gesture behavior and see the timing and the placement and really seeing that smoking, this hand-to-mouth repeated gesture, has a unique signature, looks very different from other repeated hand-to-mouth gestures like brushing teeth, eating or drinking. And so wearable smoking detection is a new system that might help enhance the care and treatment opportunities. The SmokeBeat system works with existing smart watches as well as just low-cost smart bands pictured here. It's universally compatible with iOS and Android systems. And in early validation studies, we have really high rates of smoking detection, accurate detection, with very few false alarms. There was a randomized trial that was done early on comparing wearing and using the SmokeBeat smoking detection system for 30 days compared to a weightless control, and they showed significant reductions in cigarettes per day. And so this leads me to think, well how might this wearable technology be helping somebody quit smoking? And to me I think it really comes down to this idea of self-monitoring, because quitting smoking requires changing automatic habits. For many people, the act of just picking up a cigarette, lighting it, is so automatic. And that automaticity is really a key feature of dependence and a key marker of relapse. And so if you think about it, quitting smoking requires not doing that and doing something else instead, really being aware of all these automatic behaviors, cues, triggers in the moment, and choosing an alternative. And so this is where self-monitoring can be really helpful in reducing automaticity, because self-monitoring is event recording, and it can bring awareness to behavior and associated cues. Traditional self-monitoring might look something like this. An upgraded sense might be, you know, one of those other apps I was showing you, right, where there's a cigarette tracker, where you can log each and every time that you have a cigarette. But we know for self-monitoring to be effective, it needs to be consistent, it needs to be accurate and timely, happening in the moment when it's happening and happening each and every time. And so there are a lot of problems with traditional self-monitoring, either pen and paper or even app-based delivery, and that there's low compliance, high burden, and so the resulting data is very unreliable. And so this opens an opportunity perhaps for wearable self-monitoring, right? You're already wearing this gesture detection system and it can allow for passive and continuous monitoring of smoking behavior. And so this led me to question, can smart band technology improve clinical care? And so this was a question I sought to answer through a clinical trial. So this is a visual depiction of the study timeline. So it was an eight-week study and everybody received a standard clinical care treatment. So this included up to eight counseling sessions and pharmacotherapy from their clinical provider. So this was a standard outpatient tobacco treatment program. Everyone received the standard treatment in both the intervention and control. The intervention group received the smart band to wear for eight weeks that had that passive continuous self-monitoring, and the control group had a sham smart band. So it wasn't connected, it wasn't providing any real-time feedback, but it did control for the demands associated with having to remember to wear it and charge it and bring it with them day to day. So how did we do this? We recruited adults from two outpatient hospital systems through Yale New Haven Hospital. And here's some brief eligibility information. So they had to be adults smoking cigarettes daily and seeking tobacco treatment and owning a smartphone. We excluded for very little but really just people who were acutely unstable and needed medical or psychiatric care to stabilize first. And we also did exclude if people were using other tobacco products or daily combustible cannabis use just to limit misidentification by the smart band. The smart band is really able to detect that gesture, that smoking gesture in real time, and we wanted to really focus on cigarette use for this first trial. So here's what the participant app interface looks like. So they're wearing this smart band and safe being was the name of the app. And so it says safe being has detected smoking. Did you really smoke? And so they could note in real time, yes, yes, I was smoking, or no. And so if it was a false positive, that information would be iteratively included in the data learning algorithm to improve the detection. There was a continuous cigarette counter so they could see over the course of the day, the week, the month, what their use was. So this is kind of that self-monitoring component but in an automated way. And here's an ability also if the smart band did not detect an event for some reason, that could also be manually reported. And so we were able to again update and iteratively improve the detection system. There's also a researcher and clinician dashboard that's accessible. So you know on the clinician side you're able to get real-time information on a patient smoking behavior. So here's an example of what that might look like here. So to kind of orient you, each column here is a day. The gray is before somebody enrolled. So gray is just no connectivity. The band's not on. White is showing that the band is on. There's power to it. And blue is showing some level of activity, like the band is actually being worn. And there's, even as I'm talking right now, my watch is probably going wild, right, with just all the gesture detection that's going on. So you can tell that it's being worn by a person. And then you see these timestamped event recordings that happen. So the green is an automatic detection. The orange bars are those manual reports when a cigarette wasn't detected. And you could also visualize if false detections happened, for example. So here's the participant characteristics. We randomized 58 participants. And here's the demographic information. And here's some baseline tobacco use characteristics. This really just emphasizes that randomization was successful. We were balanced across both groups in terms of quitting importance, confidence, average cigarettes per day, for example. So getting to some of the results. So in terms of feasibility, we had some early software updates that were implemented to improve connectivity, which also happened at a serendipitous time. So we were enrolling through March 2020 when all of our tobacco treatment clinics shut down and had to go fully remote. And so we kind of paused and we had to really restructure and revise the entire study. So instead of in-person patient visits, we relaunched in August with a fully remote clinical trial and with an updated band software. And so we over-enrolled in the second part of the study really just to look at what this might look like in this more, you know, new-age remote delivery model. Participants wore the smart band on average for 43 days. And there was no difference between the intervention and control group, which I was so surprised by. That control group who, you know, they weren't getting any benefit out of that band, but they still wore it, they charged it, and there was no difference between the old and new software versions. And in terms of connectivity, we see really high rates of connectivity. So on average, the band was active and connected to their phone for over 22 hours each day with at least nine active hours. So knowing that that band was on and being worn. So in terms of acceptability, these are ratings of strongly agree or agree. So 78% said the smart band was easy to use. 56% enjoyed wearing it. Very few, about 16%, said it interfered with their daily activities. But 50% said it was difficult to remember to wear each day, right? This is not something that was a part of their normal routine. They have to wear it on the hand where they normally smoke, which might not be where they would normally wear a wristwatch or something like that. So there are some feasibility things to consider. In terms of helpfulness, so this is rating from not at all or very dissatisfied to the other extreme, extremely and very satisfied. And we have high ratings of satisfaction overall. High ratings of quality of treatment and willing to use the smart band again. 94% would recommend the program to others and 83% would recommend the smart band to others. And in terms of smoking outcomes, so these are our smoking outcomes at eight weeks at the end of treatment. So the darker bars here are the biochemically confirmed seven-day point prevalence abstinence, and this was measured with expired breath carbon monoxide. And then the lighter bars are the percent days abstinent during the treatment period. So this was very small sample size, not statistically significant, but we do have double the benefit here within our intervention group at the end of treatment as compared to the control and an overall small effect size. So this leads to kind of next steps and ongoing work where to take this from here. So I'm also right now involved in an ongoing study where we're testing the dissemination of the smart band as an adjunct to population level treatment. This is funded by the American Lung Association, and we're using the NCI text message program, which is freely available and provides a high-reach population level intervention. But we know that that alone may not be enough for many people to be able to quit, so can the smart band help increase and kind of augment treatment? Even with a low or small effect size, if you can reach people at a population level, there's a high potential to really impact a lot of lives. So we are recruiting adults across the U.S. who smoke daily, and they're randomized to receive either smoke-free text alone or smoke-free text in the smart band intervention. And we're assessing seven-day point prevalence abstinence at the end of eight weeks, and then also looking at secondary outcomes during that treatment period. So stay tuned, and I'll be excited to report back, hopefully soon, about some of these ongoing study findings. And there are some other interesting, and I think important, developments for the smart band work. So I'm also working to develop and test an e-cigarette detection algorithm. As I said, e-cigarettes are the second most common tobacco product used among adults, and they're the number one most popular product used among youth. And so finding better ways to assess and treat e-cigarette use will be an important area for future work. So I conducted an initial laboratory pilot where we had standardized puffing that was occurring, so we knew exactly when and where that puff was happening while someone was wearing the smart band. And we first compared to the existing smoking algorithm with the smart band, and there was really low sensitivity, low overlap in what smoking behavior looked like and what vaping behavior looked like. And so we used the machine learning analysis to classify a new e-cigarette gesture and greatly improved the accuracy. So this really suggests that e-cigarette use gesture patterns are different and can be differentiated from smoking, which would provide some unique opportunities for real-time assessment. As a tobacco researcher and a clinician, I'm often asking, okay, so when did you use that? But when did you use this? And people have such a hard time even just recalling which product they were using when, and also opens up a number of other treatment opportunities as well. So in terms of leading us, again, I think a lot of this talk and presentation is also kind of future forward, so gaps and opportunities for further development. I think these data really suggest that wearable real-time smoking monitoring can improve smoking treatment, and that there's a lot more growth and more research that needs to be done to know how and for whom. So we need large-scale trials to examine dissemination potential and potential population level impact for these kinds of digital tech tools. And important to identify mediators and moderators of treatment. So who is going to benefit from some kind of intervention like this? Who's going to need additional, you know, in-clinic based care? And what are these important mechanisms of treatment? Is it that this wearable technology is helping reduce the automaticity of smoking? And if so, does that also open up new opportunities for treatment? So can we examine ways to optimize real-time intervention delivery in that moment, you know, in anticipation of a smoking event? Can we encourage medication use or use of alternative coping skills, for instance? And some, a number of opportunities as this e-cigarette algorithm expands to think about assessment and intervention opportunities for adults, for youth, and potentially a number of other applications as well. So I'm happy to talk with anyone who's interested in this space or maybe who wants to get involved and think about these areas forward. And with that, I'll just thank you for your time. And I'd like to highlight and acknowledge a number of the team members from the lab who've been integral in doing this work. Thank you. We'll save questions for the end. Okay. Great. Well, great to have you all here. Thanks so much for coming to this session. I know there's a lot of fantastic programming happening concurrently at this time, so thank you for choosing to come here. And John, thank you so much for organizing this. I'm going to shift to talk a bit more broadly about digital therapeutics in the space of substance use disorders. And you've heard examples as applied to smoking and e-cigarettes, but there's a tremendous scientific literature that really underscores a really strong clinical value of digital therapeutics that are really well developed and that really have key mechanisms of action to impact people's lives, not only in the space of substance use disorders, but very broadly in mental health. So we're really excited to hear what's on your mind and what you're thinking about in this space, and we've sought to preserve a good amount of time on the back end of today's session so we can have a discussion with all of you. I think that, I hope you'll see from my brief remarks here today, that there's, I think it's a very exciting space. I think that there's very compelling clinical data that shows considerable clinical utility of these tools, and I think that we're seeing not only a rapidly evolving application of digital therapeutics in in our models of mental health care now, but it's undoubtedly a key part of the future of psychiatry and how we're going to have models of the future of mental health care. So I would love to hear from you all about your thoughts in the space and what you think are key considerations as we move that forward. So I, as John mentioned, my name is Lisa Marsh. I'm a professor at Dartmouth and lead a national research center focused on applying digital health, particularly focused on digital therapeutics in the space of mental health and substance use disorders. And in addition, well first I want to acknowledge funding. We've been very fortunate to receive support from the U.S. National Institutes of Health, including we have a Center of Excellence grant mechanism from the National Institute on Drug Abuse. We're now in our, just starting our 13th year of funding as a national resource in this space. So I'll mention at the very end a little bit more about our center, but if you all are interested in this space, if you're thinking about application of this work in your clinical practice, we would be delighted to share resources with you that might be of interest. In addition to my academic affiliation at Dartmouth, I am affiliated with a few small businesses that are involved and also a global company involved in this space of digital health and digital therapeutics. Our center, which again I mentioned to you, has increasingly been growing our partnerships to really have work with the right strategic partners to really scale the most effective and engaging digital therapeutic resources to impact people's lives all over the world. So this term is probably one you've heard before, but just to start with what this means, digital therapeutics is a term that refers to software that is effective in preventing, treating, or managing a health condition. So it's not a sort of a general wellness promotion app, it's really taking a medical-grade intervention and delivering the active therapeutic ingredients of that intervention through the functionality and the content of software. So it's not a synchronous communication with a clinician like we see in telehealth, but the actual ingredients, the therapeutic model, is delivered through the software on a digital delivery platform. So it can surely complement and extend our traditional models of in-person care and telehealth care, but the idea is 24 accessible therapeutic tools available through software, and these can include behavioral treatments among others, and we'll talk about many examples. Within that umbrella category of digital therapeutics, there's a specific subcategory of what are called prescription digital therapeutics, and I would imagine many of you have heard of this and maybe are working in this space, and these are a particular type of digital therapeutics that have been authorized or cleared by the U.S. Food and Drug Administration to be prescribable. So clinicians can prescribe software that has this designation across the U.S., and to get this designation, you have to go to the FDA and present data showing that your software is safe and effective in preventing, treating, or managing some health conditions. So you're seeking a label, you're trying to make a claim, and you have to show some really robust data demonstrating safety and effectiveness in order to get this designation. I've been working in this space for a long time, and this was something that was a very exciting development that first launched in 2017, because we've seen, and you saw it from Kristen's presentation, if you go to the App Store and you search for smoking apps, you get all kinds of variability in what pops up, including things that claim to have empirical support but have none, some potentially may be harmful. It's really hard to navigate the space, so if you are a provider and you say, well, I'm excited about apps that I can have in my toolbox as a clinician that I can recommend to my patients to add value over and above what I'm doing in my clinical work with them, how do you know what works? So this is a really important development, because you know that there are very high standards, a very high bar you have to reach to get this designation from the Food and Drug Administration to have the ability to prescribe a digital therapeutic in the U.S., and we can talk more about what's happening all over the world. There's a lot of development all over the world in what people are doing and deploying these types of tools. So Kristen talked about this a bit, so I'll just kind of move through this quickly, but why is this of value? These can be additional resources that clinicians can provide to their patients. It's 24-7 accessible to your patients when you're not with them. That can be a therapeutic support in their pocket, accessible anytime and anywhere. We know we can provide resources with fidelity to best practices in the way that these are developed. We have seen that not everyone in the world yet has access to mobile devices, but all the data show that even some of the most traditionally underserved communities either have access to mobile devices or are expected to get access to mobile devices in many pockets of the world, not just high-income countries. So what a tremendous opportunity to harness the widespread availability technology to think about personalized anytime, anywhere resources in health care. So the scalability is very exciting, as is the ability to ensure quality of care. So consistent with the theme of this panel, I'm going to show you some data from several studies that particularly focus on the application of digital therapeutics to substance use disorders, and then we'll talk more broadly about what we're seeing in a growing scientific literature and an exciting clinical application of these digital therapeutics in mental health more broadly. So the first study that I'm going to highlight was one of the early ones, and I'll just briefly describe the designs of these studies, and I'm happy to follow up with anyone who would like more information or papers or whatever would be useful. This was a randomized clinical trial with adults with opioid use disorder entering outpatient treatment for their opioid use disorder, and as you all likely know, we have literally life-saving medications that are critical as part of treatment for opioid use disorder. So everyone in this study received an FDA approved medication for treatment of opioid use disorder, but they were randomized to receive one of three different types of behavioral therapy on top of that pharmacotherapy. Okay, so three-arm randomized controlled trial. If you went into the condition that's reflected here in the blue column in the center, you were randomly assigned to meet with a clinician who was highly trained to deliver with fidelity one of the gold standard behavior therapies for substance use disorders. It's called the community reinforcement approach to substance use disorder. I see heads nodding, maybe you're familiar with this. It's a pretty intensive behavioral treatment that helps people really understand and disrupt harmful cognitions and behavioral patterns around drug-taking behavior. So you met with the therapist three times a week in one-on-one sessions if you were in that condition. If you went in the condition that's reflected in the red column, you had a clinician, you checked in with them every other week, but the therapy that you experienced was offloaded entirely to a digital delivery platform. Okay, so you receive this digital version, this digital therapeutic, of this gold standard behavioral therapy. And then if you went into what's reflected here in the gray column, that was reflective of standard substance use counseling for treatment of opioid use disorders in the U.S. at the time of this study. So again, this is just a snapshot of lots of data that we have, but what you see is that even when you offload the bulk of the therapeutic intervention to a digital delivery platform, you can get comparable clinical outcomes to exclusively clinician-delivered care. And both of those sort of gold standard therapeutic approaches produce significantly greater outcomes than our standard models of care. This is on an outcome measure of objectively measured urine toxicology testing, where we're actually testing for drug use. So that's one example. Here's another example. We did this study actually here in New York City, an outpatient methadone treatment system in the city. And this was a study, again, with adults with opioid use disorder. And these were adults entering outpatient medication treatment. So they could go into treatment as usual, and as you likely know, in a methadone treatment model, that includes access to daily methadone medication, and it includes some case management and counseling offered in that setting. Or folks who joined the study could have been randomly assigned to a condition where they received methadone medication daily, they received all the other types of resources at that site, but their patient-clinician time was cut in half, and the other half of that time they spent interacting with this digital therapeutic. So let's say instead of meeting with a clinician for 60 minutes, they'd meet with them for 30 minutes, and then they'd spend the other 30 minutes interacting in this individual personalized experience with this behavioral therapy that was delivered on a digital platform. So as you can see from the data, when you cut in half patient-clinician contact time and replace it, that other half of the digital therapeutic, we had significantly greater documented opioid abstinence in that sample compared to the sample who received standard care in that setting. We could talk a lot about why that's the case, and happy to do so, but just briefly, we're in a really individualized way delivering a very potent evidence-based therapeutic intervention with fidelity to every individual through this digital platform. We had the chance in this study to track patient outcomes for one year, and this clinical benefit you see reflected in this average summary statistic, that clinical differential persisted for a whole 12-month period where we saw a benefit for people who had the digital treatment as part of care versus those who didn't. And then here's yet a different model. Let's say you didn't want to touch the underlying model of care at all in a healthcare system, and you just wanted to say, what about if I just add this as a supplement? People walk in the door, they get our traditional model of care, but on top of that, we say we could prescribe to you a digital therapeutic for your use along with our care model. And that's an example of this study, and this was, again, another example of patients with opioid use disorder. This is not the only population we work with, but as you all know, we continue to have a really striking opioid crisis in this country and very high overdose death rates, and extraordinary need to scale effective resources in this space. And in this particular study, when people came into outpatient medication treatment, they were either in standard treatment or they were given a digital therapeutic, the app that I just briefly described to you as part of care. And what we found is that when we gave people this app as part of care, we were able to retain those people in substance use treatment longer. And we know treatment retention is a really important predictor of much longer term promising clinical outcomes in folks with substance use disorder. So this is looking at the first three months of their treatment entry. We kept about 86% of the sample who got the app in treatment versus only 56% of the sample who didn't get it and had otherwise the same type of treatment. And then the same pattern emerged when we looked at drug taking behavior. So again, objective toxicology data from drug testing where we had more documented weeks of opioid negative samples in this population when you got the app versus when you didn't. So these are just snapshots of data but there's a really compelling literature that shows similar robust clinical effects in clinical trials. But I think what's also exciting is that this particular tool made it into the real world outside of clinical trials and I was delighted to see that the data we saw from people who used it sort of in the wild outside of a clinical trial, so these aren't research participants, had even stronger clinical outcomes when this was part of their care. So here's one example for the version of this that was for treatment of opioid use disorder. In the pivotal study for that that FDA looked at, we saw that about 77% documented abstinence in weeks nine through 12 of that intervention in the pivotal study. But when you go into the real world evidence, people using it outside of a study, we saw 91%. And same sort of pattern popped up when we looked at what we call the responder analysis, which is people who had at least 80% or more documented negative urine samples or self-report for opioid use disorder. Again, a higher rate of documented abstinence in the real world evidence study than in clinical trials. So those data were very exciting as were some of the striking healthcare cost reductions we've seen. So these are data that the state of Massachusetts published recently and it reflects data from the first cohort of residents in the state of Massachusetts who were prescribed this digital therapeutic for treatment of opioid use disorder. And what Massachusetts reported is that they saw in that cohort a 45% reduction in emergency department visits in that sample and a 64% reduction in hospitalization. So the data are really compelling. The clinical impact as well as when you think about if you invest in something like this and give it to your patients, not only the downstream clinical benefits but the potential cost saving benefits could be extraordinary because we know these are very costly healthcare services. So these are some examples in the addiction space but given the audience here, you should know that there's a growing, really robust now science that shows pretty striking benefits of these digital tools for lots of areas of health. Most of the work to date has been in mental health and there are incredibly promising data in mental health including decreasing mental health symptoms in folks with things like ADHD, anxiety, depression, PTSD, OCD and schizophrenia. Improving remission rates, improving people's lives, functional outcomes. I'm highlighting a lot of examples in mental health because this is where most of the science is in clinical application. But we're also seeing value in things like irritable bowel syndrome and migraine and hypertension among other areas as well. And I showed you some data showing impact on healthcare costs but we've seen that with other populations as well like patients with chronic pain, reducing ED visits among that sample when they have a science-based digital therapeutic as well as supporting medication adherence and other types of medical regimen adherence. So there's a lot of work happening in this space and really strong clinical evidence. And I wanted to just briefly tell you about a study that we did to show how you can take the science from these types of clinical trials and think about how do you transform models of healthcare delivery in particular mental healthcare delivery by using science-based digital innovation. So I'm gonna give you an example of a study just briefly that we were funded to run by the U.S. National Institute of Mental Health. And this is a project we did with a broad array of partners in Latin America. And we started in the country of Colombia and I highlight this because Colombia is just an example of a part of the world that has a really high mental health need and a limited capacity to reach that need at a population level. So Colombia has a long history of conflict and very high, well-documented rates of depression, high levels of unhealthy alcohol use often accompanied by domestic violence. But the mental health work, the psychiatrists in Colombia are in Bogota or in very urban settings. And if you go to rural farming communities, there's really no general mental healthcare other than inpatient psychiatric hospitals for high need. So this, you could think of rural America, you could think of many pockets of the world where we have a high need, high unmet need and limited workforce capacity to meet that need. So this is what we were seeking to tackle here in this project. And the way we did this was to work with primary care across the country. So primary care was much more accessible across the country. Primary care had never before screened for or treated mental health at all in primary care in this country. So there was no screening for depression, anxiety or any types of mental health embedded in routine care. So surely there was workforce development in embracing that as part of broader primary care, but we offloaded the bulk of the therapeutic intervention for screening and treatment to an integrated suite of digital health tools. And again, the idea here was to bring these tools in to help facilitate the adoption and the opportunity to scale and sustain this in that system over time. So we took clinically validated screeners for things like depression or for unhealthy rates of alcohol use. We automated the whole thing and then those data would feed into an automated clinical decision support tool that the provider would use to help do diagnosis and identify what would be best practices for treatment for a patient who had a diagnosable mental health challenge. And then everybody got a digital therapeutic that it's sort of a transdiagnostic digital therapeutic that we've developed based on a couple of decades of NIH supported research, but then we customized it to the region, the culture, the language with partners there so that whatever combination of mental health needs someone had, they could flexibly use that component of the broader digital therapeutic platform. So digital screening, these are just some screenshots to give you a flavor for what patients saw, what providers saw with their clinical decision support tool. And then these are some screenshots of the digital therapeutic. It's really around helping people identify who do you wanna be, what's getting in your way, maybe it's depression, how can we help provide resources to help you achieve goals and very focused on sort of strengthening and using these tools to help achieve that. So just a brief bit of data. Here's a couple of year window from that study. It ran longer than that, but just in this snapshot, we went from screening no one at all in primary care in the country to in the study with 22,000 people engaged with us. And of those folks, 22% had a positive screen for a mental health challenge and then 8% had a diagnosis. So again, we're flagging for the first time this unmet need in the system of care in the region. And we had a chance to, we had enough staff on this project to track all those patient outcomes for one year. And there's so much more data from the study I could share, but here's just a snapshot showing the evolution of depression symptoms among those who had a diagnosable depression disorder. And we saw a marked drop in depressive symptomatology over the year, even among those who had pretty high levels of depression symptoms at baseline when they joined the study. And here's similar data over the course of the year for folks that had problematic unhealthy levels of alcohol use, even for people who had very high levels of alcohol use at baseline. One of the striking factors with this was the cost of this. So our partners, the Ministry of Health, the payers in the region, and healthcare administrators very interested in knowing if we adopt this new clinical workflow and embrace mental health as part of our broader care model, what's the cost of that? Is it gonna be sustainable? So we did a very detailed costing process mapping using a tool that came out of Harvard called the Time-Driven Activity-Based Cost Metric. And it's a very detailed process where you get all this granular data about the cost of every process in a clinical workflow. And the bottom line of a very long process was that after people adopted this novel model of healthcare, it only cost $1.89 US dollars more per patient per year with this model that embraced mental health care versus their prior model of healthcare that didn't include mental health. So people were pretty excited about this number, particularly in light of the robust clinical effects they saw on patients' outcomes. And again, I just highlight this to illustrate the opportunities we have for creatively thinking about how can we weave digital health tools and digital therapeutics into our clinical workflow in a way that adds value and not burden and not extra cost. And this is just one example of many. And then in the final minutes I have here, I just wanna talk a little bit more. You heard about this from Kristen. I think, John, you had a session on, I heard on this theme broadly yesterday. But one thing that's really exciting in this space as well is that these mobile devices can be incredibly meaningful therapeutic tools as we're talking about, and can give you these very meaningful digital interventions in your daily life. But you can also get a lot of information about people through digital platforms. There's so much rich information you can learn about people in their daily lives that can help us have much more precision digital therapeutics, much more personalized for what you as an individual might best benefit from right now. And this is a very exciting area of research in mental health, including in substance use. And again, I'm gonna highlight the substance use example here, including a study that we were able to do with funding from the National Institute on Drug Abuse, the Clinical Trials Network. And again, we recruited patients in outpatient medication treatment for opioid use disorder. Everybody's getting medication. As I mentioned, it's been very well demonstrated that these medications literally save lives and markedly reduce mortality. But we also know people relapse, and they sometimes don't take their medication, or they drop out of treatment. And we have clinical data that helps us understand what a risk profile might look like for that. But the study here is trying to understand, could we, through digital data capture in people's daily lives, understand for a given individual what confluence of factors might predict for that person, they might be at risk of relapse of fentanyl use now, or not taking their medication, or whatever the clinically meaningful event was. So we did this by gathering data through asking people to respond to queries, these EMAs, these ecological momentary assessments, brief questions, you prompt them to answer on the phone, through pulling a lot of rich information from sensing data on smartphones and smartwatches. We also asked them if they'd let us look at their social media data, to look at things like the sentiment of their communications or social network, and to help us better understand a given person's trajectory. So you've heard about this, but there's so much rich information you can learn from passive data collection about people's activity levels, about their sleep, their light exposure, sociability, lots of different variables about people's daily lives. Here are the specific examples of what we collected in this study, including very specific metrics of sleep, and craving, withdrawal symptoms, and pain. We developed and validated a momentary self-regulation scale that measures four different dimensions of self-regulation in the moment, including emotion regulation in the moment, and we asked people to do that as well. And just briefly, there's been a lot of exciting research in this space as applied to other areas of mental health. This was, I think, maybe the first study applied to population with opioid use disorder, so we wanted to look at people even gonna do this. We were excited to see that people carried the phone on 94% of days they wore the watch, 74% of the days we had high response rates to our questions, and the vast majority, higher than we predicted, agreed to share their social media data. We have lots of papers coming out from this, but just the bottom line is that we're finding some really promising results that show that we can build models that predict really meaningful events, like non-prescribed opioid use, like fentanyl use, or non-medication adherence, and right now, a lot of our models are predicting that about seven days in advance of the event, including this momentary risk-taking measure. One of our measures on that momentary self-regulation scale was one of the strongest predictors of that. So just to underscore the promise here, so I hope you've seen from this very brief overview that we can see a really compelling, striking clinical effect of these types of digital tools for lots of different health domains, particularly in the realm of mental health and substance use, and we have tremendous opportunity to leverage these tools to scale up access, I think, to the most effective resources, and really think about population-level impact, right? In the work that we do, we often have long wait lists for patients, we have lots of unmet need, we've had a surge in global mental health challenges. This is an opportunity to not solve that just with digital health tools, but what an opportunity to leverage this to accelerate the reach and the impact of the work that we do across the globe, and I think also to capture lots of very rich information in real time and over time to provide very personalized precision care models. So last slide, if folks are interested, I guess the formatting got a little off here, but if folks are interested, this is our contact information for our center, I'd be happy to talk to anyone who's interested in this work. We have a lot of exciting work, all the way from early development to lots of translational work, but we're not just interested in the science, we're interested in impact. So we would love to hear from you all about what are your priorities and questions in this area, and to think about how can we ask those questions in meaningful ways together and think about implementing things that work in real-world healthcare systems. So thank you so much for the opportunity to share this. Thank you. Perfect, so I'll conclude us with some extra, basically the part I wanna look at in this part of the talk is I think the evidence is there for a lot of these things, but the market is also expanding really rapidly. So how do you make a good decision about which one you wanna use? And then if you have a clinic, how do you begin to think about how to implement these ones? If you can find the right one, you've heard of many examples here are ready to go. I think we're at the point where we can make systems where we can begin to implement them. So these will be some examples of how to do it. And I want to first look at, there was a study that we did with a resident in our program. It was basically looking at different patients in substance abuse programs in Boston. And that he was looking at, are those patients who are in active treatment, what are they doing with technology? What are their interests in technology? What do they think? So this was done in 2021. And in essence, in the literature overall, 79 to 96% of people with the substance use disorders would have a phone that can do most of the things today. The reason that there's a variation we found the literature is in some rural areas, there was less ownership, but I think it's catching up. In Boston, at least where we're based, it was 96%. I would argue it must be probably close to 100% today of what it is. And what's interesting is we kind of ask people questions about what they do, what to do. And you could see 88% reported that they download apps on their phone. It's not a hardship or it's not a new thing to do it. And only 40% though had downloaded mental health apps. And we'll get into this, but it's actually a lot of patients aren't aware that these programs exist, right? Here we're talking about these amazing ones, but there's almost a public education campaign that say you can use these things, these exist, and you may not even realize your phone is a tool that can be used towards your mental health recovery. So there's some actually interesting low-hanging fruit that we really can do to expand the reach and bring new people in to use it. And of our 51 participants, again, this is more detailed, so 22 were from inpatient detox, 22 were from outpatient methadone clinic. And we asked people, what are you most interested in using your phone towards your recovery? And it's interesting, of the two clinics, kind of the total was pretty similar in it. You can see appointment reminders were interesting. Medication reminders were high up there too. Again, not a complex thing. So it's almost like an entryway to using a technology to say this is something that can help you with medications. People were interested in symptom surveys. Half of people almost were interested in sharing location with us of where they were. Social information was up there. So there were a lot of things that people wanted, but they really wanted also communication with a clinician, right? They didn't want the app to kind of replace or not talk to an expert, to a colleague, to a peer, to a therapist. That they wanted that connection as well. And then we wanted to ask people, what were things that could be a benefit? And what's interesting, and this is a larger trend in the US is privacy should be a benefit of digital. In all the studies you've heard of, privacy was more than guaranteed in the studies. And we can make things private. We have banking apps. We put lots of resources in them. So it's interesting. One of these things that people perceive, right, we can make privacy, it should be 100%. That should be a thing. And I think as we get better at digital privacy, and I'll give examples of where we're moving, I think this will be a huge change in the field that people will be excited. So I think you can see that cost, people are still worried about how to pay for these. We're still learning what is the right price point for them. But I also wanna focus on this bottom one, ease to set up. So I think for a lot of patients, I think for all of us in the room, smartphones are really easy to use. I think there's a group of people out there that may have a phone, and if we give them a little bit of digital literacy, teaching, and skills, we have brought on a new person who is ready to use these tools. And so it's almost, there's a lot of apps out there, but there's a lot of people, with a little bit of training and help, they wanna use the tools, and we can reach them. So we kind of asked the inverse question, what are people worried about? And you can see, again, privacy. But again, I would say we can make things private. We know how to do it. Of course, anything can potentially be hacked, but we can put better security. And there have been some recent examples we'll cover in the news, but usually when the data is leaked, it's because it was done on for a reason. It wasn't that a malicious hacker came in and stole the data, if you look at it. So I think realizing this, what I wanna focus on is just, again, how to find an app for your patient. We don't have every single app, and Dr. Bull talked about it. We have apps that are in the wellness space. They're gonna be $10 or less to ones I'm gonna talk about, so they're accessible to your patient. They're not yet to digital therapeutics because some of them are on the market, some of them are not on the market. So having fun, I asked ChatGPT to draw me a sea of apps. She did a pretty good job. If you get ChatGPT in a new version four, you can just ask it to draw you things. It's kind of amusing if you're nerdy like me. But I think the problem is, right, is there's some not great apps out there, too. So I asked it to draw me a bad app. This is what ChatGPT drew me. Again, not bad, a little bit weird, whatever that thing is, but so it did it. But I think what's interesting in thinking about these apps, and we'll get into doing it, is it's very hard to know kind of what's happening in deregulation or payment for these things. Stat News is not a regulatory authority, but this is from May 2nd, and it says lawmakers unlikely to permanently extend telehealth flexibilities this year. Again, we can talk into question and answer. I think telehealth works really well, but it's really hard when we have these, rules keep changing if you wanna build a business or a company or a clinic, and you don't know what's gonna happen in 12 months. So I think we still are in this interesting flux period. And I would urge all of you, the American Psychiatric Association, they're called the Telehealth Blog, they do a pretty good job of updating recent policy that's up there so you can get some sense of kind of what is gonna happen in this space, can you do it? But certainly a very challenging environment to be a provider, to building a new clinic, to setting things up. And I want to also talk about the regulatory system that we heard about. So if we have a brand new app in question mark, we all in the next 20 minutes build this wonderful app, we build a digital therapeutic, then we kind of have this choice of is it gonna be software as a medical device, are we gonna make medical claims or not? Most apps in the fine print actually go down to no, it's a wellness app. And then they don't really tell you it's a wellness app, but they go down that route. And it's very hard to know it. The FDA also has this bucket called enforcement discretion where they say if you look really low risk, maybe you can make some claims, we're not gonna enforce it. And then they have the traditional route for the 510K, you're like something else, you're an app like that, or DeNovo, you've never been seen before. And then I put the FTC because the Federal Trade Commission, for those of you who are not in the US, they have a post-market regulatory scheme. If this is confusing, you're right, that this is really confusing. And they actually had something called the pre-cert model that was gonna be a new way to fix this. And in September of 2022, they said we're not gonna have the pre-cert model. So what's interesting into wellness apps, there have been some changes. So this is from March 18th of this year. So it's interesting is basically the people that make the HIPAA laws are surprised in the US that even if you're in the wellness space, we're gonna begin asking you to follow HIPAA if you have personal health identifiable data in some use cases. So I think the wellness apps are beginning, there's gonna be a crackdown on them, which there needs to be because they've been doing some pretty egregious things with people's data. What's also very tricky about in this enforcement discretion space, these are some examples of software functions which the FDA will exercise enforcement discretion. So they say on number two, software functions provide periodic education information, motivational interviewing, smokers trying to quit, or people coming from addiction. They've used very broad, unclear definitions of what it means. And they don't really, it just muddles perhaps what the space is. Number three is a software functions that use GPS location information to alert asthmatics and environmental conditions. So it's a little bit confusing the US regulatory landscape of this. And I think we're gonna probably see in the next six to 12 months some clarity because it means that things that are not ideal can kind of slip in. And sometimes the things that have more evidence don't get the credit they're doing. And I think that's not good. If you have good evidence and you're better, you should be celebrated, you should be rewarded for that. But so given kind of it's the regulatory framework is tricky, we've had the APA app evaluation framework that Dr. Bold talked about. I asked chat GPT again to draw what it was. It looks like some like old alchemy book. I wish it was that cool. So I said, try again. And it gave me this really cool version. It's not actually that cool. It's really just a pyramid or a funnel. And the idea, right, is if you bring any app, you want it to be safe, you want it to be effective, you want it to be usable, and you want it to kind of have some purpose. The data can move to places, it can be integrated into care. So there's really nothing sophisticated about the APA app evaluation framework. And different hospitals over time have used it to kind of say, well, let's think about if patients bring apps, clinicians bring apps, which ones make it through our criteria for these ones, and if different people have different criterias for it. So I want to quickly run through the levels of why they matter, and then we'll look at the database of how you can access these apps today. Some of the things that Dr. Bold showed you, you can find all of those apps that she put into research, and you can see if they're a good match for you. But I think if we look at just even the ground layer, this is a paper we did in 2023 of looking at digital technology in different kind of substance abuse centers, and what were they recommending and doing. And what's interesting, I think, is some of them are beginning to uptake the technology, which is good, but some of the ones that are more, less resourced or rural areas are not recommending digital technologies yet. And I think it's, again, because they're not aware of it, they may not have the education, they're not sending people perhaps to this room to learn about it. So I think we have a very simple way to educate teams to do it, but if we don't, we're gonna see kind of some clinics adopt these amazing technologies and some not. So I think it's at the early stages, we can probably close that digital divide and make a large difference. I think if we look at the privacy, this is from April 11th, so you're getting the newest news, and it says, alcohol addiction treatment firm will be banned from disclosing health data for advertising to settle FTC charges at shared data without consent. So we're seeing the Federal Trade Commission. And again, you're seeing it's the privacy issue. It's not that it's being hacked. It's in some cases, it's being sold. And I think we've actually seen a lot of FTC action coming out in the last couple of weeks. So I think the FTC is sending a very clear signal, this will not be tolerated anymore. And as you saw, a patient's main concern is privacy. So I think we're seeing this very nice trend where the government is cracking down on the people not respecting privacy. And that's gonna really bring more people into the field. It's gonna highlight the ones that are done well. It's certainly a unique, you can read all these things. They're very interesting what the FTC writes about what these people are trying to do. I think another risk that's interesting, if anyone's from New York, this was a website called therapyforall.nyc. Sounds terrific, like who would not want therapy for all? So I only learned about this because a reporter called me, and it said for $50, you should record your treatment with your psychiatrist or therapist, especially substance use sessions. You don't have to tell your therapist, you're recording them. Just record them and send us the data, and we'll send you $50. So this website is gone now. And so I think what it was is kind of like, there's so much interest in building good large language models. They need data. And so I think we're seeing some interesting trends where privacy could be impacting you. It's not just our patients are at risk, so it's worth, it's not a perfect ecosystem. And again, this is also from April 11th, so these are all relatively new examples. I think where it's really interesting to digital therapeutics, and because the substance abuse space is more advanced, they've thought of this more, is how do we know a thing is working? Like what does a digital placebo or control group look like? And I would never ever recommend clinically Tetris for a person, right? Tetris is not a clinical intervention. What's interesting is in some ways this is a very interesting control condition because if someone were to say play Tetris on their phone or their app, is that giving them the experience as Dr. Boll talked about? You're wearing a band. You have to charge it. Is it giving you some of those experiences? And I think as a field, we're still learning about what is Tetris. Again, I got, I just asked Chat GP to draw me different things. But again, this was a older study, but certainly I think Tetris makes a good control condition, and we'll have to learn what it is. But this was a schizophrenia study that's not substance abuse, but it's an interesting condition where one group got an app for schizophrenia, one group basically got a, it says it was played a prescription timer, a digital clock for the remaining app duration. The sham app was chosen to account for the nonspecific effects of engagement with a phone. But in this study, in essence, what the result was was the sham app did pretty well as well as the engagement app. And it brings up interesting questions of kind of what is driving the effect? How do we make these better? How can we kind of harness that digital placebo effect to be useful? And I think that's an interesting question just when people bring us these things, we can say, well, is it better compared to what? Certainly compared to nothing, but if you can type Tetris on your phone, is that something? So I think it's an interesting one. The other hard part of digital therapeutics, and I'll tell you how our clinic is addressing this, is the engagement is tricky. And this was a study from 2019 by John Cain and Amit Bhanwal of just people using apps in the general public, not for prescription or clinical reasons, but the engagement is tricky. And I think we have to acknowledge when I give people medications, engagement is not high when I recommend therapy for people, the mean number of sessions is often one. So engagement is hard across all of healthcare. But I think how to make sure people get an adequate dose of these treatments is gonna be the new challenge we all have to work on together to think about it. And what's interesting is the US Veterans Administration doesn't put out any substance abuse apps I'm aware of, but they put out a lot of other apps. And they actually did an interesting survey of veterans and they showed that about 75% thought having this PTSD app was important, only 11% were likely to download it, and 1% actually did. So I think we have a lot of work again to do around educating our clinicians and patients and helping them move from, I'm interested, it's important to doing it. So you can see that we have a lot of people interested in through some education, some digital literacy, getting clinicians motivated, we can probably move the needle very quickly, or we can really make this field faster. So as Dr. Bolt said, we took these APA pyramid things and left, and we turned it into a database. And we said, we don't know who your patient is, we don't know what you want, but you may want to ask questions about privacy. So we have questions, is there a privacy policy? Is data stored on the device? Is it stored on the server? So you don't need to answer all the questions and you may not care where the data is stored. Your patient may say, I'm an Instagram influencer and I don't even want to answer these questions. Everyone knows my diagnosis. We have some very simple clinical questions because it's hard to judge the evidence. So does it do what it claims? It's a pretty low bar. We're not going for much. And we have other ones. We have, again, what engagement features. I don't know what's gonna make an app engaging for your person, but maybe they tell you they want video. Maybe they want AI. Maybe they really want Windows dancing chatbots, and that's the thing they want. So the idea is, you don't have to know all the questions, but we have the database, and it was briefly featured in New York Times, how to find a mental health app that works for you. And I made the mistake of reading the comments that people posted to it. And someone said, the best mental health app is already on your computers, tablets, and mobiles. It is the off app. When you run this app, all the news, advertising, and misinformation that makes you miserable magically disappears. You become immersed in a real world where you get to choose what you do and makes you happy. Thoreau would be pleased. I was so sad that I drove to Walden Pond, which is in Massachusetts. And I got to Walden Pond, and there was a sign that the Walden Pond and Woods app is now available. So I sat there and said, I don't know what Thoreau would think anymore. So the point being, apps are part of our life, right? It's nice to kind of have the sentiment, but this is integrated into what we do, and we have to work at this. So we'll finish our sidetrack on Walden Pond. But so again, we have the database, and all of the apps that Kristen Boll and her team put in, we still keep them updated. Every six months, our team goes through all those criteria. So you can go to mindapps.org. It works better on a computer. It's not the best interface for mobile. But you could kind of check in, show me all the substance of these apps. There's a lot more of them, and we'll present them to you. And then you could say, show me all the ones that are totally free, and we'll filter them. You could then say, show me the ones in Spanish, and we'll probably be like, we don't have anything in a database in Spanish. So the point is, we're not here to say an app is good, it's bad, but it's a more informed way to search. Unfortunately, again, these are all of the wellness apps, because we don't have access to all the prescription apps. These have to be something that you or your patient could download to look at. The same idea probably works for peer recovery forms. There's someone at Pittsburgh who did a very nice website of onlinerecovery.pitt.edu-nearfar. So we tried to write some of these kind of online recovery forms and see what they could do if they're engaging in it. And he gave a very nice talk with an SMI advisor called Considerations for the Use of Online Peer Recovery Forms for Individuals with Serious Mental Illness. Because it's hard to know, like, is this a good forum that my patient should go to? Are they talking to good people? What are they doing? So in essence, I think these apps can work well, but the engagement is hard. So I want to conclude with very quickly, is we have a clinic we run at our hospital, and we, in essence, deliver what I like to call hybrid or blended care. So we get patients who are referred from the community, they get a usual telehealth session up here, and we ask them, say, to use an app. And you could substitute any app. You could put a digital therapeutic in the bottom. But we know if we leave it just like that, the engagement's gonna be low, and the clinicians, it's not gonna fit into the workflow. So we've basically added what we call digital navigators or coaches that meet with the people after each session and kind of say, hey, Dr. Torres wants you to do this module around sleep. Does that make sense? Let's practice the skill together. I'm gonna check in with you if you're not doing your homework because the app knows. The dog cannot eat your CVT homework. So there's nothing magical here. And then the digital navigator will also write a report in the medical record and say, hey, John, your patient did homework. Their sleep score was different. Why don't you take a look? So I don't have to spend all my time going through the data. So I think the digital navigator is a new bridge for health technology. And they can also teach people how to use phones, right? We've talked about just making it equitable, having digital literacy. So the digital navigator can kind of solve two of the problems of increasing awareness, getting everyone on board, and using the health technology. So we've worked with other teams. This is a team in Nebraska working at the digital navigator model. And we've worked with SMI Advisor to build a free training. If you go to smiadvisor.org slash GHN, it kind of says, well, how would you help support some of an iPhone or Android, because you may know one, but you may not know the other one. How do you help them pick an app? How do you drive engagement? So I think it's a useful resource, smiadvisor.org. And with that, we'll stop and do questions. And remember, we have to do questions into the microphone because for people listening online. So, excellent. Thank you. Thank you so much for such a terrific session. My name's Sandra DeYoung. I'm based at Cambridge Health Alliance. And I was responsible for getting on the child psychiatry core competencies, a milestone that requires fellows to have experience in using a digital app as part of, before they graduate. So my question is about, if you could say something a little bit about apps directed at adolescents, particularly perhaps behavioral apps, internet addiction apps, that kind of thing, and cannabis apps. Those are the hot topics for us. Thank you. Yeah, I think probably others in the room would love to hear how you were successful in doing that, because that's fantastic that you achieved that. It's so important and so needed. You know, there are, there's a growing, well, I'll start, but others might want to chime in. There's a growing line of work focused on youth, including therapeutics for sort of promoting behavior change or people who have, you know, diagnosable problems, but also preventative, going upstream and thinking about building up, building in prevention science and building up protective factors and reducing risk factors for things like mental health, but also substance use, vaping, opioid misuse. There's a lot of work in that space, including serious games for kids that can really have active therapeutic ingredients in these interactive games that, you know, games that go above and beyond entertainment value, but really are. And there's very great data for the last, I'd say, at least dozen years that shows that you can get some really big effects with these types of tools for youth. I think there's only one, to my knowledge, others may know otherwise, but I think there's only one prescription digital therapeutic that has a pediatric indication right now that's been FDA cleared, I think. But I, and that's for ADHD, but I know that there are others that people are working on, and I think there's tremendous, you know, kids live on their phones, right? So let's go to where they are and think about how do we leverage that and engage them in new ways. So, but there are some exciting research projects going on in this space. And if you wanted to know more, including in your neck of the woods, I work with some colleagues of yours that are doing work for kids. Meredith Gansner is one example. I don't know if you know her work. Yeah, so glad to connect if that's, okay, perfect. Yeah, so, but feel free to follow up if it's helpful. Hi, I've met you before. I'm Jerry Cragen from the University of Toronto, and I'm working on a book out next year called Global Digital Mental Health Ethics, Assessing the Impact of Technology with Becky Inster at Cambridge. And one of the things that in doing this book that's come up is the, there's such incentive for health data from criminal perspectives. And so the most vivid example was, there's a number of swattings that's going on. I don't know if you know about swatting, where people are calling the SWAT team from other foreign countries and then engaging people's address to then, and deaths have arised. The worst example was in December at the Fred Hutchinson Cancer Centre in Seattle, where the hackers had gained the data and then had threatened the hospital, unless they were gonna pay in Bitcoin, they were going to engage swatting. So I do share the enthusiasm for this, but I also share patients' concerns around privacy. And I know that the cost of this is so enormous that the federal government here is thinking of trying to reimburse this because it's such a, I thought it was the lowest blow to take cancer patients' data and then start threatening the Fred Hutchinson Hospital. So I just wanted to take that in terms of these risks that we're talking about here. I'll start. I think all of us have probably had our credit cards stolen at some point in this day and age or got a letter. And I think certainly digital data is the most valuable health data. But overall, if you look at kind of how, let's call it well-qualified substance abuse apps and programs have done, those have not had any hacks or leaks, right? If you actually look at, even in the mobile health space in general, there has not been a mobile health app that has had a leak or there have been healthcare systems that have been hacked, but actually, I think we go above and beyond in security. And if you look at even the FTC settlement, it's like you can have the best security if you make a decision to sell the data and open up the moat. That's different. But I do think that there's always some risk. And when I talk to patients, I say, look, there's risk your bank account could be hacked. But I think we do a better job than we get credit for, but we sometimes incentivize selling the data. Right, and there's been other leaks in Europe as well. But so I just, I think that I'm all in favor of the approach. I just think that we need to take extra precautions before we're the next victim. Thank you. Thanks so much for the talk. My name is Jack. I'm a resident at the Brigham, across the street from John at the BI. Really great talk. I was pretty struck by, particularly, Dr. Marsh, the figures you showed around the digital therapeutics impact on reducing ED visits and hospitalizations. And certainly, that's good for patients. I think health systems and payers certainly would take note of that as well, given the just overall system ramifications. And I was just wondering if you had any insight into proposed mechanisms of how the therapeutic achieved that. Was it through medication compliance, greater therapeutic alliance, and also just how, I feel like implementation of these programs and engagement are big challenges, right? So if there are any pearls that you or others want to share around how you ensure good implementation and also patient engagement. Yeah, thank you for the question. So I think for the data I showed for the digital therapy for treatment of opioid use, the data I showed for the digital therapy for treatment of opioid use disorder, there hasn't been any really systematic sort of experimental study looking at the mechanism of those effects, to my knowledge. But I think one of the biggest drivers is that these tools have been reliably shown to help people manage their use and reduce their drug use. And so reducing things like overdose events or things that might drive you to the emergency department. We did it one state where we did look at this pretty systematically. We also did here in New York City with a healthcare system that heavily provides surgical and other types of medical interventions for people with chronic pain. And we studied a digital therapeutic for that patient population to see if we developed a digital therapeutic that helps people learn skills to manage chronic pain, to help them achieve goals in very personalized ways and not letting chronic pain sort of interfere with that, but having these personalized tools that they can use in their daily lives. What we found there also was a big drop in emergency department visits in that sample. And we did look systematically at mechanisms that drove that. One of the biggest mechanisms that emerged from that analysis was we reduced catastrophizing, sort of this cognitive process where you're catastrophizing over your chronic pain in the moment and believing it to be really bad, maybe worse than it is, and feeling like your solution to that is to go to the ED, right? Like in the moment I'm in this crisis. So helping you understand that thought process and have different ways of replacing those cognitive processes with a tool in your pocket right then available to you to help prevent you from saying the only solution right now is going to the ED. So that's one study that's looked at it, but I think it's a great question. I think we should be looking at that more. The data are promising, but we could benefit from more data. Thank you for the question. Hi. So I think that this has great potential. I think that if we do this right, the benefits are exponential in terms of cost saving for the patients, access, I mean, et cetera, et cetera, right? Now, how would you make it right, right? Implement this right? And one aspect of this is generating the data to support the use of these apps, right? In this day and age, I mean, anybody, any teenager in a basement can develop an app, right? And I think that you showed how many they are and they are really not useful. So how would we generate the data? Now, I know there are people, companies, and they are generating good data to support their use. Now that costs money, right? And then there will be a need to recover that money. So there will be a cost associated with the use of that. And then there is coverage and access as a problem. So how do we see, do you see that dynamics playing? Because on one side, I think we need to incentivize the creation, the production of the data. At the same time, we may have other competing apps of resources that may not be willing to produce the data, but they are competing in the same space. So how do you see that dynamics playing out? You wanna start? It's a hard question, no. Okay, I'll start. I think it's a great question. And I think that, yeah, there's a lot going on in this space, a lot of excitement in this space, a lot of hype in this space as well. And I think what we are gonna see is that the tools that really are wedded to sort of state of the science, that really deliver that with fidelity, that really have the strong scientific testing behind them, not just the clinical trial showing it's safe and effective for very meaningful clinical outcomes, but also looking at real-world evidence and implementability of these types of tools in healthcare settings. I think we will see that those rise to the top and that this other stuff shakes out. And then we'll get a landscape of things that really have active therapeutic ingredients are gonna be the ones that persist and scale because they will show the value that you're talking about. And I do think that we ask a lot of groups to make claims and say this works, like the data that you showed were so compelling, 100% you showed, right? Saying this clinically works and then 6% had data or something out of all of those folks. So I think that we will see that some of those just don't drive the effects they claim to have and they will fade away because people won't use them. I think also a lot of our oversight bodies, like the FDA regulatory process I described, that's also gonna help identify the ones that really have the strongest empirical support and have all that rigorous evidence behind them. Otherwise you don't get that designation. So that's a very useful resource to look at the array of products that have gotten that designation. But any other comments? Yeah, and I think I'll just piggyback on and relate it to the other question too of engagement. I think that comes down to the app being helpful and doing what people need it to do, right? And so I think so often people download it, plug it in, look at it, like, well, that wasn't really what I wanted. So you're onto the next thing. And so I think having a curated set and knowing where to turn to find those things that are gonna have those key elements, I think are also gonna help boost engagement with the right tools. We should probably break because time flew. This was wonderful. So let's give a hand to our panelists. Thank you.

digital health

substance use disorders

smoking cessation

wearables

mental health

digital therapeutics

public health

patient engagement

privacy concerns

tobacco use

health monitoring

treatment innovation