- Journal List
- BMJ Open
- v.14(5); 2024
- PMC11086474
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
BMJ Open. 2024; 14(5): e081673.
Published online 2024 May 7. doi:10.1136/bmjopen-2023-081673
PMCID: PMC11086474
PMID: 38719322
Protocol
Caroline A Figueroa,
1,2 Nancy J Pérez-Flores,3 Kathleen W Guan,1 and Colleen Stiles-Shields4
Author information Article notes Copyright and License information PMC Disclaimer
Associated Data
- Supplementary Materials
Abstract
Introduction
After COVID-19, a global mental health crisis affects young people, with one in five youth experiencing mental health problems worldwide. Delivering mental health interventions via mobile devices is a promising strategy to address the treatment gap. Mental health apps are effective for adolescent and young adult samples, but face challenges such as low real-world reach and under-representation of minoritised youth. To increase digital health uptake, including among minoritised youth, there is a need for diversity, equity and inclusion (DEI) considerations in the development and evaluation of mental health apps. How well DEI is integrated into youth mental health apps has not been comprehensively assessed. This scoping review aims to examine to what extent DEI considerations are integrated into the design and evaluation of youth mental health apps and report on youth, caregiver and other stakeholder involvement.
Methods and analysis
We will identify studies published in English from 2009 to 29 September 2023 on apps for mental health in youth. We will use PubMed, Global Health, APA PsycINFO, SCOPUS, CINAHL PLUS and the Cochrane Database and will report according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Scoping Review Extension guidelines. Papers eligible for inclusion must be peer-reviewed publications in English involving smartphone applications used by adolescents or young adults aged 10–25, with a focus on depression, anxiety or suicidal ideation. Two independent reviewers will review and extract articles using a template developed by the authors. We will analyse the data using narrative synthesis and descriptive statistics. This study will identify gaps in the literature and provide a roadmap for equitable and inclusive mental health apps for youth.
Ethics and dissemination
Ethics approval is not required. Findings will be disseminated through academic, industry, community networks and scientific publications.
Keywords: Adolescent, Health Equity, MENTAL HEALTH, eHealth
STRENGTHS AND LIMITATIONS OF THIS STUDY
This is the first comprehensive assessment of diversity, equity and inclusion (DEI) criteria in the development and evaluation of youth mental health apps.
We will study DEI criteria (access, content and appearance), according to a framework for evaluating mobile health apps, the representation of marginalised youth and the involvement of youth and other important stakeholders.
Our scoping review will conform to the rigorous methodology manual by the Joanna Briggs Institute and the Preferred Reporting Items for Systematic Reviews and Meta-analyses checklist for scoping reviews.
The framework of DEI criteria we use is not specific to adolescents and young adult populations.
This review will be limited to published research, potentially missing commercialised apps without research evidence.
Background
A pressing global mental health crisis, exacerbated by the COVID-19 pandemic, affects adolescents and young people, particularly those aged 10–24 years.1–4 Annually, approximately 20% of adolescents encounter a mental health condition, and on a global scale, suicide ranks as the fourth leading cause of death among individuals aged 15–19.5 Yet globally, less than 50% of children and young people with mental health problems receive treatment.6 7 Reasons for this treatment gap are barriers, including costs, mental health stigma, low mental health literacy, poor access to appropriate services and inadequate health system structures.8 Individuals from minoritised racial and ethnic backgrounds, those with lower socioeconomic status (SES), and/or those residing in marginalised communities use mental health services even less frequently, even though they are two to three times more likely to develop mental health problems.9 Marginalised youth also often encounter lower service quality than their white and higher SES counterparts.10 11 They face higher barriers than non-marginalised youth, such as concerns about cost, confidentiality, lack of convenience, cultural and language issues, and troubles navigating the system.12 The high burden of mental health problems in young people and the lack of access to appropriate services warrant novel and accessible ways of providing mental health treatment.
Delivering mental health interventions via mobile devices holds promise to address the treatment gap.13 14 Mental health apps offer several advantages, including scalability and feasibility among youth who are avid digital health adopters.15 Additionally, these apps can be personalised, advancing technological capabilities. Various methods, such as ecological momentary assessments/interventions and AI-based methods, such as machine learning, can be used to develop novel and promising personalised and real-time intervention strategies.16 For paediatric and young adult samples, reviews and meta-analyses show that mental health apps are a promising stand-alone self-management tool in mental health service delivery.17 18 Recently, the National Institute for Health and Care Excellence (NICE) guidelines recommended four digital technologies, including apps, for use in the UK’s National Health Service to help children and young people with mild to moderate symptoms of anxiety or low mood once they achieve regulatory approval.19 Thus, Digital Mental health (DMH) could serve as a low-cost, scalable, easily accessible and efficacious approach to delivering mental healthcare which will likely become more important in the future for reducing the treatment gap for youth.
However, several pressing issues impede the equitable use of mental health apps for young people. For instance, commercially available apps often lack evidence-based foundations,14 20 grapple with privacy concerns21 and are hampered by gender and racial biases.22 In addition, mental health apps exhibit low real-world reach and uptake,23 especially in low-resource settings18 and their (cost)-effectiveness and successful implementation is still unclear.18 23 Furthermore, there is a notable under-representation of marginalised youth, including racially minoritised youth(eg, black, Indigenous and people of colour), those with low SES, migration backgrounds and diverse gender identities, in their design, testing and use of DMH.24 For instance, a review on implementation factors in mental health apps for youth found that 71% of studies had recruited university students,23 and a review on preventive digital mental health (mostly internet based) found that demographic factors, such as SES and ethnicity, were often not reported.25 These studies likely point to a lack of inclusive and equitable representation among users.
Several researchers have argued for the need for diversity, equity and inclusion (DEI) considerations in the development and evaluation of mental health apps.26 27 This includes DEI consideration in the design and evaluation of the apps (eg, access, content and appearance),26 representation and cocreation with diverse youth and community partners, and intersectionality analyses to evaluate their effectiveness.27 These steps are suggested as a crucial strategy to increase the uptake and engagement of digital mental health solutions, particularly among minoritised youth and promote equitable access.28 Despite the field’s critique for lacking diversity and addressing health equity issues, comprehensive assessments of how well DEI considerations are integrated into the design and evaluation of mental health apps for young people are lacking.
Review question
Considering these limitations, this review aims to examine to what extent DEI considerations are incorporated in the design and evaluation of mental health apps for youth. Considering the gap in knowledge on this topic, a scoping review is suited to investigate this.
We focus on apps that target crucial and timely subjects, depression, anxiety and suicidal ideation. Depression and anxiety disorders are among the leading causes of illness and disability among adolescents, and a suicide is a leading cause of death in adolescents aged 15–29.29 30 Symptoms of depression, anxiety and suicidal ideation spiked and remained high after the COVID-19 pandemic in young people.1 2 We will apply the DEI framework developed by Ramos et al, adopted from the culturally informed design framework,26 specifically tailored to mental health apps.26 Ramos et al developed this framework as a tool to assess whether mental health apps consider factors such as access to digital tools, diverse representation and culturally adapted content. Using the tool, the authors conducted a systematic review of published mental health app frameworks, and found that, of the 44 frameworks evaluated, only 58% considered at least one DEI criterion. These factors are, however, crucial to develop digital interventions inclusive of individuals with various backgrounds and promote health equity.
Objectives
Specifically, we aim to examine (1) the prevalence of ethnic minoritised youth (racial/ethnic diversity), low-income youth (socioeconomic diversity) and gender-diverse youth in mobile health apps targeting common mental health disorders (ie, anxiety, depression and suicidal ideation), among children and young adults aged 10–25 years; (2) the proportion of DEI criteria considered by included studies in the design and testing of youth DMH and (3) reported methods of involving youth and other important stakeholders (such as parents) in the development of mental health apps based on participatory research (eg, shared and trustworthy leadership, agility and adaptiveness in research activities and engagement in the community, and use of effective and ongoing communication).
The results of this study will be crucial for exploring gaps in the literature and for identifying future directions in the field for a more inclusive youth digital mental health ecosystem.
Hypotheses
Most studies lack significant representation of racial/ethnic minoritised participants, participants with low SES and gender minorities.
Most studies fail to incorporate DEI criteria (eg, access, content and appearance) into the design and evaluation of mental health apps.
Most studies do not include information about the involvement of youth, and other important stakeholders including caregivers, in the development of mental health apps, or when they do, rely on one-time participation models (eg, complete a focus group or single usability testing session).
Methods
The proposed scoping review will be conducted in accordance with the Joanna Briggs Institute methodology for scoping reviews.31 We also use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Scoping Review Extension (PRISMA-ScR).32
Registration
This review title has been registered with Open Science Framework (https://osf.io/q3cax).33
Search strategy
Development of the search strategy
The search strategy was developed by the team with the consultation of two research librarians. We used an iterative approach, with two rounds of preliminary searches and refinement of the search strategy based on initial search results. The search strategy, including all identified keywords and index terms, was adapted for each included database and/or information source. The reference list of all included sources of evidence will be screened for additional studies.
The databases searched include Medline, Global Health, APA PsycINFO, SCOPUS, CINAHL PLUS and the Cochrane Database of randomised trials. All searches were conducted on 10 September 2023–29 September 2023. Full-search strings are shown in online supplemental appendix 1. Studies published in English will be included to ensure consistency and feasibility in the review process. Studies published from 1 January 2009 29 to September 2023 will be included. We chose to include studies from 2009, as there was a significant uptake in studies on digital mental health at this time.
Supplementary data
bmjopen-2023-081673supp001.pdf
Study/source of evidence selection
Following the search, all identified citations will be collated and uploaded into Covidence Software (www.covidence.org), a web-based systematic review platform, and duplicates will be removed. Following a pilot test, titles and abstracts will then be screened by two or more independent reviewers for assessment against the inclusion criteria for the review. The full text of selected citations will be assessed in detail against the inclusion criteria by two or more independent reviewers. Reasons for exclusion of sources of evidence in full text that do not meet the inclusion criteria will be recorded and reported in the scoping review. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion, or with an additional reviewer/s. The results of the search and the study inclusion process will be reported in full in the final scoping review and presented in a PRISMA-ScR flow diagram.32
Data extraction
Data will be extracted by two or more independent reviewers. We have developed (and piloted in October 2023 with five studies) a data extraction tool (see online supplemental file 1). The data extracted will include the country of the study, the type of intervention, the racial/ethnic composition of patients, sociodemographic background and gender composition. Additionally, we will extract whether studies assessed DEI criteria and whether they reported any youth, caregiver or other stakeholder involvement. To the DEI criteria by Ramos et al, we added cultural adaptation and privacy/security considerations. We will also include brief descriptions of the methods used to assess DEI criteria (if any) and how studies operationalised youth and stakeholder involvement. The draft data extraction tool will be modified and revised as necessary during the process of extracting data from each included evidence source. Modifications will be detailed in the scoping review. Because this study examines how studies report and take into account DEI criteria, we will not contact authors to ask for additional information. Any disagreements that arise between the reviewers will be resolved through discussion or with an additional reviewer/s.
Inclusion and exclusion criteria
As inclusion criteria, articles should:
Have a primary focus on depression, anxiety or suicidal ideation.
Be published in a peer-reviewed publication.
Be published in English.
Involve smartphone applications (with or without wearable sensor, website, etc).
Involve human subjects data.
Have clinical or non-clinical populations as the primary users of the technology developed and/or tested.
Study adolescents or young people (aged 10–25). We will include studies where adolescents are in this age range, or if the mean age of the participant sample is 25 years or less, and above 10 years of age.
Articles will be excluded if the following were met (1) The primary focus was not on depression, anxiety or suicidal ideation; (2) The study exclusively involved SMS/MMS or phone calls, or web-based interventions or wearables that did not involve a smartphone app and (3) The study’s participants were under the age of 10 or above the age of 25.
Types of sources
We will include pilot and feasibility studies, app development studies and randomised trials.
Measured variables and data extracted
Main outcomes of this study
Proportion of studies that provide information on the racial and ethnic composition, SES, and gender identity of the recruited participants.
Where available, the overall prevalence of racial/ethnic minoritised patients (between-study median and range), the median age, education, immigration status (of parents), annual income (of parents) and gender.
Proportion of studies that report at least one DEI criteria in the development and/or assessment of interventions. We will examine access, content and appearance.
Access: (1) internet connectivity, (2) data usage, (3) cost, (4) system requirements.
Content: (1) language, (2) literacy, (3) tailoring of content-environment, (3) tailoring of content-disability and special needs and (4) tailoring of content-identity.
Appearance: (1) diverse visual representation.
Proportion of studies that reported involving youth in the development of mental health apps based on participatory research (eg, shared and trustworthy leadership, agility and adaptiveness in research activities and engagement in the community and use of effective and ongoing communication).
Racial/ethnic minoritised status will be identified on a per country basis. If studies only reported the percentage of white participants, all other participants will be assumed to be ethnic minorities. If participants were categorised as belonging to an ‘other’ group, they will be assumed to belong to ethnic minority groups.
Data analysis and presentation
Our data extraction template inonline supplemental material shows the data we will extract and analyse. Using the template, we will calculate the percentage of included studies that report the racial/ethnic composition of patients, sociodemographic background and gender composition. If available, we will analyse the median percentage of racial and ethnic minority patients, the gender distribution and prevailing education level.
Additionally, we will report the percentage of studies that assessed at least one DEI criteria (access, content and appearance, see above, and the percentage of studies that reported any youth and caregiver involvement). For each study, we will report which of the DEI criteria was assessed, include brief descriptions of the methods used to assess DEI criteria (if any) and how studies operationalised youth involvement. We will present this in a narrative summary and show the main outcomes in tables (see our data extraction template) and/or charts, describing how the results relate to the objectives and questions.
Patient and public involvement
No patients or the public were involved in the review. Results of the study will be discussed with youth and other stakeholders, such as social workers, caregivers and psychologists in the Netherlands and the USA to define future research priorities.
Ethics and dissemination
This study involves neither human participants nor unpublished secondary data. As such, approval from a human research ethics committee is not required. Findings of the scoping review will be disseminated through professional networks, conference presentations and publication in a scientific journal. In addition, we will disseminate findings in our nonacademic networks, including youth organisations and mental health institutions.
This study will make a novel contribution by assessing DEI considerations in research on the development and evaluation of mental health apps for young people. As the digital mental health field is expected to continue to grow in the coming years with advances in technology and the rising demand for mental healthcare, an equitable transformation is crucial. This research will help provide a roadmap for equitable, inclusive and responsible digital mental health development for adolescents and young adults.
Supplementary Material
Reviewer comments:
Click here to view.(129K, pdf)
Author'smanuscript:
Click here to view.(983K, pdf)
Acknowledgments
We would like to thank Melissa Vetter and Dirk-Jan Ligtenbelt for their assistance with the search strategy.
Footnotes
X@Colleen_SS
Contributors: Design of the protocol: CAF, KWG, NJP-F and CS-S. Draft of the manuscript: CAF. Review and final approval of the manuscript: CAF, KWG, NJP-F and CS-S.
Funding: CAF and KWG are supported through the ‘High Tech for a Sustainable Future’ capacity building programme of the 4TU Federation in the Netherlands (grant number N/A).
Disclaimer: The funders had no role in the content of the manuscript.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review: Not commissioned; externally peer reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Ethics statements
Patient consent for publication
Not applicable.
References
1. Madigan S, Korczak DJ, Vaillancourt T, et al.. Comparison of Paediatric emergency Department visits for attempted suicide, self-harm, and suicidal Ideation before and during the COVID-19 pandemic: a systematic review and meta-analysis. Lancet Psychiatry2023;10:342–51:S2215-0366(23)00036-6. 10.1016/S2215-0366(23)00036-6 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
2. Thorisdottir IE, Agustsson G, Oskarsdottir SY, et al.. Effect of the COVID-19 pandemic on adolescent mental health and substance use up to March, 2022, in Iceland: a repeated, cross-sectional, population-based study. Lancet Child Adolesc Health2023;7:347–57:S2352-4642(23)00022-6. 10.1016/S2352-4642(23)00022-6 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
3. Kauhanen L, Wan Mohd Yunus WMA, Lempinen L, et al.. A systematic review of the mental health changes of children and young people before and during the COVID-19 pandemic. Eur Child Adolesc Psychiatry2023;32:995–1013. 10.1007/s00787-022-02060-0 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
4. Statistics Netherlands (CBS). Mental health has worsened among young people, 2022. Available: https://www.cbs.nl/en-gb/news/2022/22/mental-health-has-worsened-among-young-people
5. Lovero KL, Dos Santos PF, Come AX, et al.. Suicide in global mental health. Curr Psychiatry Rep2023;25:255–62. 10.1007/s11920-023-01423-x [PMC free article] [PubMed] [CrossRef] [Google Scholar]
6. High-income countries are failing to provide adequate mental health services for children, 2021. Available: https://www.bmj.com/company/newsroom/high-income-countries-are-failing-to-provide-adequate-mental-health-services-for-children/
7. Arzamarski C, Steiguer A, Omari F, et al.. Accessing child and adolescent mental health services in Low- and middle-income countries. 2021. Cham: Springer International Publishing, n.d.: 971–86. Available: 10.1007/978-3-030-57296-9_136 [CrossRef] [Google Scholar]
8. Rickwood DJ, Deane FP, Wilson CJ. When and how do young people seek professional help for mental health problemsMed J Aust2007;187:S35–9. 10.5694/j.1326-5377.2007.tb01334.x [PubMed] [CrossRef] [Google Scholar]
9. Reiss F. Socioeconomic inequalities and mental health problems in children and adolescents: a systematic review. Soc Sci Med2013;90:24–31:S0277-9536(13)00260-8. 10.1016/j.socscimed.2013.04.026 [PubMed] [CrossRef] [Google Scholar]
10. Vora AS, Grilo SA. The influence of identity on Multiracial emerging adults’ health and experiences seeking Healthcare in the United States: a qualitative study. J Racial Ethn Health Disparities2023;Published Online First:12 . 10.1007/s40615-023-01785-1 [PubMed] [CrossRef] [Google Scholar]
11. Flores G, Research COP. Technical report--racial and ethnic disparities in the health and health care of children. Pediatrics2010;125:e979–1020. 10.1542/peds.2010-0188 [PubMed] [CrossRef] [Google Scholar]
12. Robards F, Kang M, Usherwood T, et al.. How marginalized young people access, engage with, and navigate health-care systems in the Digital age. J Adolesc Health2018;62:365–81:S1054-139X(17)30861-3. 10.1016/j.jadohealth.2017.10.018 [PubMed] [CrossRef] [Google Scholar]
13. Figueroa CA, Aguilera A. The need for a mental health technology revolution in the COVID-19 pandemic. Front Psychiatry2020;11:523. 10.3389/fpsyt.2020.00523 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
14. Denecke K, Schmid N, Nüssli S. Implementation of cognitive behavioral therapy in E-mental health Apps. J Med Internet Res2022;24:e27791. 10.2196/27791 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
15. Vogels EA, Gelles-Watnick R, Massarat N. 2022. Teens, social media and technology 2022.
16. Wang L, Miller LC. Just-in-the-moment adaptive interventions (JITAI): A meta-Analytical review. Health Commun2020;35:1531–44. 10.1080/10410236.2019.1652388 [PubMed] [CrossRef] [Google Scholar]
17. Leech T, Dorstyn D, Taylor A, et al.. Mental health Apps for adolescents and young adults: A systematic review of randomised controlled trials. Children and Youth Services Review2021;127:106073. 10.1016/j.childyouth.2021.106073 [CrossRef] [Google Scholar]
18. Lehtimaki S, Martic J, Wahl B, et al.. Evidence on Digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health2021;8:e25847. 10.2196/25847 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
19. Digital mental health Tech for children and young people recommended by NICE in first rapid Healthtech guidance | news | news | NICE.
20. Bry LJ, Chou T, Miguel E, et al.. Consumer Smartphone Apps marketed for child and adolescent anxiety: A systematic review and content analysis. Behav Ther2018;49:249–61:S0005-7894(17)30084-9. 10.1016/j.beth.2017.07.008 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
21. Domhardt M, Messner E-M, Eder A-S, et al.. Mobile-based interventions for common mental disorders in youth: a systematic evaluation of pediatric health Apps. Child Adolesc Psychiatry Ment Health2021;15:49. 10.1186/s13034-021-00401-6 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
22. Dredge S. Diversity in children’s Apps: ‘gender Depictions tend to be very stereotyped. The Guardian2015. Available: https://www.theguardian.com/technology/2015/jun/08/diversity-childrens-apps-gender-depictions [Google Scholar]
23. Bear HA, Ayala Nunes L, DeJesus J, et al.. Determination of markers of successful implementation of mental health Apps for young people. J Med Internet Res2022;24:e40347. 10.2196/40347 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
24. Lattie EG, Stiles-Shields C, Graham AK. An overview of and recommendations for more accessible Digital mental health services. Nat Rev Psychol2022;1:87–100. 10.1038/s44159-021-00003-1 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
25. Bergin AD, Vallejos EP, Davies EB, et al.. Preventive Digital mental health interventions for children and young people: a review of the design and reporting of research. NPJ Digit Med2020;3:133. 10.1038/s41746-020-00339-7 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
26. Ramos G, Ponting C, Labao JP, et al.. Considerations of diversity, equity, and inclusion in mental health Apps: A Scoping review of evaluation frameworks. Behav Res Ther2021;147:103990:S0005-7967(21)00189-3. 10.1016/j.brat.2021.103990 [PubMed] [CrossRef] [Google Scholar]
27. Figueroa CA, Luo T, Aguilera A, et al.. The need for feminist Intersectionality in Digital health. Lancet Digit Health2021;3:e526–33:S2589-7500(21)00118-7. 10.1016/S2589-7500(21)00118-7 [PubMed] [CrossRef] [Google Scholar]
28. Stiles-Shields C, Ramos G, Ortega A, et al.. Increasing Digital mental health reach and uptake via youth partnerships. Npj Ment Health Res2023;2:9. 10.1038/s44184-023-00030-1 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
29. World Health Organization. Suicide worldwide in 2019: global health estimates, Available: https://apps.who.int/iris/bitstream/handle/10665/341728/9789240026643-eng.pdf
30. World Health Organization. Mental health of adolescents, Available: https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health
31. The JBI Scoping Review Network Resources, Available: https://jbi.global/scoping-review-network/resources
32. Tricco AC, Lillie E, Zarin W, et al.. PRISMA extension for Scoping reviews (PRISMA-SCR): checklist and explanation. Ann Intern Med2018;169:467–73. 10.7326/M18-0850 [PubMed] [CrossRef] [Google Scholar]
33. Open Science Framework, Available: https://osf.io/q3cax
Articles from BMJ Open are provided here courtesy of BMJ Publishing Group
