User login
Mental health apps: What to tell patients
Have your patients asked you about smartphone apps? If they haven’t yet, they may soon, as interest in apps for mental health continues to expand. There are now >10,000 mental health–related smartphone apps.1 The rapid rise of these apps is partly due to their potential to transform a patient’s smartphone into a monitoring and therapeutic platform, capable of capturing mental health symptoms in real time and delivering on-the-go therapy. Setting aside questions about the potential of mobile health, 2 urgent questions remain for the busy psychiatrist in clinical practice: What is the current evidence base for mental health apps, and what should you tell your patients about them?
For most apps, evidence of efficacy is limited
While the evidence base for mental health smartphone apps continues to expand, for many of these apps, there is no evidence of effectiveness. The growing consensus is that most commercially available apps are not evidence-based and some are even dangerous. For example, researchers who examined >700 mindfulness apps on the iTunes and Google Play stores found that only 4% provided acceptable mindfulness training and education.2 Another study of 58 apps that claimed to offer sobriety assessments found that none had ever been formally evaluated.3 Evidence-based reviews of suicide prevention apps have identified potentially harmful apps,4 and studies evaluating apps for bipolar disorder5 and depression6 have yielded similar results—few have any evidence supporting their use, and some offer dangerous and harmful advice. For example, researchers found that one app for bipolar disorder advised patients who are experiencing a manic episode to drink alcohol.5 Currently, the vast majority of commercially available apps are not appropriate for clinical care. This finding is not unique to mental health; similar findings have been reported for apps for cancer.7 The bottom line is that the apps that your patients are finding, and perhaps already using, may not be useful or effective.
However, early studies have demonstrated efficacy of some apps for several conditions, including schizophrenia,8 depression,9 anxiety disorders,10 and suicidal ideation.11 Although many of the apps evaluated in these studies are not available to the public, or still require large-scale assessment before they are ready for mainstream clinical care, this research demonstrates that mental health apps can help improve treatment outcomes. As this research develops, a wave of evidence-based and effective mental health apps may be available in the near future.
Although it is unknown how many patients are presently using mental health apps, there is strong anecdotal evidence that an increasing number of patients who use these apps and other forms of digital technology are finding some benefits. In many cases, patients may actually be ahead of the research. For example, one study that conducted an online survey of patients with schizophrenia noted that some patients are using their smartphones to play music to help block auditory hallucinations.12
Why online reviews are of limited use
As this evidence continues to mature, and with an ever-growing number of mental health apps available on commercial marketplaces, busy psychiatrists need to navigate this complex space. Even psychiatrists who decide to not use apps as part of care still need to be knowledgeable about them, because patients are likely to ask about the benefits of using apps, and they will expect an informed response. How would you reply if your patient asked you about a new mood-tracking app he or she recently heard about? On what would you base your recommendation and opinion?
Reading online app reviews for guidance is not a good solution. A recent study found little relationship between the star ratings of health apps and the quality of those apps,13 which suggests that a 5-star rating on the app store is of limited use.
Unlike medications whose ingredients do not change over time, or manualized psychotherapies that use specific protocols, mental health apps are dynamic and constantly changing.14 Think of how often the apps on your smartphone update. Thus, the version of a mental health app that your patient downloads today may be very different from the version that received a favorable user review last month. And just as there is no single medication or therapy that is ideal for every patient, neither is there a single “best” app for all patients with the same disorder. Picking an app is a personal decision that cannot be made based on a single score or numeric rating. Furthermore, the validity of app rating systems is unclear. One study found a wide variation in the interrater reliability of measures used to evaluate apps from sources that included PsyberGuide, the Anxiety and Depression Association of America, and the research literature. Quality measures such as effectiveness, ease of use, and performance had relatively poor interrater reliability.15 This means that, for example, an app that one patient finds “easy to use” may be difficult to use for another. Thus, providing patients with suggestions based on an app’s ratings may result in providing information that sounds useful, but often is misleading.
A model for evaluating apps
One possible solution is a risk-based and personalized assessment approach to evaluating mental health apps. Although it does not offer scoring or recommendations of specific apps, the American Psychiatric Association (APA) App Evaluation Model (Figure) provides a framework to guide discussion and informed decision-making about apps. (The authors of this article helped create this model, but receive no compensation for that volunteer work.) The pyramid shape reflects the hierarchical nature of the model. To begin the process, start at the base of the pyramid and work upward.
Ground. First, consider the context of the app by determining basic facts, such as who made it, how much it costs, and its technology requirements. This ground layer establishes the credibility of the app’s creator by questioning his or her reputation, ability to update the app, and funding sources. Understanding the app’s business model also will help you determine whether the app will stand the test of time: Will it continue to exist next month or next year, or will a lack of reliable funding lead the vendor to abandon it?
Risk. The next layer assesses the risk, privacy, and security features of the app. Many mental health apps actively aim to avoid falling under the jurisdiction of U.S. federal health care privacy rules, such as the Health Insurance Portability and Accountability Act of 1996, so there is no guarantee that sensitive data supplied to an app will be protected. The true cost of a “free” app often is your patient’s personal mental health information, which the app’s developer may accumulate and sell for profit. Thus, it is wise to check the privacy policy to learn where your patient’s data goes. Furthermore, patients and psychiatrists must be vigilant that malware-infected apps can be uploaded to the app store, which can further compromise privacy.16 You may be surprised to learn that many apps lack a privacy policy, which means there are no protections for personal information or safeguards against the misuse of mental health data.17 Checking that an app at least promises to digitally protect mental health data through encryption and secure storage also is a good step.
The goal of considering these factors is not to create a score, but rather to be aware of them and consider them in the context of the specific app, patient, and clinical situation. Doing so helps determine whether the app meets the appropriate risk, privacy, and security standards for your patient.
Evidence. The next layer of the evaluation framework is evidence. The goal is to seek an app with clinical evidence of effectiveness. Simply put, if a patient is going to use an app, he should use one that works. An app without formal evidence may be effective, but it is important to make sure the patient is aware that these claims have not been verified. Many apps claim that they offer cognitive-behavioral therapy or mindfulness therapy, but few deliver on such claims.18 It is wise to try an app before recommending it to a patient to ensure that it does what it claims it does, and does not offer dangerous or harmful recommendations.
Ease of use. Across all health apps, there is growing recognition that most downloaded apps are never used. Patient engagement with mental health apps appears to rapidly decline over the first week of use.19 There also is emerging evidence that many apps are not user-friendly. A recent study of several common mood-tracking apps found that patients with depression had difficulty entering and accessing their data.20 Because many psychiatric disorders are chronic or last at least several months, it is especially important to consider how engaging and usable the app will be for your patient. Usability varies from patient to patient, so it is best to check directly with your patient regarding his comfort with apps and mobile technology. Offering check-ins and support to help patients keep on track with apps may be critical for successful outcomes.
Interoperability. The final layer of the model is data sharing and interoperability. It is important to determine if the data collected or generated by the app are available to you, the patient, the treatment team, and others involved in the patient’s care. As mental health treatment moves toward integrated care, apps that fragment care (by not sharing information) impede care. Check if the app can share data with an electronic medical record, or if there is a plan to review and act on data from the app as part of your patient’s treatment plan.
More information about the APA App Evaluation Model, including additional factors to consider within each layer, is available from the APA for free at https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/app-evaluation-model. For a sample of factors to consider when evaluating a mental health app, see the Table.
A reasonable strategy
Although the APA App Evaluation Model does not endorse any particular app, it can help guide more informed decision-making. As the evidence on mental health apps continues to evolve, it will become easier to make definitive statements on what constitutes a useful app. For now, the best strategy when discussing mental health apps with patients is to combine the use of this model with your clinical judgment.
1. Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry. 2017;74(5):437-438.
2. Mani M, Kavanagh DJ, Hides L, et al. Review and evaluation of mindfulness-based iPhone apps. JMIR Mhealth Uhealth. 2015;3(3):e82. doi: 10.2196/mhealth.4328.
3. Wilson H, Stoyanov SR, Gandabhai S, et al. The quality and accuracy of mobile apps to prevent driving after drinking alcohol. JMIR Mhealth Uhealth. 2016;4(3):e98. doi: 10.2196/mhealth.5961.
4. Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PLoS One. 2016;11(4):e0152285. doi: 10.1371/journal.pone.0152285.
5. Nicholas J, Larsen ME, Proudfoot J, et al. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res. 2015;17(8):e198. doi: 10.2196/jmir.4581.
6. Shen N, Levitan MJ, Johnson A, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth. 2015;3(1):e16. doi: 10.2196/mhealth.3713.
7. Davis SW, Oakley-Girvan I. Achieving value in mobile health applications for cancer survivors. J Cancer Surviv. 2017;11(4):498-504.
8. Ben-Zeev D, Brenner CJ, Begale M, et al. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr Bull. 2014;40(6):1244-1253.
9. Mohr DC, Tomasino KN, Lattie EG, et al. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res. 2017;19(1):e10. doi: 10.2196/jmir.6645.
10. Tighe J, Shand F, Ridani R, et al. Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: a pilot randomised controlled trial. BMJ Open. 2017;7(1):e013518. doi: 10.1136/bmjopen-2016-013518.
11. Firth J, Torous J, Nicholas J, et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord. 2017;218:15-22.
12. Gay K, Torous J, Joseph A, et al. Digital technology use among individuals with schizophrenia: results of an online survey. JMIR Mental Health. 2016;3(2):e15. doi: 10.2196/mental.5379.
13. Singh K, Drouin K, Newmark LP, et al. Many mobile health apps target high-need, high-cost populations, but gaps remain. Health Aff (Millwood). 2016;35(12):2310-2318.
14. Larsen ME, Nicholas J, Christensen H. Quantifying app store dynamics: longitudinal tracking of mental health apps. JMIR Mhealth Uhealth. 2016;4(3):e96. doi: 10.2196/mhealth.6020.
15. Powell AC, Torous J, Chan S, et al. Interrater reliability of mHealth app rating measures: analysis of top depression and smoking cessation apps. JMIR Mhealth Uhealth. 2016;4(1):e15. doi: 10.2196/mhealth.5176.
16. Ducklin P. Apple’s XcodeGhost malware still in the machine…. https://nakedsecurity.sophos.com/2015/11/09/apples-xcodeghost-malware-still-in-the-machine. Published November 9, 2015. Accessed May 11, 2017.
17. Rosenfeld L, Torous J, Vahia IV. Data security and privacy in apps for dementia: an analysis of existing privacy policies. Am J Geriatr Psychiatry. 2017;25(8):873-877.
18. Torous J, Levin ME, Ahern DK, et al. Cognitive behavioral mobile applications: clinical studies, marketplace overview, and research agenda. Cogn Behav Pract. 2017;24(2):215-225.
19. Owen JE, Jaworski BK, Kuhn E, et al. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health. 2015;2(1):e7. doi: 10.2196/mental.3935.
20. Sarkar U, Gourley GI, Lyles CR, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med. 2016;31(12):1417-1426.
Have your patients asked you about smartphone apps? If they haven’t yet, they may soon, as interest in apps for mental health continues to expand. There are now >10,000 mental health–related smartphone apps.1 The rapid rise of these apps is partly due to their potential to transform a patient’s smartphone into a monitoring and therapeutic platform, capable of capturing mental health symptoms in real time and delivering on-the-go therapy. Setting aside questions about the potential of mobile health, 2 urgent questions remain for the busy psychiatrist in clinical practice: What is the current evidence base for mental health apps, and what should you tell your patients about them?
For most apps, evidence of efficacy is limited
While the evidence base for mental health smartphone apps continues to expand, for many of these apps, there is no evidence of effectiveness. The growing consensus is that most commercially available apps are not evidence-based and some are even dangerous. For example, researchers who examined >700 mindfulness apps on the iTunes and Google Play stores found that only 4% provided acceptable mindfulness training and education.2 Another study of 58 apps that claimed to offer sobriety assessments found that none had ever been formally evaluated.3 Evidence-based reviews of suicide prevention apps have identified potentially harmful apps,4 and studies evaluating apps for bipolar disorder5 and depression6 have yielded similar results—few have any evidence supporting their use, and some offer dangerous and harmful advice. For example, researchers found that one app for bipolar disorder advised patients who are experiencing a manic episode to drink alcohol.5 Currently, the vast majority of commercially available apps are not appropriate for clinical care. This finding is not unique to mental health; similar findings have been reported for apps for cancer.7 The bottom line is that the apps that your patients are finding, and perhaps already using, may not be useful or effective.
However, early studies have demonstrated efficacy of some apps for several conditions, including schizophrenia,8 depression,9 anxiety disorders,10 and suicidal ideation.11 Although many of the apps evaluated in these studies are not available to the public, or still require large-scale assessment before they are ready for mainstream clinical care, this research demonstrates that mental health apps can help improve treatment outcomes. As this research develops, a wave of evidence-based and effective mental health apps may be available in the near future.
Although it is unknown how many patients are presently using mental health apps, there is strong anecdotal evidence that an increasing number of patients who use these apps and other forms of digital technology are finding some benefits. In many cases, patients may actually be ahead of the research. For example, one study that conducted an online survey of patients with schizophrenia noted that some patients are using their smartphones to play music to help block auditory hallucinations.12
Why online reviews are of limited use
As this evidence continues to mature, and with an ever-growing number of mental health apps available on commercial marketplaces, busy psychiatrists need to navigate this complex space. Even psychiatrists who decide to not use apps as part of care still need to be knowledgeable about them, because patients are likely to ask about the benefits of using apps, and they will expect an informed response. How would you reply if your patient asked you about a new mood-tracking app he or she recently heard about? On what would you base your recommendation and opinion?
Reading online app reviews for guidance is not a good solution. A recent study found little relationship between the star ratings of health apps and the quality of those apps,13 which suggests that a 5-star rating on the app store is of limited use.
Unlike medications whose ingredients do not change over time, or manualized psychotherapies that use specific protocols, mental health apps are dynamic and constantly changing.14 Think of how often the apps on your smartphone update. Thus, the version of a mental health app that your patient downloads today may be very different from the version that received a favorable user review last month. And just as there is no single medication or therapy that is ideal for every patient, neither is there a single “best” app for all patients with the same disorder. Picking an app is a personal decision that cannot be made based on a single score or numeric rating. Furthermore, the validity of app rating systems is unclear. One study found a wide variation in the interrater reliability of measures used to evaluate apps from sources that included PsyberGuide, the Anxiety and Depression Association of America, and the research literature. Quality measures such as effectiveness, ease of use, and performance had relatively poor interrater reliability.15 This means that, for example, an app that one patient finds “easy to use” may be difficult to use for another. Thus, providing patients with suggestions based on an app’s ratings may result in providing information that sounds useful, but often is misleading.
A model for evaluating apps
One possible solution is a risk-based and personalized assessment approach to evaluating mental health apps. Although it does not offer scoring or recommendations of specific apps, the American Psychiatric Association (APA) App Evaluation Model (Figure) provides a framework to guide discussion and informed decision-making about apps. (The authors of this article helped create this model, but receive no compensation for that volunteer work.) The pyramid shape reflects the hierarchical nature of the model. To begin the process, start at the base of the pyramid and work upward.
Ground. First, consider the context of the app by determining basic facts, such as who made it, how much it costs, and its technology requirements. This ground layer establishes the credibility of the app’s creator by questioning his or her reputation, ability to update the app, and funding sources. Understanding the app’s business model also will help you determine whether the app will stand the test of time: Will it continue to exist next month or next year, or will a lack of reliable funding lead the vendor to abandon it?
Risk. The next layer assesses the risk, privacy, and security features of the app. Many mental health apps actively aim to avoid falling under the jurisdiction of U.S. federal health care privacy rules, such as the Health Insurance Portability and Accountability Act of 1996, so there is no guarantee that sensitive data supplied to an app will be protected. The true cost of a “free” app often is your patient’s personal mental health information, which the app’s developer may accumulate and sell for profit. Thus, it is wise to check the privacy policy to learn where your patient’s data goes. Furthermore, patients and psychiatrists must be vigilant that malware-infected apps can be uploaded to the app store, which can further compromise privacy.16 You may be surprised to learn that many apps lack a privacy policy, which means there are no protections for personal information or safeguards against the misuse of mental health data.17 Checking that an app at least promises to digitally protect mental health data through encryption and secure storage also is a good step.
The goal of considering these factors is not to create a score, but rather to be aware of them and consider them in the context of the specific app, patient, and clinical situation. Doing so helps determine whether the app meets the appropriate risk, privacy, and security standards for your patient.
Evidence. The next layer of the evaluation framework is evidence. The goal is to seek an app with clinical evidence of effectiveness. Simply put, if a patient is going to use an app, he should use one that works. An app without formal evidence may be effective, but it is important to make sure the patient is aware that these claims have not been verified. Many apps claim that they offer cognitive-behavioral therapy or mindfulness therapy, but few deliver on such claims.18 It is wise to try an app before recommending it to a patient to ensure that it does what it claims it does, and does not offer dangerous or harmful recommendations.
Ease of use. Across all health apps, there is growing recognition that most downloaded apps are never used. Patient engagement with mental health apps appears to rapidly decline over the first week of use.19 There also is emerging evidence that many apps are not user-friendly. A recent study of several common mood-tracking apps found that patients with depression had difficulty entering and accessing their data.20 Because many psychiatric disorders are chronic or last at least several months, it is especially important to consider how engaging and usable the app will be for your patient. Usability varies from patient to patient, so it is best to check directly with your patient regarding his comfort with apps and mobile technology. Offering check-ins and support to help patients keep on track with apps may be critical for successful outcomes.
Interoperability. The final layer of the model is data sharing and interoperability. It is important to determine if the data collected or generated by the app are available to you, the patient, the treatment team, and others involved in the patient’s care. As mental health treatment moves toward integrated care, apps that fragment care (by not sharing information) impede care. Check if the app can share data with an electronic medical record, or if there is a plan to review and act on data from the app as part of your patient’s treatment plan.
More information about the APA App Evaluation Model, including additional factors to consider within each layer, is available from the APA for free at https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/app-evaluation-model. For a sample of factors to consider when evaluating a mental health app, see the Table.
A reasonable strategy
Although the APA App Evaluation Model does not endorse any particular app, it can help guide more informed decision-making. As the evidence on mental health apps continues to evolve, it will become easier to make definitive statements on what constitutes a useful app. For now, the best strategy when discussing mental health apps with patients is to combine the use of this model with your clinical judgment.
Have your patients asked you about smartphone apps? If they haven’t yet, they may soon, as interest in apps for mental health continues to expand. There are now >10,000 mental health–related smartphone apps.1 The rapid rise of these apps is partly due to their potential to transform a patient’s smartphone into a monitoring and therapeutic platform, capable of capturing mental health symptoms in real time and delivering on-the-go therapy. Setting aside questions about the potential of mobile health, 2 urgent questions remain for the busy psychiatrist in clinical practice: What is the current evidence base for mental health apps, and what should you tell your patients about them?
For most apps, evidence of efficacy is limited
While the evidence base for mental health smartphone apps continues to expand, for many of these apps, there is no evidence of effectiveness. The growing consensus is that most commercially available apps are not evidence-based and some are even dangerous. For example, researchers who examined >700 mindfulness apps on the iTunes and Google Play stores found that only 4% provided acceptable mindfulness training and education.2 Another study of 58 apps that claimed to offer sobriety assessments found that none had ever been formally evaluated.3 Evidence-based reviews of suicide prevention apps have identified potentially harmful apps,4 and studies evaluating apps for bipolar disorder5 and depression6 have yielded similar results—few have any evidence supporting their use, and some offer dangerous and harmful advice. For example, researchers found that one app for bipolar disorder advised patients who are experiencing a manic episode to drink alcohol.5 Currently, the vast majority of commercially available apps are not appropriate for clinical care. This finding is not unique to mental health; similar findings have been reported for apps for cancer.7 The bottom line is that the apps that your patients are finding, and perhaps already using, may not be useful or effective.
However, early studies have demonstrated efficacy of some apps for several conditions, including schizophrenia,8 depression,9 anxiety disorders,10 and suicidal ideation.11 Although many of the apps evaluated in these studies are not available to the public, or still require large-scale assessment before they are ready for mainstream clinical care, this research demonstrates that mental health apps can help improve treatment outcomes. As this research develops, a wave of evidence-based and effective mental health apps may be available in the near future.
Although it is unknown how many patients are presently using mental health apps, there is strong anecdotal evidence that an increasing number of patients who use these apps and other forms of digital technology are finding some benefits. In many cases, patients may actually be ahead of the research. For example, one study that conducted an online survey of patients with schizophrenia noted that some patients are using their smartphones to play music to help block auditory hallucinations.12
Why online reviews are of limited use
As this evidence continues to mature, and with an ever-growing number of mental health apps available on commercial marketplaces, busy psychiatrists need to navigate this complex space. Even psychiatrists who decide to not use apps as part of care still need to be knowledgeable about them, because patients are likely to ask about the benefits of using apps, and they will expect an informed response. How would you reply if your patient asked you about a new mood-tracking app he or she recently heard about? On what would you base your recommendation and opinion?
Reading online app reviews for guidance is not a good solution. A recent study found little relationship between the star ratings of health apps and the quality of those apps,13 which suggests that a 5-star rating on the app store is of limited use.
Unlike medications whose ingredients do not change over time, or manualized psychotherapies that use specific protocols, mental health apps are dynamic and constantly changing.14 Think of how often the apps on your smartphone update. Thus, the version of a mental health app that your patient downloads today may be very different from the version that received a favorable user review last month. And just as there is no single medication or therapy that is ideal for every patient, neither is there a single “best” app for all patients with the same disorder. Picking an app is a personal decision that cannot be made based on a single score or numeric rating. Furthermore, the validity of app rating systems is unclear. One study found a wide variation in the interrater reliability of measures used to evaluate apps from sources that included PsyberGuide, the Anxiety and Depression Association of America, and the research literature. Quality measures such as effectiveness, ease of use, and performance had relatively poor interrater reliability.15 This means that, for example, an app that one patient finds “easy to use” may be difficult to use for another. Thus, providing patients with suggestions based on an app’s ratings may result in providing information that sounds useful, but often is misleading.
A model for evaluating apps
One possible solution is a risk-based and personalized assessment approach to evaluating mental health apps. Although it does not offer scoring or recommendations of specific apps, the American Psychiatric Association (APA) App Evaluation Model (Figure) provides a framework to guide discussion and informed decision-making about apps. (The authors of this article helped create this model, but receive no compensation for that volunteer work.) The pyramid shape reflects the hierarchical nature of the model. To begin the process, start at the base of the pyramid and work upward.
Ground. First, consider the context of the app by determining basic facts, such as who made it, how much it costs, and its technology requirements. This ground layer establishes the credibility of the app’s creator by questioning his or her reputation, ability to update the app, and funding sources. Understanding the app’s business model also will help you determine whether the app will stand the test of time: Will it continue to exist next month or next year, or will a lack of reliable funding lead the vendor to abandon it?
Risk. The next layer assesses the risk, privacy, and security features of the app. Many mental health apps actively aim to avoid falling under the jurisdiction of U.S. federal health care privacy rules, such as the Health Insurance Portability and Accountability Act of 1996, so there is no guarantee that sensitive data supplied to an app will be protected. The true cost of a “free” app often is your patient’s personal mental health information, which the app’s developer may accumulate and sell for profit. Thus, it is wise to check the privacy policy to learn where your patient’s data goes. Furthermore, patients and psychiatrists must be vigilant that malware-infected apps can be uploaded to the app store, which can further compromise privacy.16 You may be surprised to learn that many apps lack a privacy policy, which means there are no protections for personal information or safeguards against the misuse of mental health data.17 Checking that an app at least promises to digitally protect mental health data through encryption and secure storage also is a good step.
The goal of considering these factors is not to create a score, but rather to be aware of them and consider them in the context of the specific app, patient, and clinical situation. Doing so helps determine whether the app meets the appropriate risk, privacy, and security standards for your patient.
Evidence. The next layer of the evaluation framework is evidence. The goal is to seek an app with clinical evidence of effectiveness. Simply put, if a patient is going to use an app, he should use one that works. An app without formal evidence may be effective, but it is important to make sure the patient is aware that these claims have not been verified. Many apps claim that they offer cognitive-behavioral therapy or mindfulness therapy, but few deliver on such claims.18 It is wise to try an app before recommending it to a patient to ensure that it does what it claims it does, and does not offer dangerous or harmful recommendations.
Ease of use. Across all health apps, there is growing recognition that most downloaded apps are never used. Patient engagement with mental health apps appears to rapidly decline over the first week of use.19 There also is emerging evidence that many apps are not user-friendly. A recent study of several common mood-tracking apps found that patients with depression had difficulty entering and accessing their data.20 Because many psychiatric disorders are chronic or last at least several months, it is especially important to consider how engaging and usable the app will be for your patient. Usability varies from patient to patient, so it is best to check directly with your patient regarding his comfort with apps and mobile technology. Offering check-ins and support to help patients keep on track with apps may be critical for successful outcomes.
Interoperability. The final layer of the model is data sharing and interoperability. It is important to determine if the data collected or generated by the app are available to you, the patient, the treatment team, and others involved in the patient’s care. As mental health treatment moves toward integrated care, apps that fragment care (by not sharing information) impede care. Check if the app can share data with an electronic medical record, or if there is a plan to review and act on data from the app as part of your patient’s treatment plan.
More information about the APA App Evaluation Model, including additional factors to consider within each layer, is available from the APA for free at https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/app-evaluation-model. For a sample of factors to consider when evaluating a mental health app, see the Table.
A reasonable strategy
Although the APA App Evaluation Model does not endorse any particular app, it can help guide more informed decision-making. As the evidence on mental health apps continues to evolve, it will become easier to make definitive statements on what constitutes a useful app. For now, the best strategy when discussing mental health apps with patients is to combine the use of this model with your clinical judgment.
1. Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry. 2017;74(5):437-438.
2. Mani M, Kavanagh DJ, Hides L, et al. Review and evaluation of mindfulness-based iPhone apps. JMIR Mhealth Uhealth. 2015;3(3):e82. doi: 10.2196/mhealth.4328.
3. Wilson H, Stoyanov SR, Gandabhai S, et al. The quality and accuracy of mobile apps to prevent driving after drinking alcohol. JMIR Mhealth Uhealth. 2016;4(3):e98. doi: 10.2196/mhealth.5961.
4. Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PLoS One. 2016;11(4):e0152285. doi: 10.1371/journal.pone.0152285.
5. Nicholas J, Larsen ME, Proudfoot J, et al. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res. 2015;17(8):e198. doi: 10.2196/jmir.4581.
6. Shen N, Levitan MJ, Johnson A, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth. 2015;3(1):e16. doi: 10.2196/mhealth.3713.
7. Davis SW, Oakley-Girvan I. Achieving value in mobile health applications for cancer survivors. J Cancer Surviv. 2017;11(4):498-504.
8. Ben-Zeev D, Brenner CJ, Begale M, et al. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr Bull. 2014;40(6):1244-1253.
9. Mohr DC, Tomasino KN, Lattie EG, et al. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res. 2017;19(1):e10. doi: 10.2196/jmir.6645.
10. Tighe J, Shand F, Ridani R, et al. Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: a pilot randomised controlled trial. BMJ Open. 2017;7(1):e013518. doi: 10.1136/bmjopen-2016-013518.
11. Firth J, Torous J, Nicholas J, et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord. 2017;218:15-22.
12. Gay K, Torous J, Joseph A, et al. Digital technology use among individuals with schizophrenia: results of an online survey. JMIR Mental Health. 2016;3(2):e15. doi: 10.2196/mental.5379.
13. Singh K, Drouin K, Newmark LP, et al. Many mobile health apps target high-need, high-cost populations, but gaps remain. Health Aff (Millwood). 2016;35(12):2310-2318.
14. Larsen ME, Nicholas J, Christensen H. Quantifying app store dynamics: longitudinal tracking of mental health apps. JMIR Mhealth Uhealth. 2016;4(3):e96. doi: 10.2196/mhealth.6020.
15. Powell AC, Torous J, Chan S, et al. Interrater reliability of mHealth app rating measures: analysis of top depression and smoking cessation apps. JMIR Mhealth Uhealth. 2016;4(1):e15. doi: 10.2196/mhealth.5176.
16. Ducklin P. Apple’s XcodeGhost malware still in the machine…. https://nakedsecurity.sophos.com/2015/11/09/apples-xcodeghost-malware-still-in-the-machine. Published November 9, 2015. Accessed May 11, 2017.
17. Rosenfeld L, Torous J, Vahia IV. Data security and privacy in apps for dementia: an analysis of existing privacy policies. Am J Geriatr Psychiatry. 2017;25(8):873-877.
18. Torous J, Levin ME, Ahern DK, et al. Cognitive behavioral mobile applications: clinical studies, marketplace overview, and research agenda. Cogn Behav Pract. 2017;24(2):215-225.
19. Owen JE, Jaworski BK, Kuhn E, et al. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health. 2015;2(1):e7. doi: 10.2196/mental.3935.
20. Sarkar U, Gourley GI, Lyles CR, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med. 2016;31(12):1417-1426.
1. Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry. 2017;74(5):437-438.
2. Mani M, Kavanagh DJ, Hides L, et al. Review and evaluation of mindfulness-based iPhone apps. JMIR Mhealth Uhealth. 2015;3(3):e82. doi: 10.2196/mhealth.4328.
3. Wilson H, Stoyanov SR, Gandabhai S, et al. The quality and accuracy of mobile apps to prevent driving after drinking alcohol. JMIR Mhealth Uhealth. 2016;4(3):e98. doi: 10.2196/mhealth.5961.
4. Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PLoS One. 2016;11(4):e0152285. doi: 10.1371/journal.pone.0152285.
5. Nicholas J, Larsen ME, Proudfoot J, et al. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res. 2015;17(8):e198. doi: 10.2196/jmir.4581.
6. Shen N, Levitan MJ, Johnson A, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth. 2015;3(1):e16. doi: 10.2196/mhealth.3713.
7. Davis SW, Oakley-Girvan I. Achieving value in mobile health applications for cancer survivors. J Cancer Surviv. 2017;11(4):498-504.
8. Ben-Zeev D, Brenner CJ, Begale M, et al. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr Bull. 2014;40(6):1244-1253.
9. Mohr DC, Tomasino KN, Lattie EG, et al. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res. 2017;19(1):e10. doi: 10.2196/jmir.6645.
10. Tighe J, Shand F, Ridani R, et al. Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: a pilot randomised controlled trial. BMJ Open. 2017;7(1):e013518. doi: 10.1136/bmjopen-2016-013518.
11. Firth J, Torous J, Nicholas J, et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord. 2017;218:15-22.
12. Gay K, Torous J, Joseph A, et al. Digital technology use among individuals with schizophrenia: results of an online survey. JMIR Mental Health. 2016;3(2):e15. doi: 10.2196/mental.5379.
13. Singh K, Drouin K, Newmark LP, et al. Many mobile health apps target high-need, high-cost populations, but gaps remain. Health Aff (Millwood). 2016;35(12):2310-2318.
14. Larsen ME, Nicholas J, Christensen H. Quantifying app store dynamics: longitudinal tracking of mental health apps. JMIR Mhealth Uhealth. 2016;4(3):e96. doi: 10.2196/mhealth.6020.
15. Powell AC, Torous J, Chan S, et al. Interrater reliability of mHealth app rating measures: analysis of top depression and smoking cessation apps. JMIR Mhealth Uhealth. 2016;4(1):e15. doi: 10.2196/mhealth.5176.
16. Ducklin P. Apple’s XcodeGhost malware still in the machine…. https://nakedsecurity.sophos.com/2015/11/09/apples-xcodeghost-malware-still-in-the-machine. Published November 9, 2015. Accessed May 11, 2017.
17. Rosenfeld L, Torous J, Vahia IV. Data security and privacy in apps for dementia: an analysis of existing privacy policies. Am J Geriatr Psychiatry. 2017;25(8):873-877.
18. Torous J, Levin ME, Ahern DK, et al. Cognitive behavioral mobile applications: clinical studies, marketplace overview, and research agenda. Cogn Behav Pract. 2017;24(2):215-225.
19. Owen JE, Jaworski BK, Kuhn E, et al. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health. 2015;2(1):e7. doi: 10.2196/mental.3935.
20. Sarkar U, Gourley GI, Lyles CR, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med. 2016;31(12):1417-1426.
