- Therapists, Trainers, and Researchers from Lykos’ Phase 3 MDMA-Assisted Therapy Studies Respond to Concerns in ICER’s Draft Evidence Report - May 7, 2024
- Towards an Ethos of Equity and Inclusion in the Psychedelic Movement - December 18, 2019
- Blinded by the White: Addressing Power and Privilege in Psychedelic Medicine - March 13, 2019
The Institute for Clinical and Economic Review (ICER), is an independent non-profit research organization that evaluates the evidence on the clinical and economic value of prescription drugs, medical tests, devices, and health system delivery innovations.
April 22, 2024
Foluso Agboola, MBBS, MPH
Vice President of Research
Institute for Clinical and Economic Review (ICER)
Two Liberty Square, Ninth Floor
Boston, MA 02109
Dear Ms. Agboola,
We, the undersigned, comprise clinical investigators, clinicians, supervisors, trainers, and medical monitors who worked on the Phase 3 clinical trials of MDMA-Assisted Psychotherapy (MDMA-AP) for PTSD. We share a deep commitment to scientific inquiry, ethical research practices, and the integrity and validity of research outcomes. Drawing from our extensive research and clinical experience at major teaching institutions, hospitals, and individual practices, we are compelled to share our concerns about ICER’s March 26, 2024 draft evidence report, entitled 3,4-Methylenedioxymethamphetamine Assisted Psychotherapy for Post-Traumatic Stress Disorder (PTSD). The report draws attention to a number of important issues in psychedelic therapy research, such as the problem of functional unblinding, the potential role of expectancy in clinical outcomes, and the importance of ethical practice. However, certain aspects of the Phase 3 MAPP1 and MAPP2 trials of MDMA-AP for PTSD (Mitchell et al., 2023, Mitchell et al., 2021) are misrepresented in the draft report. We are writing to correct and contextualize some of the assertions made there, as failing to do so would represent a disservice to both the scientific endeavor and the patients it aims to benefit.
One hundred and nine therapists and principal/co-investigators contributed to the Phase 3 trials of MDMA-AP for PTSD. To our knowledge, none of them were consulted before the preliminary report was issued. However, this group is in the strongest position to describe the studies and address accusations related to inappropriate study design and conduct. In the absence of such input, a number of assertions in the ICER report represent hearsay, and should be weighted accordingly. This consideration is particularly important because the two sources referenced in section 2.1, “Concerns About Trials of MDMA-AP,” are a podcast and an online article written and produced by individuals who have repeatedly and publicly expressed strongly negative views about the medicalization of psychedelic substances (see Devenot, 2024; Nickles, 2018, 2020), underscoring the high risk of bias in the current draft report.
Choice and validity of the primary outcome measure
The draft report calls into question the validity of the primary outcome measure of the Phase 3 trials, namely, the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5). The CAPS-5 is the gold-standard measure of symptom severity used in clinical trials for PTSD (Weathers, et al., 2013). Closely adhering to diagnostic criteria in the DSM-5, the CAPS-5 measures PTSD symptomatology anchored to a specific index trauma (or series of traumatic events) that meets DSM-5 Criterion A for a PTSD diagnosis. Although many participants may have had multiple distinct life events that contribute to their overall experience of PTSD, rigorous research demands that outcome measures be standardized. As such, the traumatic event selected as the index trauma for the baseline CAPS-5 assessment was also used for subsequent assessments. The ICER report notes that a limitation of this measure is the fact that a participant’s symptoms may improve with respect to their identified index trauma, while actually worsening with respect to one or more other traumatic memories; that is, a positive CAPS-5 result might inaccurately represent the patient’s overall clinical status. We do not dispute this possibility. However, the accusation that it undermines the validity of the Phase 3 data supporting the use of MDMA-AP for PTSD could be leveled against any other clinical study using this outcome measure—which again, though flawed, is the gold-standard in the field of PTSD clinical research. We do not dispute that some participants may have experienced worsening psychological distress; however, secondary outcome measures (e.g., the Sheehan Disability Scale, the Beck Depression Inventory) and adverse event reporting (e.g., exacerbation of anxiety, suicidality, insomnia) would have captured that distress, even if it was not associated with the index trauma identified on the CAPS-5. However, results from these secondary outcome measures did not show statistically significant worsening; on the contrary, they favored active treatment with MDMA.
Functional unblinding
The ICER report raises the concern that functional unblinding in the Phase 3 trials of MDMA-AP for PTSD may have biased, and might therefore invalidate, the reported outcomes. Though frequently discussed in psychedelic research, functional unblinding is also a concern in
conventional clinical research. For example, pharmaceutical interventions are frequently unblinded due to medication side effects, and neither the patient nor the clinician is blinded to treatment in a comparative-efficacy psychotherapy trial (e.g., EMDR vs. CPT). The fact of functional unblinding therefore cannot undermine efforts to approve new treatments for PTSD. Instead, study design should take measures to minimize the effect of functional unblinding. To do so, the study design in the Phase 3 trials of MDMA-AP for PTSD (Mitchell et al., 2023; Mitchell et al., 2021) used independent raters blind to the participant’s treatment to assess the primary outcome, obviating any concerns of bias that might emerge if the main efficacy endpoint was administered by a participant or their study therapist.
Standardized intervention
The draft report notes that the challenge of standardizing psychotherapy is not unique to the Phase 3 trials of MDMA-AP for PTSD. However, the draft report relies on hearsay (Institute for Clinical and Economic Review, 2024, page 6) to call into question the generalizability of the Phase 3 MDMA-AP results. It does not note the many measures taken to train, support, and evaluate therapists on those trials—measures that met, and in some cases exceeded, the accepted standards in the field of psychotherapy research (Roth et al., 2021; Wang, 2021; Schoenwald & Garland, 2013;). In addition to therapist training, adherence was rigorously assessed in the Phase 3 trials. Over a number of years, many individuals were trained to rate therapists on their adherence to the MDMA-assisted therapy treatment manual (Wang et al., 2021). Rater cohorts went through a rigorous standardization process to establish strong inter-rater reliability before they were certified for Phase 3 trials (Mithoefer, 2017, 2021). By rating therapist adherence to the treatment manual, a standard of fidelity and quality was assured.
Phase 3 clinical trials are highly structured and standardized by design, and the rigidity of clinical research protocols may not be able to meet all patients’ needs. As stated in the draft report, the MAPP1 and MAPP2 protocols did allow for additional integration therapy sessions if these were clinically indicated. Although this flexibility could introduce the potential confound of variable “dose” of therapy, it was driven by the ethical imperative to protect participants and minimize harm. Indeed, our Phase 3 trial experience suggested that some patients might benefit from an extended treatment arc; however, the need for standardized dosing cycles and clear termination time points was a limiting factor, as it is in any clinical trial. Should MDMA-AP receive approval for clinical use, we hope that clinicians will be able to individualize treatment to meet a patient’s unique needs in a way that is not possible within the more rigid framework of a Phase 3 clinical trial.
Expectancy and Accurate Reporting
Given the enormous unmet clinical need that PTSD represents, patients and clinicians are predictably enthusiastic about the prospect of a novel treatment for this life-threatening disorder. This phenomenon is by no means unique to MDMA-AP research. In any clinical trial, the pre-treatment hope for, and expectation of, a clinical benefit may account for a substantial proportion of the overall therapeutic effect (Colloca et al., 2023; Weimer, Colloca, & Enck, 2015). In the Phase 3 trials, therapists discussed with participants what their expectations were regarding the study, and took measures to manage them. In some cases, the expectations might have applied to the acute MDMA experience itself—e.g., a participant may have hoped for an experience of euphoria or relaxation. While such experiences did happen, they were not universal, and the Phase 3 protocols called for significant time to be spent discussing the broad range of possible medication-day experiences, which may have helped limit expectancy bias.
Some who benefit from an investigational treatment, or clinicians who see participants do so, may be tempted to idealize the treatment experience, exaggerating any clinical improvements while minimizing any adverse events in an effort to accelerate other patients’ access to the treatment in the future. In initial research training and monthly clinical investigator meetings, therapists on the Phase 3 trials of MDMA-AP for PTSD discussed the possibility of biased reporting, which might be well-intentioned, but ill-informed. Therapists encouraged participants to be comprehensive in their description of their acute and long-term experiences in the study, noting that Phase 3 trials are designed to identify not only the potential efficacy of a novel treatment, but also its risks. Therapists noted that the participant’s candor in describing the full spectrum of their experience during the study would allow the therapists to support them as fully as possible for the duration of the trial. Moreover, participants were reassured that such candor—and the accurate documentation of adverse events that followed—would allow future patients and clinicians to engage in a comprehensive discussion of risks and benefits that allowed for shared, and patient-centered, decision-making.
Of note, the psychotherapy platform in the Phase 3 trials took a non-pathologizing approach to the participant’s emotional experience and expression, whatever its intensity or valence. As practiced in those trials, MDMA-AP comprises elements from a number of other psychotherapies, but particularly exposure therapies, as participants were invited to access and process traumatic memories. Short-term destabilization was therefore expected, as it is in any psychotherapy incorporating elements of exposure. This was discussed with Phase 3 participants in the informed consent process and throughout the treatment. The ICER draft report intimates that study therapists might have underreported adverse events. Though this seems unlikely, another safeguard against intentional or unintentional bias in the Phase 3 trials was that the entire study team was trained in, and collectively responsible for, adverse event reporting. Clinically significant destabilization (e.g., worsening depression or anxiety) was always documented as an adverse event, either by the study therapists or other study staff.
Ethical Concerns
The draft report indicates that one or more participants in MDMA-AP trials suffered significant boundary violations at the hands of study therapists, and suggests that such experiences would alter the risk/benefit analysis for this combination treatment. Unfortunately, the report relies heavily on one particular, well-publicized case of ethical misconduct in a Phase II trial, as well as anecdotal comments made by a small number of undisclosed study participants and unnamed “experts” rather than validated research outcomes (for validated research see Mitchell et al., 2023; Mitchell et al., 2021; Wagner et al., 2021; Jerome et al., 2020). Moreover, treatment-emergent adverse events should not be confused or conflated with malpractice. That being said, the potential for ethical transgressions in this emerging field should not be minimized.
The Phase 3 trials of MDMA-AP for PTSD included a number of features that were intended to protect the participants from undue harm. First, the principle of active, ongoing informed consent was embedded in the research protocols, and explicitly assessed in therapist training and adherence rating. Second, the therapeutic approach centered the participant’s autonomy and empowerment, and aimed to minimize power imbalances between participants and therapists. Third, Phase 3 therapists were either licensed psychotherapists or on the path to licensure, which ensured a level of personal and professional training and accountability even outside the study framework. Finally, study-specific therapist training and supervision efforts addressed ethical considerations in the practice of MDMA-AP; therapists were expected to adhere to the MAPS Code of Ethics (Carlin et al., 2019); and challenging cases were discussed in multi-site consultation calls during the Phase 3 trials. Indeed, several of the signatories here have dedicated significant time, energy, and scholarship toward education and advocacy regarding ethical practice, safety, and consent so as to prevent future violations (see Luoma et al., 2024; Rosa et
al., 2023; Stauffer et al., 2022; Carlin et al., 2019). We remain committed to self-examination and peer supervision, cultivating self-awareness and seeking out guidance to ensure the safe, ethical practice of MDMA-AP if this treatment receives federal approval.
Conclusion
We appreciate ICER’s efforts to evaluate the strength of the evidence in MDMA-AP research, and to prioritize transparency and collaboration in doing so. We appreciate the opportunity to comment on the draft report, which draws attention to some known challenges in psychedelic research design. It also draws attention to some important considerations related to the practice and implementation of MDMA-AP for PTSD, which will require rigorous training of ethical therapists who are held accountable by clinical practice standards and licensing boards. These challenges and considerations do not undermine the dramatic efficacy data and favorable safety profile seen in the rigorous Phase 3 trials to which we contributed. We hope that the final report will take into consideration our input, which draws on the collective effort and scientific data amassed by hundreds of contributors across over a dozen sites, all under the oversight of institutional, state, and federal regulatory bodies.
Post-traumatic stress disorder is a debilitating mental health condition that significantly disrupts the lives of the 13 million patients who suffer from it, and the many more who care for them. New treatments are urgently needed. Robust primary outcomes in two Phase 3 trials support a positive benefit-risk ratio for MDMA-AP in patients with PTSD, even in severe cases where other treatments have failed. Future research will help quantify whether the benefit-risk profile changes when studying different patient populations, treatment protocols, and/or models of care. But given the enormous unmet need, coupled with robust Phase 3 efficacy data and a favorable drug safety profile, the 13 million patients with PTSD have good reason to hope that access to the combination treatment of MDMA-AP could change their symptoms and their lives. We look forward to reading a future version of the ICER report that more accurately represents the weight of the evidence behind that hope.
Sincerely,
Jamie Beachy, MDiv, PhD, Emory Center for Psychedelics & Spirituality
Kelley C. O’Donnell, MD, PhD, NYU Langone Center for Psychedelic Medicine
Ingmar Gorman, PhD, Fluence International, Inc.
Genesee Herzberg, PsyD, Sage Integrative Health
Evan Sola, PsyD, University of California, San Francisco
Sara Gael Beauregard, MA, Memoru Center for Visionary Healing Arts
Casey A. Paleos, MD, InnerMost PBC; Nautilus Sanctuary
Yevgeniy Gelfand, MD, Zen Therapeutic Solutions
Jim Hopper, PhD, Cambridge Health Alliance; Harvard Medical School
Signatories
Michael Mithoefer, MD
Ann Mithoefer, BSN
Marcela Ot’alora G. MA, LPC, Aguazul-Bluewater Inc.
Bruce D Poulter, RN, MPH, Aguazul-Bluewater, Inc; Memoru Center for Visionary Healing Arts
Ido Siemion, PhD
Keren Tzarfaty, PhD
Jennifer Mitchell PhD, University of California, San Francisco
Elizabeth M Nielson, PhD
Wael Garas, MD
Harvey L. Schwartz, PhD
Susan Walker, MD
Ami Silver, MSW
Ayelet Givati, MSW
Collin M Reiff, MD, New York University, Grossman School of Medicine
Jeffrey Richard Guss, MD, NYU School of Medicine; NYU Postdoctoral Program
Naftali Halberstadt, PhD
Wende Reynolds, MEd, LPC
Tali Nachshoni, MD, Beer Yaakov Mental Health Center
Willa Hall, PhD
Erin Passow, PsyD
Andrea Rosati, MD, PhD, University of California, San Francisco
Elizabeth Call, PsyD
Veronika Gold, MA, Polaris Insight Center
Sara Lewis, PhD, LCSW, Naropa University
William R. Wolfe, MD, University of California, San Francisco
Joseph McCowan, PsyD
Nicole Sylvestre, MA, LPC, Multidisciplinary Association for Psychedelic Studies
Jayne Gumpel, LCSW
Karen M. Peoples, PhD, Psychoanalytic Institute of Northern California
Eve Halpern MA
Jim Grigsby, PhD
Seema Desai, MD
Andrew Penn, MS, PMHNP, University of California, San Francisco
Cristie Strongman, MA, Ed.M. LMHC
Scott Shannon, MD, University of Colorado, Department of Psychiatry
Eric Sienknecht, PsyD, Polaris Insight Center
Charlotte Jackson, MA
Simon Amar, MD, Monash University
Joanna Simundic, MSW RCSW RCC
Andrew Linares, MA, LPC
Shirli Shanta Mishori, MA
Assaf Lederman, MA
Yair Wallach, MD, Beer Yaakov Mental Health Center
José F. Mata, LMFT
Christopher S Stauffer, MD, Oregon Health & Science University
Juan Pablo Galindo, DO
David Gumpel, MA
Annah Schnaitter, MA
Sylvestre Quevedo MD MPH, University of California, San Francisco
Jill Silverman, LCSW
Gregory Wells, PhD
Michael Alpert, MD, Beth Israel-Deaconess Medical Center and Harvard Medical School
Emily Willow, MD
Daniel E. Roberts, MD, MSW
Brooke Balliett, LMFT, California Center for Psychedelic Therapy
Drew Ribadeneyra, BA, Nautilus Sanctuary
Jessie Holladay Duane LCSW
Alexandra Haas, MA, LMFT UCSF
SungLim Shin, PhD
German F. Ascani, MD, MS, Evolve Mind Wellness
Jennifer Jones, MD, Medical University of South Carolina
Jason Sienknecht, LPC
Jay Bradley Foote, MD
References
Carlin, S. C., & Scheld, S. (Eds.). (2019). MDMA-assisted psychotherapy code of ethics. In C. Barstow, M. Cassity, K. M. Cooper, L. Friedman, I. L. Ali, A. Mithoefer, M. Mithoefer, A. Okawa, M. Ot’alora, B. Poulter, D. A. Sisti, K. Taylor, & V. Wieloch (Reviewers), MAPS Bulletin, 29(1). Retrieved from https://maps.org/news/bulletin/maps-mdma-assisted-psychotherapy-code-of-ethics-spring-2019/
Colloca, L., Nikayin, S., & Sanacora, G. (2023). The intricate interaction between expectations and therapeutic outcomes of psychedelic agents. JAMA Psychiatry, 80(9), 867-868.
ClinicalTrials.gov. (2021). Informed consent form for A randomized, double-blind, placebo-controlled, multi-site phase 3 study of the efficacy and safety of manualized MDMA-assisted psychotherapy for the treatment of posttraumatic stress disorder of moderate or greater severity. Retrieved from https://classic.clinicaltrials.gov/ProvidedDocs/37/NCT04077437/ICF_002.pdf
Devenot, N. (2024, April 16). On psychedelic cults and SXSW, why critique the psychedelics industry? Chemical Poetics. Retrieved from https://chemicalpoetics.substack.com/p/on-psychedelic-cults-and-sxsw
Institute for Clinical and Economic Review. (2024). Draft evidence report – MDMA-AP for PTSD (p. 6). Retrieved from https://icer.org/wp-content/uploads/2024/03/PTSD_Draft-Report_For-Publication_03262024.pdf
Jerome, L., Feduccia, A. A., Wang, J. B., Hamilton, S., Yazar-Klosinski, B., Emerson, A., Mithoefer, M. C., & Doblin, R. (2020). Long-term follow-up outcomes of MDMA-assisted psychotherapy for treatment of PTSD: a longitudinal pooled analysis of six phase 2 trials. Psychopharmacology, 237(8), 2485–2497. https://doi.org/10.1007/s00213-020-05548-2
Luoma, J., Allen, L. R., Gold, V., & Stauffer, C. (2024). Getting in touch with touch: The importance of studying touch in MDMA-assisted therapy and the development of a new self-report measure. Psychedelic Medicine.
Mitchell, J. M., Bogenschutz, M., Lilienstein, A., Harrison, C., Kleiman, S., Parker-Guilbert, K., Ot’alora G, M., Garas, W., Paleos, C., Gorman, I., Nicholas, C., Mithoefer, M., Carlin, S., Poulter, B., Mithoefer, A., Quevedo, S., Wells, G., Klaire, S. S., van der Kolk, B., Tzarfaty, K., … Doblin, R. (2021). MDMA-assisted therapy for severe PTSD: a randomized, double-blind, placebo-controlled phase 3 study. Nature medicine, 27(6), 1025–1033. https://doi.org/10.1038/s41591-021-01336-3
Mitchell, J. M., Ot’alora G, M., van der Kolk, B., Shannon, S., Bogenschutz, M., Gelfand, Y., Paleos, C., Nicholas, C. R., Quevedo, S., Balliett, B., Hamilton, S., Mithoefer, M., Kleiman, S., Parker-Guilbert, K., Tzarfaty, K., Harrison, C., de Boer, A., Doblin, R., Yazar-Klosinski, B., & MAPP2 Study Collaborator Group (2023). MDMA-assisted therapy for moderate to severe PTSD: a randomized, placebo-controlled phase 3 trial. Nature medicine, 29(10), 2473–2480. https://doi.org/10.1038/s41591-023-02565-4
Mithoefer, M. (2017). A manual for MDMA-assisted psychotherapy in the treatment of posttraumatic stress disorder. Version 8.1. Retrieved from https://maps.org/wp-content/uploads/2022/05/MDMA-Assisted-Psychotherapy-Treatment-Manual-V8.1-22AUG2017.pdf
Mithoefer, M. (2021). A manual for adherence ratings of MDMA-assisted therapy for treatment of posttraumatic stress disorder. Version 6. Retrieved from https://mapscontent.s3.us-west-1.amazonaws.com/research-archive/Adherence+Ratings+Manual+Version+6_3AUG2021.pdf
Muschalla, B., & Schönborn, F. (2021). Induction of false beliefs and false memories in laboratory studies—A systematic review. Clinical Psychology & Psychotherapy, 28(5), 1194-1209.
National Center for PTSD. (n.d.). DSM-5 criteria for PTSD. Retrieved from https://www.ptsd.va.gov/professional/treat/essentials/dsm5_ptsd.asp#one
Nayak, S., & Johnson, M. W. (2021). Psychedelics and psychotherapy. Pharmacopsychiatry, 54(04), 167-175.
Nickles, D. (2018, October 25). It’s time to debunk prohibitionist narratives and calls for monopolies within psychedelic science. Psymposia. Retrieved from https://www.psymposia.com/magazine/its-time-to-debunk-prohibitionist-narratives-and-calls-formonopolies-
within-psychedelic-science/
Nickles, D. (2020, July 17). We need to talk about maps supporting the police, the military, and violent white supremacism. Psymposia. Retrieved from https://www.psymposia.com/magazine/acab/
Rosa, W. E., Sager, Z., Miller, M., Bernstein, I., Doerner Rinaldi, A., Addicott, K., Ljuslin, M., Adrian, C., Back, A. L., Beachy, J., Bossis, A. P., Breitbart, W. S., Cosimano, M. P., Fischer, S. M., Guss, J., Knighton, E., Phelps, J., Richards, B. D., Richards, W. A., Tulsky, J. A., …
Beaussant, Y. (2022). Top Ten Tips Palliative Care Clinicians Should Know About Psychedelic-Assisted Therapy in the Context of Serious Illness. Journal of palliative medicine, 25(8), 1273–1281. https://doi.org/10.1089/jpm.2022.0036
Roth, A. D., Pilling, S., & Turner, J. (2010). Therapist training and supervision in clinical trials: Implications for clinical practice. Behavioural and Cognitive Psychotherapy, 38(3), 291-302.
Schoenwald, S. K., & Garland, A. F. (2013). A review of treatment adherence measurement methods. Psychological Assessment, 25(1), 146.
Stauffer, C. S., Brown, M. R., Adams, D., Cassity, M., & Sevelius, J. (2022). MDMA-assisted psychotherapy; Inclusion of transgender and gender diverse people in the frontiers of PTSD treatment trials. Frontiers in Psychiatry, 13, 932605.
Wagner, A. C., Liebman, R. E., Mithoefer, A. T., Mithoefer, M. C., & Monson, C. M. (2021). Relational and growth outcomes following couples therapy with MDMA for PTSD. Frontiers in Psychiatry, 12, 702838.
Wang, J. B., Lin, J., Bedrosian, L., Coker, A., Jerome, I., Feduccia, A., … & Doblin, R. (2021). Scaling up: multisite open-label clinical trials of MDMA-assisted therapy for severe posttraumatic stress disorder. Journal of Humanistic Psychology.
Weathers, F. W., Blake, D. D., Schnurr, P. P., Kaloupek, D. G., Marx, B. P., & Keane, T. M. (2013). The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5). Interview available from the National Center for PTSD at www.ptsd.va.gov.
Weimer, K., Colloca, L., & Enck, P. (2015). Placebo effects in psychiatry: mediators and moderators. The Lancet Psychiatry, 2(3), 246-257.
Note: You can download a copy of the letter here.
Take a minute to browse our stock:
Did you enjoy reading this article?
Please support Chacruna's work by donating to us. We are an independent organization and we offer free education and advocacy for psychedelic plant medicines. We are a team of dedicated volunteers!
Can you help Chacruna advance cultural understanding around these substances?
Become a Chacruna Member
To make a direct donation click the button below:
