Using Data to Drive Equity in the Learning Environment: A Discussion of How MCW is Working to Understand our Data
Leon J. Gilman, Tavinder K. Ark, PhD, Michael N. Levas, MD, MS, Karen Mann, and Jerel Ballard, Malika L. Siker, MD
The authors were panelists in the Second Annual Summit on Advancing Equity in the Learning Environment on April 20, 2023. The following article summarizes what was presented and discussed, and provides next steps we should take for all students to thrive…
Every student deserves to learn in an environment where they are respected and valued. Unfortunately, data from multiple sources over the last few years reveal that our health science students who identify with groups historically underrepresented in medicine (URiM) experience the learning environment as less supportive and inclusive. Consistent with findings from many other health science schools, these results are not unique to MCW. As part of our institutional effort to examine how the learning environment may be contributing to these differences, we have been designing data-based strategies to better understand root causes, design tools and strategies to mitigate areas of concern, and monitor changes. The panelists were tasked with sharing and discussing examples of how we are using data to inform sustainable changes needed to ensure all students and trainees have a learning environment that supports their academic excellence.
Looking at National Residency Matching Program (NRMP) Data (presented by Leon J. Gilman)
Through a partnership with the Office of the Dean from the Medical School, Medical College of Wisconsin Affiliated Hospitals, and the Office of Diversity and Inclusion, data was collected from all residency programs shortly after the 2022 National Resident Matching Program (NRMP) and linked with the Electronic Residency Application Service (ERAS) to better understand how MCW students perceive and are perceived by MCW residency programs through the selection process.
This unique dataset linkage enabled demographics and interview information to be connected to the ranked ordered list of applicants to residency programs. Outcomes of the selection process for female, URiM as a combined group, Black and Hispanic applicants, MCW as unique groups, and non-MCW applicants. The residency selection process was divided into five stages: applications, interview invitations, interviews, being ranked by a program, and matching to a MCWAH residency.
Figure 1 shows overall results for all groups across all stages. Results for 2022 showed an ever-increasing percentage of female, URiM, and Black applications as applicants went through the process. One interpretation of these results suggests that female and Black applications to our graduate medical programs were more likely to be ranked highly. Analyses exploring MCW compared with non-MCW applications showed that female and URiM MCW applications - applicants more likely to be well known to the programs - were more likely to be ranked more highly than their non-MCW counterparts.
Although, in general, Black applicants were likely to be ranked relatively highly, MCW Black applicants were less likely to match to MCWAH residencies (Figure 2). This could indicate that the applicants who are MCW students did not rank MCW programs highly on their own ranking lists because they preferred to train elsewhere. Following this data trend in future years and using other methods (e.g., surveys, interviews and focus groups) will help us understand the underlying explanations and implications of this phenomena.
The Role of Individuals in Advancing Equity in the Learning Environment (Presented by Malika Siker, MD)
As unique individuals who play a role in the learning environment, we can each be intentional in addressing equity by following these 3 steps: 1. Remember; 2. Recognize; and 3. React.
As educators, we must remember that our students may face challenges outside the classroom, clinic, or lab that impact their ability to flourish in the learning environment. The impact of racism, sexism, violence, prejudice, discrimination, socioeconomic challenges, and more can produce inequities, including access to resources, dedicated time for studying, and attention to mental health. We must cultivate an environment where challenges that impede our students’ success are reduced when possible so that students can focus on becoming excellent health professionals.
Data can expose the systemic contributions to inequities in education and how each of us, as individuals, may be contributing to this. We should recognize and productively examine our own individual biases, faults, and imperfections. While these should not define us as educators, what we do to mitigate them will. There are many tools to assist us in doing this work. One research-based tool that has been used to better understand individual unconscious biases and build curricula to mitigate the negative impacts of such biases on educational programs is the implicit association test through Harvard University and other sites. Other individual data to monitor and explore include the diversity of the students we mentor, evaluating the language used in letters of support for bias, and considering which businesses we support.
Finally, we each need to react by seeking ways to have an impact within our circle of control and by expanding our circle of influence while advocating for those who may not have the same privileges or power. In the spirit of continuous improvement and lifelong learning, we can check in with ourselves and hold ourselves accountable by tracking our own data.
We have an individual responsibility to remember the importance of addressing equity, recognize how our individual behaviors can reveal bias or prejudice -- either implicit or otherwise -- and react to mitigate our shortcomings.
Individual data is a powerful tool for understanding the scope of these inequities, empowering change, and ensuring accountability so that our community is a place where all students have the opportunity to thrive.
How Clinical Departments use Data to Ensure Equity (Presented by Michael Levas, MD)
The Department of Pediatrics (DOP) uses data in diversity, equity, and inclusion (DEI) efforts at the department level in many ways. Recently, the DOP fielded an anonymous survey of the entire faculty and staff. Much was learned through this process about the various identities held by members of the department and the range of opinions on DEI efforts. Of the over 250 responses (45% response rate), over 95% supported continued DEI efforts. This finding suggests that while our current efforts align with the majority of the DOP, input from those who are less supportive of the DEI efforts will be considered to tailor future practices and tactics intended to enhance our institutional culture. The DOP also follows participation in DEI trainings actively within each section and uses that data to offer targeted solutions for increased participation.
Acknowledging both the need to address well established inequities based on national historical data and the imperfection of current data tracking systems, the DOP is moving forward with tactics intended to increase diversity and representation while striving to continually improve the data collection.
What data are we missing? (Discussion among panelists and audience)
We need to understand the context of what and why we see the data we do see. National data gives us one perspective. But local data is needed to round out the picture and address local issues. There is so much we do not yet know. For instance, what is the percentage of URiM applicants that apply to our residency programs? Is MCW above or below those averages? Who is being accepted for interviews? Of those that do match and decide not to stay at MCW, is this a success story or something to be concerned about?
The Panel Discussion
Through the panel discussion, we learned that many participants demonstrated curiosity about the data and potential underlying root causes for the trends that emerged. We sought to provide transparency and opportunities for discussion with the diverse group of stakeholders present in the audience. As panelists, we presented the data so that the audience could see what is available to us, including the data’s strengths and limitations, what we are discovering through integrating data from different sources, how we are approaching data analysis, and the potential these efforts have in informing our DEI work.
After engaging with the probing questions from the moderators and panel audience members, a prevailing theme emerged. We learned how important it is to clearly describe our data collection as well as our wrangling and analysis methodology in order to engage the community in interpreting the findings and refining the insights. In this way, we can ensure that our work provides tangible, valid and inspiring support for progress toward our institutional goals.
Overall, there are several takeaways from our panel presentations and the community discussion that followed:
- First, we should not let perfection become the enemy of progress, an all too easy trap to fall into. While the data is not perfect, we must continue to collect it and strive to improve the data quality and processes without stalling progress.
- Second, advancing equity in the learning environment is not the responsibility of one office. Each one of us has a role in contributing to achieving equity. Collaboration is a powerful way to make impactful change.
- We will strive to combine different data sources to get an ever more robust, meaningful, and complete picture of the learning environment.
- It is essential to continue to collect data at all levels for the purpose of working to develop tools, strategies and interventions to mitigate the measured inequities.
At every level of an institution, we can use data to help all students thrive in an equitable learning environment.
Leon J. Gilman is a Data Analytics and Research Specialist in the Office of Diversity and Inclusion and the Center for the Advancement of Women in Science in Medicine at MCW.
Tavinder K. Ark, PhD, is the Director of the Data Science Lab at the Robert D. and Patricia E. Kern Institute for the Transformation of Medical Education.
Michael N. Levas, MD, MS, is an Associate Professor in the Department of Pediatrics at MCW. He is also the Vice Chair for Inclusion, Diversity, and Equity in the Department of Pediatrics.
Karen Mann is a Learning Specialist III in the Office of Diversity and Inclusion at MCW.
Jerel Ballard is a Communications Consultant for the Office of Diversity and Inclusion, the Center for the Advancement of Women in Science and Medicine, and the Office of Student Inclusion and Diversity at MCW.
Malika Siker, MD, is an Associate Professor in the Department of Radiation Oncology at MCW. She is also the Associate Dean for Student Inclusion and Diversity.