A Peer Review Model for Program Evaluation in Massachusetts
Read this published article featured in the MIE Spring 2024 Journal by MLAC’s Erin Horan, Director of Communications, and Christian Rivera, Communications Associate.
As the state’s largest funding source for civil legal aid organizations, the Massachusetts Legal Assistance Corporation (MLAC) upholds its commitment to
supporting quality services by conducting regular program evaluations. Each year, MLAC uses a peer review model to assess a select number of its grantees. MLAC contracts with legal aid professionals outside of Massachusetts to serve as independent consultants for the peer review process. Peer reviewers identify program strengths and challenges and provide recommendations to improve performance and service.
MLAC recently sat down with three veteran peer reviewers, John Tobin, Althea Hayward, and Niya Swint to hear their perspectives about the experience
and the benefits of serving the larger legal aid community in this way.
MLAC: How would you describe MLAC’s peer review process?
John: While parachuting into an organization may appear daunting, MLAC provides you with all the resources you need. Patricia [Swansey], MLAC’s
Program Evaluation and Monitoring Director, gives peer reviewers a wealth of data to start with, including casework, information on the staff – such as experience, roles, and diversity — program accomplishments, and more. You are also given the most recent peer evaluation report from the organization, to use to measure progress and get a sense of the organization’s recent history.
You are a part of a team of peer reviewers, and you have a preliminary team meeting before you begin and can plan out your interview schedule for the days ahead — you have two days to meet with a variety of staff at the organization. You can meet with people one-on-one or partner with another reviewer for the meetings.
It is a big responsibility, and I appreciate that MLAC gives its reviewers a lot of autonomy in directing their own interviews and assessments. The experience is extremely rewarding.
Althea: MLAC created a concise set of program standards based on the American Bar Association’s standards. I made a chart based on MLAC’s standards,
and then created questions designed to elicit whether these standards are being met.
I use my document to follow up on issues that need to be addressed; I like structure!
Since the COVID-19 pandemic, the entire process has been conducted over Zoom. I like that flexibility because it allows me to continue to help care for my granddaughter.
Niya: Patricia presents which programs are available for review each fiscal year and the programs’ preferred timeline for the process. Our goal is to acknowledge and celebrate the strengths of an organization, and then assess opportunities for growth.
MLAC: How do peer evaluations benefit programs, and how do peer reviewers benefit from the experience?
John: At the end of the day, it is always about the clients — the people served by legal aid. Peer evaluators are working to address client needs by reporting on what works well for organizations and what does not. The client should remain at the forefront of your mind as you embark on the journey. You may feel that you are not knowledgeable, or well equipped to take on this task, but it is important to remember that many of the issues experienced in the Massachusetts civil legal aid community are felt across the country. You will be working with a small team of other evaluators, all bringing your own insights to the table to help an organization better serve its clients.
Niya: Peer reviewers can offer organizations guidance based on our experiences, including what we have learned from conducting other peer evaluations.
We can share ideas for accomplishing solutions and addressing issues.
On a personal note, I have been able to apply learnings from the experience to my role at Legal Aid of North Carolina. It is fulfilling to know I am able to make an impact, help an organization better show up for our neighbors, and influence the client experience. I am reminded that it is a privilege to partner with clients as they navigate the legal justice system, acting as a trusted guide.
I also should mention that peer reviewers are paid for their time and work – that is important!
Althea: The MLAC peer review process stresses technical assistance—that is the approach we take. We often ask the question, “How can we help the program do things differently?” The team weighs in, valuing each person’s contributions and experience. It works!
MLAC: What are you looking to learn or teach in conversations about diversity, equity, and inclusion?
Niya: Equity is the goal. We are beyond performative actions and words. I start these conversations with data and tie it to the client population statistics
provided by MLAC. Then I ask: “How can we better serve our clients?”
We need to be better at making sure staff reflect the populations they serve and bring cultural sensitivity to clients.
One trend I notice all too often in legal aid organizations is using Spanish-speaking staff as translators or interpreters without providing them extra support for their actual roles or compensating them for this skill.
Althea: My approach to peer review is grounded in client access and the quality of services provided by a program. Usually, I approach peer review interviews with those things at the forefront of my mind; the second lens I apply is that of racial justice. I aim to ensure that all people who cannot afford an attorney get access to quality services. I have found that people of color, immigrants, and other marginalized persons face greater barriers when you factor in other demographics aside from socioeconomic status.
In a country where the tenor of racial justice issues confronts us regularly, we take the time to look closely at an organization’s policies, practices, and staffing. We look to see whether eligible persons can receive the highest level of services possible; whether they get to be interviewed comprehensively, and whether they can talk to a staff member or pro bono attorney who is culturally sensitive and informed about the challenges their community may be facing.
MLAC: What is some advice you would give for conducting interviews?
Niya: I think a good strategy is treating each interview like a meeting with new clients. Your job is not to embarrass or shame an organization – you are not there to have a “gotcha” moment. Instead, be intentional about building a rapport.
My unfamiliarity with the civil legal landscape in Massachusetts does not make the job impossible. It just requires humility and a willingness to learn and teach. I begin by setting clear expectations, sharing my identity, role, and limitations, to ease an organization’s initial hesitation and create a sense of trust.
John: Meet with as many people as possible, including volunteers, support staff, client board members, and attorneys. You may want to include another peer reviewer in some of your interviews, because it is helpful to get different perspectives. The best part is sitting down with someone — and before COVID, going into their office, meeting them where they are — and hearing their stories.
Althea: Plan and prepare. Our teams spend time preparing for interviews. The interview schedule is critical, so we try to conduct as many as possible. And, we always interview community organizations and client eligible board members so we can get a full picture of the program’s profile in the community.
MLAC: Can you talk about the final report — how does that come together?
John: MLAC has a helpful template for the final report that reviewers use, and each person is responsible for a different part. We take all that we witnessed, read, and any information collected through the interview process, and we meet to reconcile notes gathered between different reviewers. Then we work on our sections and put the report together.
Althea: The team decides together which performance standards best suit our expertise, although we agree that we will provide feedback on all the standards. Following the completion of interviews, the team meets to synthesize the initial information gathered and plan a preliminary feedback session with the program.
MLAC: What is the best part of being a peer reviewer?
Althea: I enjoy doing this work. It is my way of contributing to the legal services and equal justice community. I get to watch programs go from frustration to accomplishment as they finally start to see the positive impact of the peer review recommendations. The teams share a wealth of experience to help programs solve challenges. We want to help programs perform at their greatest potential. To see it happen during the peer review process is exciting. It is what makes this work worth it
John Tobin dedicated his career to helping those in need of civil legal aid services. He served as the Executive Director of New Hampshire Legal Assistance for 38 years, retiring in 2014. He is also a former member of the MIE Journal Editorial Board. Since his retirement, John has kept busy as one of the founders of the New Hampshire School Funding Fairness Project, a grassroots organization advocating for greater equity in public school funding, where he helps bring the issues of inadequate school funding and unfair property taxes into the forefront of public discussion in New Hampshire.
Althea Hayward’s career as a legal administrator spans more than 30 years. The early part of her career was spent as an administrator with a Legal Services Corporation (LSC) program in Virginia, where she became the Deputy Director and then Executive Director before assuming a position at LSC. She was hired by LSC as a program counsel and member of the State Planning Team. Later, she became the Deputy Director of LSC’s Office of Program Performance (OPP). In that position, she assisted the OPP Director in managing approximately 29 staff who were responsible for quality oversight, grantmaking, and special program initiatives for LSC’s field programs. Her experience in a field program and as Deputy Director of LSC’s OPP provided
her with a wealth of experience in board governance, intake management, client access, and program operations.
Niya Swint is the Chief Community Engagement Officer for Legal Aid of North Carolina (LANC), where she serves as a key liaison for the firm with its external stakeholders. Her leadership role involves designing and executing an external affairs program that amplifies support from diverse stakeholders, expanding LANC’s impact across the state. Before transitioning to that leadership role, she served as a supervising and staff attorney with LANC, providing free legal services to individuals with low incomes throughout North Carolina for over ten years. Her experience and ability to effectively engage with individuals from diverse backgrounds is an integral part to her work.
MLAC is actively seeking peer reviewers for Fall 2024! If you are interested in becoming a peer reviewer, please contact Patricia Swansey, MLAC’s Director of Program Evaluation and Monitoring (pswansey@mlac.org). Peer reviewers have at least five years of experience working at a legal aid organization. Specific areas of expertise such as intake, fundraising, or pro bono programs are also valued.
