Oakland Measure Z Evaluation Scope of Work
DRAFT 2022/03/17
Evaluation Activities
The contractor will conduct process and outcome evaluations of DVP and OPD activities funded by Measure Z. The following sections describe the overarching research questions and then the evaluation components for DVP and OPD separately. Where possible, the contractor will streamline activities across the DVP and OPD evaluations.
Research Questions
The evaluation will focus on the ten research questions listed below.
Process Evaluation
1. Are Measure Z activities implemented with fidelity?
2. What are the facilitators of and barriers to success for each DVP and OPD sub-strategy and activity?
3. How are violence dynamics and other relevant social dynamics (e.g., the COVID-19 pandemic) affecting Measure Z activities?
4. How do the different Measure Z components interact and relate to an overall approach to violence reduction?
5. How many individuals were served in each program? How many incidents were responded to? How many community activities occurred? What were the characteristics of these clients/incidents/activities?
6. What was the intervention dosage, at the client, family, and community levels? Outcome Evaluation
7. Do Measure Z activities reduce gun and gender-based violence at the community level? If so, what are the contributions of specific activities to that reduction?
8. Do people engaged in Measure Z services fare better in terms of safety and well-being than similarly situated people who do not participate in services?
9. Do Measure Z Community Healing and Restoration activities affect community perceptions of safety and well-being?
10. Do Measure Z OPD activities affect community perceptions of police?
Department of Violence Prevention Evaluation Activities The contracted evaluation will consist of a process evaluation and an outcome evaluation of the sub- strategies from the DVP’s FY22-24 Strategic Spending Plan (Table 1), which are grouped into the following categories: (1) gun, group, and gang violence response, (2) gender-based violence response, and (3) community healing and restoration. These strategies will be implemented through contracts with 30+ CBOs.
The evaluation will use mixed qualitative and quantitative methods to fully assess the implementation of the activities and the impact on individual, family, school, and community outcomes.
The evaluation will assess how program activities are carried out in accordance with Measure Z’s goals and objectives. Randomization and control groups will be employed where possible. The specific research design will be developed in collaboration with DVP to ensure the most, rigorous feasible design. The evaluation will focus on DVP activities from July 2022 through June 2024.
Table 1. DVP Measure Z Activities
Strategy Area Sub-Strategy Activities
Gun/Group/
Gang
Violence
Response
Violent Incident Crisis Response
Community Violence Responders
Hospital-Based Intervention
Family Support
Emergency, Temporary Relocation
Youth Diversion & Youth and Adult Life Coaching
Youth Diversion
Youth Life Coaching
Adult Life Coaching
Employment & Education Support Services Youth Career Exploration & Education Support Adult Employment & Education Support
School-Site Violence Intervention & Prevention Teams
Gender-
Based
Violence
Response
Crisis Response
24-Hour Hotlines
Bedside Advocacy
Accompaniment
Housing Emergency Shelter
Transitional Housing
Wraparound Services
Life Coaching
Legal Advocacy
Therapeutic Support
GBV-Employment
Drop-In Center
Community
Health and
Restoration
Neighborhood & Community Teams with Town Nights
Healing/Restorative Activities
Therapeutic Supports
Community Capacity Building & Mini-Grants
The contractor will be responsible for:
• Development of all aspects of evaluation design, including the selection of metrics and the development of data collection tools and protocols, in collaboration with the DVP. • Implementation and oversight of all evaluation activities, in coordination with the DVP and DVP- funded providers/grantees.
• Development of memorandum(s) of understanding with city and county entities such as the Alameda County Probation Department, Oakland Police Department, and Oakland Unified School District to access administrative data.
The data collection activities for the DVP evaluation may include:
• Semi-structured interviews with DVP staff, staff at CBOs providing DVP contracted services, and staff at key partners in the Measure Z-funded work
• Focus groups with individuals and family members receiving DVP services through the three strategy areas subject to this evaluation
• Collection of performance measurement data reported by DVP contracted CBOs to the data system maintained by DVP
• Surveys of Oakland community members
• Collection of community-level data on violence outcomes of focus for Measure Z • Collection of individual-level data on outcomes from administrative data maintained by Alameda County Probation, the Oakland Unified School District, and other potential sources of relevant outcome data such as the California Employment Development Department
Throughout the evaluation, the contractor will communicate regularly with DVP. This may include phone or email communication, and phone, videoconference, or in-person meetings. More frequent or extensive collaboration may be necessary during the project design phase and leading up to publications. DVP will have the opportunity to review any reports for factual accuracy before publication.
Oakland Police Department Evaluation Activities
The contractor will conduct a process and outcome evaluation of Oakland Police Department (OPD) services funded by Measure Z (excluding the Ceasefire strategy). These services are divided into the following three categories: (1) Geographic policing and crime reduction teams, (2) Community policing services, and (3) Special victims’ services.
The primary goal of the OPD evaluation is to assess whether Measure Z funds are effectively used by OPD to achieve its objectives, which include 1) reducing homicides, robberies, burglaries, and gun-related violence; 2) improving 911 response times and other police services; and 3) investing in violence intervention and prevention strategies. The contractor acknowledges the importance of promoting police accountability by tracking policing activities funded under Measure Z and providing performance analysis as required by the Measure Z legislation. The contractor will also assess the extent to which policing services funded by Measure Z led to a measurable reduction in violence.
The specific evaluation activities and design will be decided in consultation with OPD and relevant stakeholders. The contractor will explore the feasibility of experimental and quasi-experimental research designs. The data collection activities may include surveys, interviews, or focus groups with OPD staff; collection of administrative data from OPD; interviews with people who receive special victims’ services; and a community survey. The evaluation will focus on OPD’s Measure Z services since January 2021. However, the contractor may need data from years prior to 2021 to establish trends and may ask staff about activities prior to 2021 for context.
Throughout the evaluation, the contractor will communicate regularly, as needed with OPD. This may include phone or email communication, and phone, videoconference, or in-person meetings. More frequent or extensive collaboration may be necessary during the project design phase and leading up to publications. OPD will have the opportunity to review any reports for factual accuracy before publication.
Deliverables
Multiple reports and presentations will be prepared for the DVP and OPD outcome evaluations. The deliverables are described below and summarized in Table 2. The quantity and audience of presentations will be decided in consultation with DVP and OPD stakeholders.
Department of Violence Prevention Deliverables
• Annual program data overviews for each of the DVP’s 30+ contracted providers (anticipated Nov 2023 and 2024)
• One FY22-23 interim process and outcome evaluation report, describing methods and preliminary findings, for all 11 DVP sub-strategies (anticipated Dec 2023) • One FY22-24 comprehensive process and outcome evaluation report, describing all methods and findings, for all 11 DVP sub-strategies (anticipated Mar 2025) • Two PowerPoint presentation slide decks that summarize the combined process and outcome evaluation findings to use in communicating to community and professional audiences (anticipated Dec 2023 and Mar 2025)
• Presentations of findings from each combined process and outcome evaluation report to community and professional audiences
Oakland Police Department Deliverables
• One interim evaluation report, describing methods and initial findings from process and outcome evaluations of OPD’s geographic policing and crime reduction teams; community policing services; and special victims’ services (anticipated Nov 2023) • One final evaluation report, describing all methods and findings of the process and outcome evaluations of OPD’s geographic policing and crime reduction teams; community policing services; and special victims’ services (anticipated Nov 2024) • Two PowerPoint presentation slide decks that accompany each evaluation report to use in communicating findings to community and professional audiences (anticipated Nov 2023 and 2024)
• Presentations of findings from each evaluation report to community and professional audiences Table 2. Deliverables Schedule
Org Deliverable Date
DVP FY22-23 program data overviews for DVP’s 30+ providers Nov 2023 OPD Interim process and outcome evaluation report for OPD’s 3 services Nov 2023 OPD Slide deck to accompany process and outcome evaluation report Nov 2023 DVP FY22-23 process and outcome evaluation report for DVP’s 11 sub-strategies Dec 2023 DVP Slide deck to accompany FY22-23 process and outcome evaluation report Dec 2023 DVP FY23-24 program data overviews for DVP’s 30+ providers Nov 2024 OPD Final process and outcome evaluation report for OPD’s 3 services Nov 2024 OPD Slide deck to accompany process and outcome evaluation report Nov 2024 DVP FY22-24 process and outcome evaluation report for DVP’s 11 sub-strategies Mar 2025 DVP Slide deck to accompany FY22-24 process and outcome evaluation report Mar 2025