San Jose City Council
October 17, 2023

TO: HONORABLE MAYOR AND FROM: Toni J. Taber, CMC
CITY COUNCIL City Clerk

SUBJECT: SEE BELOW DATE: October 17, 2023

SUBJECT: Housing Department’s Performance Measures Audit Report

Recommendation

As recommended by the Community and Economic Development Committee on September 25, 2023, accept the audit report on the Housing Department’s performance measures audit.

CEQA: Not a Project, File No. PP17-009, Staff Reports, Assessments, Annual Reports, and Informational Memos that involve no approvals of any City action. (City Auditor) [Community and Economic Development Committee 9/25/2023 - Item (d)2]

COUNCIL AGENDA: 10/17/2023
ITEM: 3.5
FILE NO: 23-1385

Office of the City Auditor

Report to the City Council
City of San José

HOUSING PERFORMANCE
MEASURES: THE CITY

SHOULD FOCUS AND ALIGN

MEASURES TO SUPPORT

DECISION-MAKING

Report 23-04
September 2023

This page was intentionally left blank

200 E. Santa Clara Street, San José, CA 95113
Telephone: (408) 535-1250 Email: [email protected] Website: www.sanjoseca.gov/auditor/ Office of the City Auditor
Joe Rois, City Auditor

September 15, 2023

Honorable Mayor and Members
Of the City Council
200 East Santa Clara Street
San José, CA 95113

Housing Performance Measures: The City Shou ld Focus and Align Measures to Support Decision-Making

In recent years, the City of San José (City) has prioritized preventing homelessness and creating affordable housing. The City’s Housing Department (Housing) plays a major role in achieving the City’s long-term goals in these areas. Other Housing services includ e rent stabilization and tenant protection programs, portfolio management of affordable housing loans, neighborhood capital investment, and others. Housing reports progress and activities of its programs in di fferent ways, such as performance measures included in the City’s Adopted Operating Budgets, Housing’s Annual Impact report, online dashboards, and others.

The objective of this audit was to assess Housing’s performance metrics. Though this audit was focused on Housing’s services and performance metrics, other departments may serve the unhoused community or have programs impacted by homelessness. We did not include those departments in this review. This work is intended to assist the Administration to enhance the City’s performance management systems in alignment with its Outcomes, Equity Indicators, and Performance Management initiative.

Finding 1: The Housing Department Can St reamline and Improve Current Performance Measures. The Housing Department reports more than two hundred performance measures in different forms across various reports and online dashboards. Many of the reported measures are important and useful for users of the information; others are required because of funding sources. However, we found:
 The City’s Operating Budget can provide more useful and meaningful measures that better reflect department
priorities.
 Better alignment of performance measures across the
various reports and dashboards would help users of the information better understand results and save staff time compiling the information.
 Housing can also better document consistent
methodologies for calculating measures and provide
meaningful targets to put context around the results.

RECOMMENDATIONS:

The City Manager’s Office and
Housing Department should:
 Reduce and streamline the
number of performance measures
for Housing’s programs,
 Align measures across Housing’s
different reporting platforms and
document the methodology for
calculating measures to ensure
consistency, and
 Report meaningful targets to
provide context for results.
4 This report has three recommendations. We plan to present this report at the September 25, 2023 meeting of the Community & Economic Development Committee of the City Council. We would like to thank the Housing Department, the Budget Office, and City Manager’s Office for their time and insight during the audit process. The Administration has reviewed the information in this report, and their response is shown on the yellow pages.

Respectfully submitted,

Joe Rois
City Auditor

Audit staff: Stephanie Noble

cc: Jennifer Maguire Rosalynn Hughey Kemit Mawakana
Lee Wilcox Rachel VanderVeen Nora Frimann
Dolan Beckel Ragan Henninger Kevin Fisher
Omar Passons Kelly Hemphill Jim Shannon
Erik Jensen Gregory Pensinger Claudia Chang
Jennifer Piozet Nathaniel Montgomery James Gold

This report is also available online at www.sanjoseca.gov/audits

T able of Contents
Cover Letter ........................................................................................................... 3 Background ............................................................................................................. 7 Finding I
The Housing Department Can Streamline and Improve Current Performance Measures ........................................................................................ 13 The Housing Department Reports More than Two Hundred Performance Measures ............................................................................................................................... 13

Clear and Meaningful Targets Can Provide Insight into Incremental Progress Toward Larger Goals ......................................................................................................... 22 Conclusion ............................................................................................................. 25 Appendix A
Audit Objective, Scope, and Methodology ...................................................... A-1 Appendix B
Partial Listing of Housing Department Performance Measures From Departmentwide Reports and Reports and Dashboards Focused on Affordable Housing and Homelessness............................................................. B-1 Administration’s Response ................................................................. yellow pages

T able of Exhibits
Exhibit 1: Measures Should Provide Information on Inputs, Processes, Outputs, Impacts, Preconditions, and Efficiency .................................................................... 9 Exhibit 2: Housing’s Program Focus Has Shifted With Available Funding ....... 15 Exhibit 3: The Number of Measures for Each Core Service Do Not Correspond to Funding or Activity .............................................................................................. 16 Exhibit 4: Measures Relating to Affordable Housing Portfolio Provide Limited Useful Information ................................................................................................... 17 Exhibit 5: Few Housing Measures Overlap Between Reports ............................. 18 Exhibit 6: Reports Have Similar, But Slightly Different Measures ..................... 19 Exhibit 7: Illustration of a Simplified Logic Model That Can Put Performance Measures in Context ................................................................................................ 21

7 Background
The mission of the Housing Department (Housing) is to “strengthen and revitalize our community through housing and neighborhood investment.” Housing has five core services:
 Homeless interventions and solutions,
 Affordable housing portfolio management,
 Affordable housing production and preservation,
 Neighborhood capital investment and public services, and  Rent stabilization and tenant protection.

In addition to these core services, Hous ing has a strategic support service, which serves in administrative capacity and co nducts policy planning and development.

The strategic support service also provid es for the coordination and delivery of emergency services and recovery activiti es related to the pandemic or other emergencies.

The Housing Department Supports Homeless Prevention and Affordable Housing Development in the City
Housing and homelessness have been a maj or concern in San José, and the City Council has prioritized creating affordable housing and preventi ng homelessness.

On average, in 2022, an apartment cost over $2,700 per month to rent in San José and the median home sold for nearly $1.6 million. In 2023, it was estimated that nearly 6,300 residents did not have homes to live in.1
The City’s efforts to address homelessness and to create affordable housing have consistently been among the lowest rated City services by residents. In 2022, only 3 percent of surveyed residents rated the City’s efforts to address homelessness as “excellent” or “good” and only 10 percent rated efforts to facilitate the creation of affordable housing as “excellent or “good.”2

1 This figure is based on the results of the 2023 Point-in-Time Count conducted by the County of Santa Clara and the City of San José. See https://www.sanjoseca.gov/your-government/departments-offices/housing/resource- library/homeless-reports/homeless-reports-executive-summary. 2 https://www.sanjoseca.gov/home/showpublisheddocument/93470/638088822817770000 Audit of Housing Performance Measures
8 Through its various core services, Hous ing supports the City’s work in these areas.3 The City has identified short- and long-term goals to address homelessness and create affordable housing.
 The City Manager’s FY 2023-24 Budget Message describes a “path to achieving the goal of moving 1,000 people by December 31, 2023 into newly created City-funded safe alternatives for people who are experiencing homelessness and are uns heltered.” The path includes assisting individuals th rough permanent supportive housing, emergency interim housing, and other opportunities.
 The Housing Element, supported by both Housing and the Department of Planning, Building and Code Enforce ment (PBCE), sets market-rate and affordable housing goals toward the Regional Housing Needs Allocation (RHNA) set by the Association of Bay Area Governments. For the upcoming cycle (2023-31), the City’s RHNA is 62,200 new housing units, 23,775 of which are to be affordable units.
4  The Annual Action Plan describes pe rformance targets for how the City will use funding from the U.S. Department of Housing and Urban Development (HUD) to meet its five-ye ar goals. The five-year goals are outlined in a Consolidated Plan that is required to qualify for funding from HUD. For example, among other items, the 2022-23 Annual Action Plan specifies goals for developing at least 50 new affordable rental apartments with HOME Investment Partnership (HOME) program funding and providing homeless prevention assistance to 30 households via Emergency Solutions Grants (ESG). In additi on to the Annual Action Plan, annual results are captured in another report, the Consolidated Annual Performance Evaluation Report.

Performance Measures Serve Multiple Purposes
City departments, including the Ho using Department, publish performance measures in the City’s Annual Operatin g Budget and other reports. Housing reports out performance measures in its Annual Impact Report, reports to HUD, the California Department of Housing and Community Development (HCD) and the Governor’s Office of Planning and Research, and status reports to City

3 In addition to Housing, other City departments provide services directly to the city’s unhoused residents, such as trash pickup at homeless encampments. Other generally available services, such as Library se rvices or Fire Department emergency response services, are also provided to the city’s homeless residents. Lastly other City functions may be impacted by homelessness in the community, such as addressing water quality impacts from homeless encampments.

This report only reviews performance measures related to Housing’s services. 4 15,088 of the allocation are in the very low-income category (less than 50% of the Area Median Income) and 8,687 are in the low-income category (50-80% of the Area Median Income).

Background
9 Council, as well as through online dashboards and other reports. Many of these are found on the Housing Department’s website in their resource library.5 Performance Reporting vs. Performance Management

Departments report measures for different reasons. In some cases, performance reports provide measures to granting agencies or decision-makers for compliance or accountability purposes. They can show program results and progress toward goals.

Measures can also provide information about the levels of resources dedicated to a program (inputs), the amount of work performed (outputs), or levels of efficiency or effectiveness. Measures can also provide context for additional factors related to a program (these are referred to as “preconditions” and can relate to such things as the cost of living or other environmental factors).

Exhibit 1: Measures Should Provide Information on Inputs, Processes, Outputs, Impacts, Preconditions, and Efficiency
Types of measures
Inputs The money, personnel, equipment, and othe r resources that a program requires to operate. Frequently these are in a budget.

Processes
The administrative, organizational, and political activities that transform inputs into policy outputs and impacts. Process measures may include efficiency or timeliness type measures.

Outputs The direct outcome of a policy program or process. These are often unit counts, and represent the goods or services received by target groups.

Impacts The broader changes in behaviors or attitudes that result from policy outputs. Impact measures may include measures of a program's effectiveness.

Preconditions
The non-manipulable variables present prior to policy implementation. Preconditions provide contextual factors that affect a policy or process. For example, the cost of living provides context around programs related to homelessness.

Source: Adapted from William N. Dunn, Public Policy Analysis 5th Edition (2012), 248-253; Ammons, David N., Performance Measurement in Local Government: Developing Measures for Managing Your Operation

While public reporting helps to suppo rt transparency, accountability, and oversight, at their most effective, perf ormance measures also support decision- making to improve services. The use of performance information for management and policy decisions that improve service quality, efficiency, and program results is referred to as performance management.

5 https://www.sanjoseca.gov/your-government/departments-offices/housing/resource-library Audit of Housing Performance Measures
10 The Administration Has Signaled th at Performance Management Is a Priority for the City
The City has been working to improve its use of performance measures for over a decade. Recently, the Administrati on and Mayor have renewed focus on performance management in the City.
 In 2022, the City Council approved an initiative with a goal of “transforming the City’s current processes, culture, and infrastructure for managing and measuring the City’s performance.” This work is part of the Administration’s Outcomes, Equity Indicators, and Performance Management initiative.6 This work, led by the City Manager’s Office, is ongoing. At the time of the audit, staff had reviewed and made changes to the performance measures within the Neighborhood Se rvices City Service Area (CSA), updating the mission and strategic go als of the CSA and incorporating equity and outcome measures. They also reviewed and made changes to performance measures reported by departments within the Neighborhood CSA (e.g., Parks, Recr eation and Neighborhood Services, Library).
 In 2023, five Mayoral transition committees created performance measures for pressing community issues, including homelessness. 7 The
2023-24 Operating Budget incorporates these community issues as “focus areas” for the City. For these issue areas, the Administration is developing scorecards and dashboards, and a reporting cadence for City Council deliberation on progress. The Administ ration is also establishing team structures for Deputy City Managers, department directors, and senior and executive staff to continuously monitor performance in these areas.

The recent and ongoing improvements to the City’s structure and culture of performance management respond in part to concerns that current measures do not clearly relate to City priorities or goals, and that the number of measures is overwhelming.

The Administration is responsible for im plementing a culture and practice of performance management. The Outcomes, Equity Indicators, and Performance Management initiative and staff’s work to date signal that performance management, including equitable service deliv ery, are a top priority for the City.

In beginning its review of performance measures for the Operating Budget, the Administration has followed several best practices, including:
 seeking input from stakeholders on measures to track,  focusing CSA mission statements on expected benefits,

6 See https://www.sanjoseca.gov/home/showpublisheddocument/85735/637879053009170000. 7 The five focus areas identified as pressing community issues include: clean neighborhoods, community safety, downtown vibrancy, and planning and permitting.

Background
11  including broad impact indicators,
 using resident survey data on outcomes and experiences, and  including disaggregated data to assess equity impacts.

Prior City Auditor Reports
The City Auditor publishes an Annual Repo rt on City Services, the purpose of which is to provide an overview of City services and performance data to the public, the City Council, and City staff. 8 The report includes performance measures for all City departments as we ll as the results of an annual resident survey.

After the first Annual Report on City Services, in 2009 the City Manager and City Auditor released a whitepaper on performance management. 9 It found the City had many components of a performance management system but fell short in using data to inform decision-making. Specif ically, the report concluded “the City currently reports performance measures, but they are not consistently used as a tool to improve performance,” and that management did not routinely discuss measures for performance improvements. It also included various “next steps” that the Administration could take improve the City’s performance management system.

Since then, our office has released mult iple performance measure related audits, such as reviewing sewer cleaning performance and cost data (2010), and reviewing performance measures and methodologies for the Fire Department (2012), the Office of Economic Development (2013), and the Departments of Housing and Public Works (2015). For the 2015 Housing and Public Works audit, the objective was limited to reviewing the methodologies for calculating reported performance measures.

8 https://www.sanjoseca.gov/your-government/appointees/city-auditor/services-report 9 https://www.sanjoseca.gov/Home/ShowDocument?id=33716
Audit of Housing Performance Measures
12 This page was intentionally left blank

13 Finding I The Housing Department Can
Streamline and Improve Current
Performance Measures
Summary
Housing reports more than two hundred performance measures in different forms across various reports and online dashboards. The City’s Adopted Operating Budget, which has been including performance metrics for many years, can provide more useful and meaningful measures that better reflect department priorities. Performance measures also do not align across the various reports and dashboards, which can both lead to confusion for users of the information and additional work for Housing staff who compile and report the measures.

Using a logic model or similar approach to identify the most useful and meaningful measures, we recommend the City Man ager’s Office work with the Housing Department to reduce and align perfor mance measures acro ss the different reporting platforms. Housing should also document the methodology for calculating measures to ensure consistency over time. Lastly, the inclusion of targets can provide context for results and insight into incremental progress toward larger goals.

The Housing Department Reports More than Two Hundred Performance Measures Housing reports more than two hundred performance
measures, which mostly do not align across the
Department’s different reporting platforms (i.e., reports or online dashboards).
10 As an example, in reviewing the FY
2022-23 Adopted Operating Budget, Housing’s Annual
Impact Report, other state-required reports, and online dashboards, there were:
 47 measures relating to homelessness,
11 and
 143 measures relating to affordable housing development.

Additional performance measures rela te to other Housing services. (See Appendix B for a list of performance measures)

10 Total performance measure figures are an estimate. Audi t staff reviewed what we considered primary reporting platforms, but not all documents such as Council memos or all program-specific reports. Performance measures were reported in different ways (e.g., tables, text, graphics) and audit staff used judgment in determining how best to classify reported measures.
11 This does not include measures in the Consolidated A nnual Performance Evaluation Report (CAPER) prepared for HUD, since those are specific to individual programs and grantees. “Public officials, such as
legislators and legislative
staff, are short on time. If
presented with hundreds of
measures to wade through,
they’re likely to ignore
them all.”
–The Urban Institute
Audit of Housing Performance Measures
14 Many of these measures are useful and pr ovide valuable information. Some of these are also required because of the source of funding or other requirements.

However, the scope of reported measures can be confusing for decision makers or other users of the information. It also takes staff time to compile and report the measures.

The Operating Budget Performance Measures Can Be More Useful and Better Reflect Department Priorities
Housing reports performance measures for its five core services in the City’s Adopted Operating Budget. Because of changes in services over time, the measures may not reflect Housing’s current priorities. In addition, they should also be reviewed to ensure they cont inue to provide useful information for decision makers.

The Scope of Reported Performance Measur es Does Not Align With Spending or Priorities
Housing’s operating budget increased substantially over the past six years, with some programs or services growing fa r more than others. The total 2022-23 adopted budget was over five times more than 2018-19 actual expenditures and 65 percent more than 2021-22 actual expenditures, with major increases in spending for Homelessness Intervention s and Solutions and Strategic Support. 12 Smaller, but still notable increases in the operating budget went toward Affordable Housing production and Neighborhood Capital Investment and Public Services.

12 Strategic Support is tasked with coordination of emergency services and recovery activities. Much of the growth in spending and budget was for pandemic-related funding and programs. Funds for Housing increased based on grants related to COVID-19, but in the past two years, the fundin g from the General Fund has also increased. In 2022-23, General Funding for Housing increased to $57.4 million, a nearly seven-fold increase from 2021-22.

Finding 1
15 Exhibit 2: Housing’s Program Focus Has Shifted With Available Funding

Source: Auditor analysis of actual expenditures and adopted budgets.

Note: 2021-22 shows actual expenditures as reported in the Services Report. 2022-23 shows adopted budget values. All other years show actual expenditures as reported in subsequent budgets.

As Housing’s program delivery has ch anged in accordance with funding opportunities, the scope of performance measures has generally remained the same and may not reflect current priorities. For example, the Adopted Operating Budget has 20 measures relating to th e affordable housing loan portfolio.13 Staff report that because there have been no new loans coming along, this is not a program that is actively managed anymor e. Exhibit 3 shows the number of performance measures relating to th is program in the Operating Budget compared to other Housing programs. Although the number of measures does not need to be proportional to fundin g, Exhibit 3 does illustrate how measures should be reviewed periodically to ensure they reflect a department’s priorities.

13 The 20 measures include sub-parts. For example, the “monetary default rate of loan portfolio” measure includes the percent of total loan principal for project loans, rehabilitation loans, and homebuyer loans as line items (see Exhibit 4).

Affordable Housing
Portfolio Management
Affordable Housing
Production and
Preservation
Homelessness
Interventions and
Solutions
Neighborhood Capital
Investment and Public
Services
Rent Stabilization and
Tenant Protection
Strategic Support
$
$10
$20
$30
$40
$50
$60
$70
$80
$90
2017-18 2018-19 2019-20 2020-21 2021-22* 2022-23**
$millions
Audit of Housing Performance Measures
16 Exhibit 3: The Number of Measures for Each Core Service Do Not Correspond to Funding or Activity

Source: Auditor analysis of performance measures in the Housing chapters of adopted budgets.

Note: There are additional performance measures related to housing development in the Community and Economic Development City Service Area section of the budgets that are not reflected in Exhibit 3.

Measures in the Operating Budget Can be Improved to Provide More Useful Information In addition to the scope of reporting, some measures may provide limited utility.

Defaults are exceedingly rare for the A ffordable Housing Portfolio Management core service. However, there are multiple measures reported around defaults as seen in Exhibit 4. The reported perf ormance has been zero defaults since FY 2017-18 when 1 percent of total homebuyer loans were in default.

The second measure shown, percentage of portfolio units brought into compliance with safe and sanitary condition requirements within 90 days, also appears to provide limited utility. Starting in FY 2019-20, the results have been reported as 100 percent compliance, which could signify that the targets should be reviewed, or some other adjustme nt made to provide more meaningful information.

Affordable Housing
Portfolio
Management
Homelessness
Interventions and
Solutions
Affordable
Housing
Production
and
Preservation Neighborhood
Capital Investment
and Public Services
Rent Stabilization and
Tenant Protection
Finding 1
17 Exhibit 4: Measures Relating to Affordable Housing Portfolio Provide Limited Useful Information

Source: 2022-23 Adopted Operating Budget.

The Operating Budget’s Performance Measures Have Not Kept Up with Changes in Services.

Staff report that rapid rehousing has been declining in importance and has not lately been a primary home lessness intervention service. Rather, emergency interim housing has been a priority. Ho wever, the Operating Budget has three measures within the Operating Budget re lated to rapid rehousing but does not have any measures on emergency interim housing. The Department reports out on those in an online homelessness dashboard and a state-required report. As a result, reviewing measures wi thin the Budget may give a less useful or partial picture of performance.

Measures within the Operating Budget should balance consistency and relevance to provide the most value to users. Consistency in performance measures enables review of performance and progre ss over time. Relevance ensures that reported measures reflect current operations and priorities.

Analyzing performance over time can prov ide insight into different factors that affect performance and inform potential changes to service delivery. However, management and administrative staff report the performance measures within the budget do not help to inform their deci sion-making. Rather, staff treat these measures as a means of public reporting more than for performance management.

Housing Can Better Align Performance Measures Across Reports Measures in Housing’s Annual Impact Report 14 have almost no overlap with measures in the Operating Budget. Ju st one measure overlaps between the

14 https://www.sanjoseca.gov/home/showpublisheddocument/94461/638120496770570000

Audit of Housing Performance Measures
18 Impact Report and Budget (See Exhibit 5). No measures overlap between the homelessness dashboard and the Budget. According to the Administration, this reflects the challenge in keeping up with rapid changes in the delivery of services that has occurred in recent years. It also reflects the ongoing demand for performance data from funders, policymakers, and the public.

Exhibit 5: Few Housing Measures Overlap Between Reports

Source: Auditor analysis of the FY 2022-23 Adopted Operating Budget, FY 2021-22 Annual Impact Report, and Homelessness Dashboard. All measures refer to performance in FY 2021-22.

Variation Across Reports Can Add Value, But Also Create Confusion Having different measures within different reports can add value in that it allows for different levels of detail for different uses and audiences. In some, but not all instances, Housing is required to repo rt certain measures as a condition of funding. Some measures, however, vary only slightly across reports and may be calculated by a different basis or timeframe. (See examples in Exhibit 6.) This can create confusion, making performance evaluation and oversight more difficult.

Finding 1
19 Exhibit 6: Reports Have Similar, But Slightly Different Measures Report Operating Budget
(FY 2021-22 Actuals)
Homelessness Dashboard
(FY 2021-22)
Annual Impact Report
(FY 2021-22)
Measure # of homeless individuals (affiliated
with San José) who secured new
permanent housing
- chronically
- non-chronically
Individuals from San José
served in permanent
supportive housing

# individuals placed in
permanent housing
Reported
Value
Chronically homeless- 819
Non-chronically homeless - 1,038 2,510 1,857

Report AB2176 Annual Report
(Calendar Year 2021)
Homelessness
Dashboard (FY 2021-22)
Services Report
(FY 2021-22)
Annual Impact Report
(FY 2021-22)
Measure # of program participants
housed in every BHC (and
EIH) (Calendar Year 2021)
# of individuals served
through interim housing
(BHCs and EIHs)

# of individuals served
through interim housing
(BHCs and EIHs)

# of individuals moved
into interim housing
Value 677 759 742 742
Source: Auditor analysis of the FY 2022-23 Adopted Operating Budg et, FY 2021-22 Annual Impact Report, Homelessness Dashboard, a nd the City Auditor’s Annual Report on City Services
Note: The Services Report refers to the City Auditor’s Annual Report on City Services. The data for that report is provided by Housing, and Housing reviews and provides feedback on the report prior to publication by the City Auditor’s Office. According to Housin g, the variance between the # of individuals served through interim ho using reported on the Homelessness Dashboard and the other repor ts is because of a methodological error in how data was compiled.

Documenting Methodologies Can Better Ensure Consistency and Reliability of Performance Measures Over Time
Changing the calculation method or definition of a measure also makes it difficult to understand the extent to which a difference is attributable to actual change or methodological change.

In 2013, the Budget Office began requiring departments to submit methodology sheets for all performance measures in the operating budget to document measures’ meaning, data source, and ca lculation. These methodology sheets helped to ensure the accuracy and co nsistency of performance data. The methodology sheet templates also provided criteria to support the selection of meaningful, useful, and sustainable perfor mance measures. The Budget Office’s goal was to have a complete set of methodology sheets for all performance measures by August 2013.

Audit of Housing Performance Measures
20 Neither Housing nor the Budget Office have current methodology sheets for the Department’s performance measures. As a result, there has been some loss of institutional knowledge on methods of ca lculating measures. Additionally, there have been inconsistent measure calcul ations between reports. For example, because of differences in how the units were counted, the number of affordable housing units developed with City assistance in FY 2020-21 varied between the Adopted Operating Budget (216 units) and data provided to the City Auditor’s Office for the Annual Report on Services (0 units).

Documented methodologies are partic ularly important for performance measures that use nuanced terms or understanding. For example, the affordable housing units developed with City assistance is based on the number of units with certificates of occupancy issued with in the year, less any units for building managers. Without knowing this, staff may count the number of units based on construction completion (before occupancy is allowed), or some other criteria.

Logic Models Can Provide a Framework for Continuous Learning and Assessment
Housing reports that their performance measures are mostly used for public reporting purposes. While public repo rting helps to support transparency, accountability, and oversight, at their most effective, performance measures also support decision-making to improve the e ffectiveness, efficiency, and equity of public services. This requires developing or identifying performance measures in a way that relates program inputs to processes, targets, and overall goals.

Modeling Helps to Consider External Factors and Support Decision-Making In some instances, it is difficult to understand whether a measure indicates a program’s precondition – contextual fact ors that affect resource needs – or impact. For example, a frequent performance measure
for Housing is the estimated number of people
experiencing homelessness. In isolation, this measure
gives a sense for the extent of the problem to be
addressed. However, it may not give a sense of impact
unless presented relative to historical data or a target.

Considering precondition measures may help facilitate
better understanding of changes in homelessness which
in turn, can help make decisions about the needs for
additional resources, or a different approach.

“If the logic model is your
program’s to-do list, then you
need it to also be your staff’s
to-do list. And the process of
updating your logic model at
least annually forces agencies
to assess whether they are
satisfied with their results and
whether they should try new
strategies.”
– What Gets Measured, Gets
Done: A Toolkit on
Performance Measurement for
Ending Homelessness
Finding 1
21 One tool that can be used to help identify useful performance measures are logic models.15 Logic models clarify the relati onship between a program’s inputs, processes, outputs, and impacts in a wa y that facilitates decision-making at different levels. According to What Gets Measured, Gets Done: A Toolkit on Performance Measurement for Ending Homelessness, program staff should complete logic models to match goals with activities. Exhibit 7 shows a sample logic model developed by audit staff as an illustration.

Exhibit 7: Illustration of a Simplified Logic Model That Can Put Performance Measures in Context

Source: Auditor adaptation of performance measurement framework for a prevention program, using Dunn’s general framework for monitoring. Spellman and Abbenante, What Gets Measured, Gets Done:

A Toolkit on Performance Measurement for Ending Homelessness (2008). William N. Dunn, Public Policy Analysis 5
th Edition (2012), 254.

15 “A [logic] model is a schematic diagram that shows how one stage of a program’s output affects the next and outlines the manager’s ‘theory of change’ by identifying program factors thought to influence short-term or immediate outcomes, which in turn presumably will influence the long-term or end outcomes they are pursuing.” Hatry, Harry P. 2014. Transforming Performance Measurement for the 21st Century. Washington D.C. Urban Institute

Audit of Housing Performance Measures
22 Consistency and Focus Will Make Housing’s Performance Measures More Usable, Meaningful
As the Administration continues to review performance
measures as part of the Outcomes, Equity Indicators, and Performance Management initiative, the City Manager’s Office should work with Housing to id entify measures that are most useful for decision makers. Housing should also look to reduce and align measures more closely across reports to the extent possible and ensure measures ar e calculated consistently.

Consistency in this way will e liminate any confusion. This would also benefit Housing staff who currently must gather and report these data.

Recommendations:

1: In conjunction with the Administration’s initiative to revise performance measures in the Operating Budget, and
implement the City Focus Areas, the City Manager’s Office and Housing Department should, using logic models or some other means:
a. Revise the Housing Department’s performance
measures in the City’s Operating Budget to reflect
better indicators of performance for the Department
and
b. Reduce and align the number of performance
measures across the City’s Operating Budget and
other Housing-related performance reports or
dashboards (to the extent allowable based on funding
sources).
2: To ensure consistency and reliability of performance measures, the Housing Department should document
methodology for measures in the Operating Budget and other year-over-year reports and dashboards.

Clear and Meaningful Targets Can Provide Insight into Incremental Progress Toward Larger Goals
Targets are useful to provide a clear id ea of what a program is expected to accomplish and how well a program is perf orming in meeting its goals. Often, goals may be broad and too big to accomp lish in one year, so more specific or incremental targets for a given year ca n help understand progress toward those goals.
“Considerations around the
number of measures to report
and the frequency of reporting
them should begin by thinking
about the purpose or purposes
of the report and its audience’s
needs.”
– Performance Measurement
for Managing Local
Government: Getting it Right
Finding 1
23 Reporting the broad goals or targets is still important even when goals may go beyond a program’s control. This is the case for Housing’s role to address homelessness and affordable housing deve lopment. Housing’s programs were established to address these broader community concerns, and it is important to clarify the degree of influence of Housing’s activities in these areas.

Better Communication of Targets Can Help Decision Makers The homelessness and affordable housin g development performance measures within the Adopted Operating Budget and Annual Impact Report do not present incremental progress towards the long-term goals. For example, while there are measures relating to the estimated number of people experiencing homelessness and the number of affordable housing units developed with City assistance, there is not context relating those measures to the broader goals outlined in the regional Community Plan to End Homelessness or Housing Element.

The Community Plan to End Homelessness sets several Countywide goals, including reducing the number of new unhoused individuals and families in a given year by 30 percent. The current target in the operating budget for the reduction in the number of people experiencing homelessness from the prior two years is 200 (or about 3 percent of the estimated num ber of homeless individuals). It is not apparent to a reader where this figure comes from, or how it relates to the broader Countywide goals, nor what it means in the context of homelessness in general.

The 2014-2023 Housing Element set an affordable housing goal of 14,661 units by 2022, which was not meet.16 While City-supported affordable housing makes up only part of this goal (some affordable developments move forward without City support), having an interim target tied to the larger goal could help decision- makers identify necessary resources to maintain incremental progress.

Currently, staff set the target for th e number of affordable housing units developed with City assistance primarily based on the number of units in the development pipeline. As a result, it generally reflects what is currently occurring rather than a broader, incremental go al. The FY 2023-24 Proposed Operating Budget forecast 342 new affordable housing units.

Recommendation

3: The Administration and the Housing Department should identify and report meaningful targets to provide context for reported results for key performance measures in its different reporting platforms.

16 The upcoming Housing Element proposed a 62,000-unit Regional Housing Needs Allocation (RHNA), which would further increase the overall affordable housing goal.

Audit of Housing Performance Measures
24 This page was intentionally left blank

25 Conclusion
Housing reports progress and activities of its programs in different ways, including performance measures in the City’s Adopted Operating Budgets, its Annual Impact reports, online dashboards, and others. We found that Housing can better report results of its programs by streamlinin g and improving its current performance measures. The Department currently re ports more than two hundred measures across different platforms. More alignm ent across the different platforms would help decision makers better understand results and save staff time currently tracking and reporting these measures. Housing can also better document the methodologies for calculating measures to ensure consistency over time and across reports and report meaningful targets to provide context for results.

RECOMMENDATION

Finding 1:

Recommendation #1: In conjunction with the Admini stration’s initiative to revise performance measures in the Operating Budget, and implement the City Focus Areas, the Administration and Housing Department should, using logic models or some other means:
a) Revise the Housing Department’s performance measures in the City’s Operating Budget to reflect better indicators of performance for the Department, and b) Reduce and align the number of performance measures across the City’s Operating Budget and other Housing-related performance report s or dashboards (to the extent allowable based on funding sources).

Recommendation #2: To ensure consistency and reliability of performance measures, the Housing Department should document methodology for measures in the Operating Budget and other year- over-year reports and dashboards.

Recommendation #3: The Administration and the Housing Department should identify and report meaningful targets to provide context for reported results for key performance measures in its different reporting platforms.

Audit of Housing Performance Measures
26 This page was intentionally left blank

APPENDIX A

Audit Objective, Scope, and Methodology
A-1
The mission of the City Auditor’s Office is to inde pendently assess and report on City operations and services. The audit function is an essential elemen t of San José’s public accountability, and our audits provide the City Council, City management, and the general public with independent and objective information regarding the economy, efficiency, effectiveness, and equity of City operations and services.

In accordance with the City Auditor’s Fiscal Year (FY) 2023-24 Audit Work Plan, we have completed an audit of Housing Department performance measures. The audit is expected to be the first in a series of recurring projects to assess the validity and useful ness of performance measures by City departments, programs, or City Service Areas. This work is in tended to be ongoing to augment the City Auditor’s Annual Services Report and assist the Administrati on to enhance the City’s performance management systems in alignment with the Outcome, Equity Indicators, and Performance Management initiative.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan an d perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findin gs and conclusions based on our audit objectives.

We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

The objective of this audit was to assess the valid ity and usefulness of performance measures by the Housing Department, with a focus on measures re lating to homelessness and affordable housing production. Though this audit was focused on Housing’s services and performance metrics, other departments may serve the unhoused community or have programs impacted by homelessness. We did not include those departments in this review. We sought to understand the relevant management controls over the Housing Department’s performance measures. To meet our audit objectives, we did the following:
 Reviewed Department performance reports, as well as performance reports and dashboards pertaining to homelessness and affordable housing to understand the scope and alignment of reported measures;
 Interviewed staff in the Housing Department to understand how staff calculate and use performance measures and targets relating to homelessness and affordable housing, as well as issues related to source data and data reliability;
 Interviewed staff in the City Manager’s Office to understand ongoing work in the Outcomes, Equity Indicators, and Performance Management initiative and Focus Ar eas resulting from the Mayoral Transition Committee on Homelessness;
 Reviewed available methodology sheets for performance measures in the Operating Budget;  Researched best practices in performance measurement, evaluation, and management, including resources specific to homelessness and affordable housing; and  Benchmarked performance measures relating to homelessness and affordable housing in San Francisco, Oakland, San Diego, Santa Clara County, as well as Phoenix, Arizona; Austin, Texas; Seattle, Washington; and Boulder, Colorado.

We would like to thank the Housing Department and City Manager’s Office for their time and insight during the audit process.

A-2
This page was intentionally left blank

APPENDIX B

Partial Listing of Housing Department Performance Measures From Departmentwide Reports and Reports and Dashboards Focused on Affordable Housing and Homelessness

B-1
Measure Source Service
Project loans monetary default rate of loan portfolio by % of total loan principal Budge t Residential Developmen t Rehabilitation loans monetary default rate of loan portfolio by % of total loan principal Budge t Residential Developmen t Homebuyer loans monetary default rate of loan portfolio by % of total loan principal Budge t Residential Developmen t Project loans monetary default rate of loan portfolio by % of total loans Budge t Residential Developmen t Rehabilitation loans monetary default rate of loan portfolio by % of total loans Budge t Residential Developmen t Homebuyer loans monetary default rate of loan portfolio by % of total loans Budge t Residential Developmen t % of portfolio units brought into compliance with safe and sanitary condition requirements within 90 days Budge t Residential Developmen t # of single family loan management transactions (refinances, subordinations, assumptions, payoffs) Budge t Residential Developmen t Total project loan principal Budget Residential Developmen t Total rehabilitation loan principal Budget Residential Developmen t Total homebuyer loan principal Budget Residential Developmen t Total number of project loans Budget Residential Developmen t Total number of rehabilitation loans Budget Residential Developmen t Total number of homebuyer loans Budget Residential Developmen t Total Housing Department loan principal Budget Residential Developmen t Total number of loans Budget Residential Developmen t
# of major projects in loan portfolio inspected annually Budge t Residential Developmen t # of units in loan portfolio inspected annually Budget Residential Developmen t # of City facilitated affordable rental units Budget Residential Developmen t # of income restricted for sale homes Budget Residential Developmen t Cumulative ratio of non-City funds to City funds over the last five years in the New Construction Program Budge t Residential Developmen t % of annual target achieved for production of affordable housing Budge t Residential Developmen t # of units of affordable housing produced [with City assistance] Budge t Residential Developmen t % of funding committed to extremely low-income households Budge t Residential Developmen t # of affordable housing units completed in the fiscal year [with City assistance] Budge t Residential Developmen t Average per unit subsidy in funding commitments for new construction projects Budge t Residential Developmen t Reduction in the number of homeless individuals from prior two years Budge t Homeless Solutions

B-2
Measure Source Service
% of target achieved for completion of affordable housing units Budget (CSA) Residential Developmen t % of City's 10,000 unit affordable target with funding commitment (pending construction) Budget (CSA) Residential Developmen t % of City's 10,000 unit affordable target under construction Budget (CSA) Residential Developmen t % of City's 10,000 unit affordable target completed Budget (CSA) Residential Developmen t % of Housing Department funds reserved by income levels over 5 years: Extremely Low (<=30% of median) Budget (CSA) Residential Development % of Housing Department funds reserved by income levels over 5 years: Very Low (31-50% of median) Budget (CSA) Residential Deve lopment % of Housing Department funds reserved by income levels over 5 years: Low (51-80% of median) Budget (CSA) Residential Developme nt % of Housing Department funds reserved by income levels over 5 years: Moderate (81-120% of median) Budget (CSA) Residential Dev elopment % of households who exit the Rapid Rehousing Program into permanent housing that maintain housing for at least 6 months Budge t Homeless Solutions Average number of days from enrollment households in rapid rehousing programs got into permanent housing Budge t Homeless Solutions # of households who maintained permanent housing in City-funded rapid rehousing programs in the last 12 months Budge t Homeless Solutions Estimated number of chronically homeless individuals counted in San José Budge t Homeless Solutions Estimated number of non-chronically homeless individuals counted in San José Budge t Homeless Solutions # of chronically homeless individuals (affiliated with San José) who secured new permanent housing Budge t Homeless Solutions # of non-chronically homeless individuals (affiliated with San José) who secured new permanent housing Budge t Homeless Solutions # of homeless individuals assessed for permanent housing Budge t Homeless Solutions % of CDBG-funded City projects meeting all stated outcomes Budge t Grants Managemen t % of CDBG-funded non-City projects meeting all stated outcomes Budge t Grants Managemen t % of CDBG invoices processed with 30 days of receipt of all required documentation Budge t Grants Managemen t % of CDBG contracts executed by July 1 Budget Grants Managemen t # of rehabilitation projects completed Budget Grants Managemen t # of minor repair rehabilitation projects completed Budge t Grants Managemen t Total # of rehabilitation and minor repair projects completed Budge t Grants Managemen t % of tenant/landlord mediations that resulted in voluntary agreement Budge t Rent Stabilization % of cases resolved within 60 days of completed petition date Budge t Rent Stabilization # of unduplicated mobile home and apartment clients served by the Rent Stabilization Program Budge t Rent Stabilization Average number of days from completed petition to resolution Budge t Rent Stabilization # of individuals and families served through interim housing (BHCs and EIHs) Annual Impact Report Homele ss Solutions Rental aid facilitated via Eviction Help Center Annual Impact Report Rent Stabilization Formerly homeless people placed in permanent or interim housing by Homelessness Response Team Annual Impact Report Homeless Sol utions

B-3
Measure Source Service
# of formerly homeless people moved into interim housing Annual Impact Report Homeless Solutions # of development projects underway supported by Residential Development Division Annual Impact Report Residential Developmen t
# of apartments underway via Residential Development Division Annual Impact Report Residential Developmen t # of units of affordable housing underway via Residential Development Division Annual Impact Report Residential Developmen t Grant funding disbursed to provide necessities to low-income residents Annual Impact Report Grants Managemen t # of low-income residents receiving grant-funded necessities Annual Impact Report Grants Managemen t # of construction projects at interim housing locations managed by housing inspectors Annual Impact Report Housing Inspection # families with young children placed in hotels and motels while looking for stable housing Annual Im pact Report Homeless Solut ions # of individuals served by the motel voucher program (including children) Annual Impact Report Homeless Solutions # of children served by the motel voucher program Annual Impact Report Homeless Solutions # of San José residents prevented from becoming homeless via Homeless Prevention System Annual Impact Report Homeless Solutions # individuals experiencing homelessness that participated in the City's Overnight Warming Location (OWL) program Annual Impact Report Homeless Solutions # of showers provided via Dignity on Wheels Annual Impact Report Homeless Solutions # of loads of laundry provided via Dignity on Wheels Annual Impact Report Homeless Solutions # of homeless individuals served via Dignity on Wheels Annual Impact Report Homeless Solutions Approximate # of residents impacted by Housing disbursed grants Annual Impact Report Grants Managemen t
# of programs supported by Housing disbursed grants Annual Impact Report Grants Managemen t # of organizations supported by Housing disbursed grants Annual Impact Report Grants Managemen t New funding committed by the Residential Development Division for affordable housing Annual Impact Report Residential Developme nt # of identified future residential projects that qualify for City funding Annual Impact Report Residential Developmen t # of affordable apartments possible via identified future residential projects that qualify for City funding Annual Impact Repo rt Residential Developmen t # of units of supportive housing possible via identified future residential projects that qualify for City funding Annual Impac t Report Residential Developmen t # of individuals placed in permanent housing Annual Impact Report Homeless Solutions # of medically vulnerable individuals placed in non-congregate shelters (pandemic-specific) Annual Impact Report Homeless Solut ions # of apartments with newly extended rent limits Annual Impact Report Residential Developmen t # of emergency shelters operated (pandemic-specific) Annual Impact Report Grants Managemen t # of existing units refinanced to keep them affordable Annual Impact Report Residential Developmen t # of low-income residents living in rent-stabilized apartments Annual Impact Report Residential Developmen t # of rent-stabilized apartments with property owners that submitted required information to the City Annual Impact Report Rent Stabiliza tion # of inquiries handled by the Rent Stabilization Program Annual Im pact Report Rent Stabilization

B-4
Measure Source Service
# of inquiries that came to Rent Stabilization Program legal services hotline Annual Impact Report Rent Stabilization # of beds in congregate shelters (pandemic-specific) Annual Impact Report Housing Inspection # of individuals served by City congregate shelters (pandemic-specific) Annual Impact Report Housing Inspection # of individuals and families served by City congregate shelters Annual Impact Report Ho using Inspection # of City congregate shelters (pandemic-specific) Annual Impact Report Housing Inspection # of inspections conducted of shelters, apartment buildings and affordable housing Annual Impact Report Housing Inspection # of agreements with service providers, grantees, and contractors managed by Housing administrative staff Annual Impact Report Administration # of Housing Department followers on Twitter and Facebook Annual Impact Report Communications Individuals sheltered in Emergency Shelter or Interim Housing Homelessness Dashboard Homeless Solutions Individuals served in Permanent or Rapid Rehousing Homelessness Dashboard Homeless Solutions # of households connected with rent relief funds to avoid eviction Homelessness Dashboard Homeless Solutions Individuals remaining housed 1 year after receiving aid through the homeless prevention system Homelessness Dashboard Homeless Solutions Individuals provided shelter (Count
y-funded shelters) Homelessness Dashboard Homeless Solutions % of households that leave County-funded shelters to permanent housing Homelessness Dashboard Homeless Solutions % of households that leave County-funded shelters to temporary accommodations Homelessness Dashboard Homeless Solutions % of households that leave County-funded shelters to return to homelessness Homelessness Dashboard Homeless Solutions % of individuals that leave OWL shelters to permanent housing Homelessness Dashboard Homeless Solutions % of individuals that leave OWL shelters to temporary accommodations Homelessness Dashboard Homeless Solutions % of individuals that leave OWL shelters to return to homelessness Homelessness Dashboard Homeless Solutions % of individuals that leave interim housing to permanent housing Homelessness Dashboard Homeless Solutions % of individuals that leave interim housing to temporary accommodations Ho melessness Dashboard Homeless Solutions % of individuals that leave interim housing to return to homelessness Ho melessness Dashboard Homeless Solutions Individuals from San José served in permanent supportive housing Homelessness Dashboard Homeless Solutions Individuals from San José served in rapid rehousing Homelessness Dashboard Homeless Solutions % of households remaining housed after one year moving into permanent supportive housing Homelessness Dashboard Homeless Soluti ons % of households remaining housed after one year moving into rapid rehousing Homelessness Dashboard Homeless Solutions # of unsheltered homeless Homelessness Focus Areas Homeless Solutions # of people housed in permanent housing solutions and interim housing solutions Homelessness Focus Areas Homeless Solutions # of people who return to unsheltered homelessness from interim and permanent housing Homelessn ess Focus Areas Homeless Solutio ns # of emergency interim units Homelessness Focus Areas Homeless Solutions

B-5
Measure Source Service
# of affordable housing units completed (since 2018) Affordable Housing Dashboard Residential Developmen t # of affordable housing units under construction Affordable Housing Dashboard Residential Developmen t # of affordable housing units in pipeline Affordable Housing Dashboard Residential Developmen t # of prospective affordable housing units Affordable Housing Dashboard Residential Developmen t # of preserved affordable housing units (since 2018) Affordable Housing Dashboard Residential Developmen t # of completed affordable housing units for people with disabilities Affordable Housing Da shboard Residential Developmen t # of completed affordable housing units for families Affordable Housing Dashboard Residential Developmen t # of completed affordable housing units for permanent supportive housing Affordable Housing Dashboard Residential Developmen t # of completed affordable housing units for rapid rehousing Affordable Housing Dashboard Residential Developmen t # of completed affordable housing units for seniors Affordable Housing Dashboard Residential Developmen t # of completed affordable housing units for veterans Affordable Housing Dashboard Residential Developmen t # of completed affordable housing units for youth Affordable Housing Dashboard Residen tial Development # of affordable housing units for people with disabilities Affordable Housing Dashboard Residential Developmen t # of affordable housing units in construction for families Affordable Housing Das hboard Residential Developmen t # of affordable housing units in construction for permanent supportive housing Affordable Housing Dashboard Residential Develo pment # of affordable housing units in construction for rapid rehousing Affordable Housing Dashboard Residential Developmen t # of affordable housing units in construction for seniors Affordable Housing Das hboard Residential Developmen t # of affordable housing units in construction for veterans Affordable Housing Dashboard Residential Developmen t # of affordable housing units in construction for youth Affordable Housing Dashboard Residential Developmen t # of affordable housing units in pipeline for people with disabilities Affordable Housing Dashboard Residential Developmen t # of affordable housing units in pipeline for families Affordable Housing Dashboard Residential Developmen t # of affordable housing units in pipeline for permanent supportive housing Affordable Housing Dashboard Residential Developmen t # of affordable housing units in pipeline for rapid rehousing Affordable Housing Das hboard Residential Developmen t # of affordable housing units in pipeline for seniors Affordable Housing Dashboard Residential Developmen t # of affordable housing units in pipeline for veterans Affordable Housing Dashboard Residential Developmen t # of affordable housing units in pipeline for youth Affordable Housing Dashboard Residential Developmen t # of preserved affordable housing units for people with disabilities Affordable Housing Dashboard Residential Developmen t # of preserved affordable housing units for families Affordable Housing Dashboard Residential Developmen t # of preserved affordable housing units for permanent supportive housing Affordable Housing Dashboard Residential Developmen t # of preserved affordable housing units for rapid rehousing Affordable Housing Dashboard Residential Developmen t

B-6
Measure Source Service
# of preserved affordable housing units for seniors Affordable Housing Dashboard Residential Developmen t # of preserved affordable housing units for veterans Affordable Housing Dashboard Residential Developmen t # of preserved affordable housing units for youth Affordable Housing Dashboard Residential Developmen t # of program participants housed in every BHC (and EIH) AB2176 Annual Repor t Homeless Solutions # of BHC (and EIH) program participants who have moved to permanent affordable housing AB2176 Annual Repor t Homeless Solutions Average time required for a BHC program participant to receive a permanent affordable housing unit AB2176 Annual Repor t Homeless Solutions Actual number of permanent affordable housing units available AB2176 Annual Repor t Homeless Solutions Projected number of permanent affordable housing units available AB2176 Annual Repor t Homeless Solutions # of proposed very low-income deed-restricted units Annual Housing Element Progress Repor t Residential Developmen t # of proposed very low-income non deed-restricted units Annual Housing Element Progress Repor t Residential Developmen t # of proposed low-income deed restricted units Annual Housing Element Progress Report Residential Developmen t # of proposed low-income non deed-restricted units Annual Housing Element Progress Repor t Residential Developmen t # of proposed moderate-income deed-restricted units Annual Housing Element Progress Repor t Residential Developmen t # of proposed moderate-income non deed-restricted units Annual Housing Element Progress Repor t Residential Developmen t # of proposed above moderate-income Annual Housing Element Progress Report Residential Developmen t Total # of proposed units within housing development applications submitted Annual Housing Element Progress Repor t Residential Developmen t # of approved units of housing development applications submitted Annual Housing Element Progress Repor t Residential Developmen t # of disapproved units of housing development applications submitted Annual Housing Element Progress Repor t Residential Developmen t # of very low-income deed restricted units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t # of very low-income deed-restricted units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t # of low-income deed-restricted units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t # of low-income non deed-restricted units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income deed-restricted units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income non deed-restricted units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t # of above moderate-income units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t Total # of units that completed entitlement Annual Housing Element Progress Report Residential Developmen t # of extremely low income units that completed entitlement Annual Housing Element Progress Repor t Residential Developmen t # of projects approved using SB 35 streamlining Annual Housing Element Progress Report Residential Developmen t # of demolished/destroyed units Annual Housing Element Progress Report Residential Developmen t # of very low-income deed-restricted units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t

B-7
Measure Source Service
# of very low-income non deed-restricted units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t # of low-income deed restricted units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t # of low-income non deed-restricted units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income deed-restricted units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income non deed-restricted units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t # of above moderate-income units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t Total # of units issued building permits Annual Housing Element Progress Report Residential Developmen t # of extremely low-income units with building permits issued Annual Housing Element Progress Repor t Residential Developmen t # of very low-income deed-restricted units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t # of very low-income non deed-restricted units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t # of low-income deed-restricted units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t # of low-income non deed-restricted units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income deed-restricted units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income non deed-restricted units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t # of above moderate-income units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t Total # of units issued certificates of occupancy Annual Housing Element Progress Report Residential Developmen t # of extremely low income units issued certificates of occupancy Annual Housing Element Progress Repor t Residential Developmen t Total # of very low-income units permitted to date Annual Housing Element Progress Repor t Residential Developmen t Total # of low-income units permitted to date Annual Housing Element Progress Report Residential Developmen t Total # of moderate-income units permitted to date Annual Housing Element Progress Repor t Residential Developmen t Total # of above moderate-income units permitted to date Annual Housing Element Progress Repor t Residential Developmen t Total # of extremely low-income units permitted to date Annual Housing Element Progress Repor t Residential Developmen t Total housing applications submitted Annual Housing Element Progress Report Residential Developmen t # of proposed units in all applications received Annual Housing Element Progress Report Residential Developmen t Total # of housing units approved Annual Housing Element Progress Report Residential Developmen t Total # of housing units disapproved Annual Housing Element Progress Report Residential Developmen t # of applications for streamlining under SB 35 Annual Housing Element Progress Report Residential Developmen t # of streamlining applications approved Annual Housing Element Progress Report Residential Developmen t Total # of developments approved with streamlining Annual Housing Element Progress Repor t Residential Developmen t Total # of units constructed with streamlining Annual Housing Element Progress Report Residential Developmen t

B-8
Measure Source Service
# of very low-income rental units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of low-income rental units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income rental units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of above moderate-income rental units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t Total # of rental units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of very low-income ownership units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of low-income ownership units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income ownership units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of above moderate-income ownership units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t Total # of ownership units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of very low-income units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of low-income units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of moderate-income units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t # of above moderate-income units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t Total # of units constructed with SB 35 streamlining permits Annual Housing Element Progress Repor t Residential Developmen t

Source: Auditor analysis of most recent Ad opted Operating Budgets, Annual Impact Reports, Housing Production Reports (Affordabl e Housing Dashboards), Homelessness Dashboards, and state-required reports on affordable housing (Annual Housing Element Progress Report and Housing Successor Report) and homelessness (Emergency Bridge Housing (AB 2176) Annual Report.

Note: We did not include the Homeless Response Team Annual Report, since it has been discontinued. We also did not include meas ures reported in the Consolidated Annual Performance Evaluation Report (CAPER) submitted to HUD since those measures are provider- and program-specific, or the Santa Clara County Continuum of Care Benchmarks as those are not City-specific. Additional reports and measures, outside the focus of this audit, may relate to services beyond from affordable housing and homelessness prevention and solutions.

TO:  JOE ROIS  FROM:  Rosalynn Hughey

CITY AUDITOR
SUBJECT:  See Below DATE:  September 11, 2023
_______________________________________________________________ ______________
Approved         Date
_____________________________________________________________________________ SUBJECT:   RESPONSE TO THE AUDIT OF HOUSING PERFORMANCE MEASURES: THE CITY SHOULD FOCUS AND ALIGN MEASURES TO

BETTER SUPPORT DECISION-MAKING

BACKGROUND
The Administration has reviewed the Audit of the Housing Department’s Housing Performance Measures and agrees with the recommendations identified in the report. The Administration’s responses to each of the City Auditor’s recommendations are provided in this response to the report.

Housing and homelessness are major concerns in San José, and the City Council has prioritized producing new affordable housing and reducing unsheltered homelessness. On average, in 2022, an apartment cost over $2,700 per month to rent in San José. An estimated 6,600 residents were homeless in the City on any given night during 2022.

The Housing Department supports the City’s work in these areas. The City has identified short- and long-term goals to address homelessness and create affordable housing. For example:
 The City Manager’s Fiscal Year 2023-2024 Budget Message describes a “path of achieving the goal of
moving 1,000 people by December 31, 2023, into newly-created, City-fund
ed safe alternatives for people who are experiencing homelessness and are unsheltered.
” The path includes assisting individuals through permanent supportive housing, emergency
interim housing, and other opportunities.
 The Housing Element, supported by the Housing Department and Department of Planning, B
uilding and Code Enforcement, sets market-rate and affordable housing goals toward the
Regional Housing Needs Allocation (RHNA) set by the Association of Bay 9/12/2023

JOE ROIS, CITY AUDITOR
September 11, 2023
Subject: Response to the Audit of Housing Performance Measures

Area Governments. For the cycle that runs from 2023 to 2031, the City’s RHNA goal is 62,200 new housing units.

Performance Reporting
City departments, including the Housing Department, publish performance measures in the City’s Annual Operating Budget and other reports. For example, the Housing Department reports performance measures in its Annual Impact Report and in reports to the federal government, the California Department of Housing and Community Development (HCD) and the Governor’s Office of Planning and Research. Performance measures are also provided in status reports to the City Council and through online dashboards. Many of these are found on the Housing Department’s website.

Measures can also provide information about the levels of resources dedicated to a program (inputs), the amount of work perfo
rmed (outputs), or levels of efficiency or effectiveness. Broad measures can also provide context for additional factors related to a program. These are referred to as “preconditions” and can relate to such things as the cost of living or other environmental factors.

RECOMMENDATIONS AND ADMINISTRATION’S RESPONSE
Finding I: The Housing Department Can Streamline and Improve Current Performance Measures
Recommendation #1: In conjunction with the Administration’s initiative to revise performance measures in the Operating Budget, and implement the City Focus Areas, the City Manager’s Office and Housing Department should, using logic models or some other means:
a. Revise the Housing Department’s performance measures in the City’s Operating Budget to reflect better indicators of performance for the Department. b. Reduce and align the number of performance measures across the City’s Operating Budget and other Housing-related performance reports or dashboards (to the extent allowable based on funding sources).

Administra
tion’s Response: The Administration agrees with this recommendation.

Green: As part of the City Service Area (CSA) performance modernization through the Outcomes, Community Indicators, and Performance Management Initiative, the Housing Department will revise its performance measures in the Operating Budget in coordination with the City Manager’s Office of Administration, Policy, and Intergovernmental Relations, the Budget Office, and the Office of Racial Equity. In doing so, the Housing Department will make its performance measures more reflective of current programs and make them better aligned with other recurring reports that leverage Housing Department performance measures. The Housing

JOE ROIS, CITY AUDITOR
September 11, 2023
Subject: Response to the Audit of Housing Performance Measures

Department will phase out performance measures that are no longer meaningful for decision- making. The performance modernization will utilize the CSA structure as a type of logic model that aligns measures around inputs (budget resources), activities (core services and budget programs), outputs (activity and workload highlights), processes (performance measures), and outcomes (community indicators).

Target Date for Completion: Spring 2024 as part of City Manager’s Proposed Operating Budget for Fiscal Year 2024-2025.

Recommendation #2. To ensure consistency and reliability of performance measures, the Housing Department should document methodology for measures in the Operating Budget and other year-over-year reports and dashboards.

Administration’s Response: The Administration agrees with this recommendation.

Green: As part of the City Service Area (CSA) performance modernization through the Outcomes, Community Indicators, and Performance Management Initiative, Housing Department staff will collect historical performance data and document the methods used for collecting and visualizing that data. This work will be done in coordination with the City Manager’s Office of Administration, Policy, and Intergovernmental Relations. This documentation will help the Department demonstrate accountability, share progress, and inform stakeholders in a manner that is both repeatable and reliable, which is essential to build trust and confidence in the Department’s reporting.

Target Date for Completion: July 2024

Recommendation #3. The City Manager’s Office and the Housing Department should identify and report meaningful targets to provide context for reported results for key performance measures in its different reporting platforms.

Administration’s Response: The Administration agrees with this recommendation.

Green: Following the revision of its performance measures in the Operating Budget and documenting the process by which Department performance metrics are collected, the Department will review those metrics for the entirety of Fiscal Year 2022-2023. Based on those results, which will give the Department a strong understanding of its performance in both producing new affordable housing and addressing the City’s homelessness crisis, the Department will establish and report on meaningful performance targets for Fiscal Year 2023-2024. This will be done in coordination with the City Manager’s Budget Office and Office of Racial Equity.

JOE ROIS, CITY AUDITOR
September 11, 2023
Subject: Response to the Audit of Housing Performance Measures

The intention is to give the Department a strong foundation for guiding service improvements, providing visibility into its challenges, and soliciting ideas in areas where they can have the greatest impact.

Target Date for Completion: August 2024

CONCLUSION
The Housing Department extends its gratitude to the City Auditor’s Office for dedicating their time to the completion of this audit. It has been a positive experience that yielded opportunities to improve how the Department serves the residents and elected officials of San José.

/s/

R O S A L Y N N H U G H E Y
D e p u t y C i t y M a n a g e r a n d
A c t i ng Director of Housing

For questions, please contact Rachel VanderVeen, Assistant Director of Housing, at [email protected] or 408-535-825