12100 Wilshire Boulevard, Suite 350 | Los Angeles, CA 90025 Phone: (310) 828-1183 | Fax: (310) 453-6562
1999 Harrison St., Suite 2020 | Oakland, CA 94612
Phone: (510) 451-9521 | Fax: (510) 451-0384
TO Leslie Caviglia, City Manger
City of Visalia
FROM Richard Bernard & Adam Sonenshein
FM3 Research
RE: City of Visalia Resident Satisfaction 2023 Survey Proposal DATE July 25, 2023
Fairbank, Maslin, Maullin, Metz & Associates (FM3) is pleased to submit this scope and cost proposal to conduct a 15- to 20-minute address-based Resident Satisfaction Survey among a random sample of 500 City of Visalia adult residents. This survey will be available in English and Spanish both online and by telephone. FM3 has had the privilege to conduct research on behalf of the City of Visalia on four separate occasions from 2003 to 2015.
Evidence of FM3’s ability to accurately measure the sentiments of your residents was last demonstrated over the November 2016 election cycle, where the results of your successful sales tax measure matched the anticipated yes vote in our 2015 survey, within the margin of error. Given our firm’s successful past work with the City of Visalia, we believe we are well suited to conduct the proposed Resident Satisfaction Survey.
The remainder of this proposal provides FM3’s Work Plan and Metho dology (including three phases: Research Design; Data Collection; and Data Analysis and Reporting); Approximate Project Timeline and Cost for the Proposed Research.
1 TECHNICAL APPROACH AND WORK PLAN/METHODOLOGY
1.1 Research Approach: Methodology and Work Plan
Although the final methodology, sample size, and length of the Resident Satisfaction Survey will be decided in consultation with the City, FM3 recommends conducting a 15- to 20-minute survey among 500 City of Visalia adult residents. This sample size will result in a margin of sampling error of +/ - 4.4 percentage points (the margin of sampling error for subgroups within the City’s population will be higher).
FM3 recommends an address-based, dual -mode (online and telephone) methodology to provide the most inclusive opportunity to participate in the survey. An address -based methodology enables us to collect opinions from a sample of residents reflecting the demographic characteristics of your community's population and their geographic distribution. Furthermore, by employing at least three different contact methods (telephone calls, emails, and text messages ) and two different data collection modes (telephone and online interviews), this
approach provides a wide range of ways for residents to participate in the survey. We also provide the option for sending postcards as a fourth contact method , as it helps to ensure that potential survey respondents whose telephone numbers or email addresses are not available via public or consumer records (or who are unwilling to respond to a phone call or email) can be included in the survey's sample. In all, an address -based methodology provides for a more inclusive and representative sample by best allowing all residents to have a chance to be selected.
Our survey methodology will provide a representative sample with results that can be generalized to the larger population of adult residents. If desired, once the FM3 survey is completed, we will provide the City with a link to the survey instrument that the City can provide on its website, other social media, or in person for those residents who may not have been selected but want to participate. This allows more residents to share their opinions and voice while preserving a statistically valid survey separately.
1.1.1 Phase 1 - Research Design
1.1.1.1 Sample Preparation
FM3 will construct a sample by randomly selecting stratified clusters of addresses from a list of all residential addresses in the City of Visalia. FM3 will then match the addresses in the sample a gainst public and commercial databases to acquire residents’ contact information. In the end, each address in the sample will have at least a mailing address, but also potentially telephone numbers and/or email addresses for residents living at the address.
FM3 will then analyze the sample and select which contact method to use for each address.
While FM3 is acquiring and preparing the sample, we will also establish demographic and geographic quotas of your residents ages 18+ using data provided by the U.S . Census Bureau. FM3 will use those quotas as necessary during the data collection phase —and after the interviews are completed —to ensure the final sample reflects your community’s overall adult population.
1.1.1.2 Questionnaire Design
In designing the research for this project, FM3 will draw from its knowledge of public opinion survey methodology; our comprehensive review of the City of Visalia’s past survey research (including tracking past questions when appropriate); the City’s current and future objectives an d needs; and our own vast library of research for other municipalities. The process will begin with an initial kickoff meeting (via conference call) between the FM3 team and City staff members assigned to this project, during which we will hold a comprehen sive discussion of the topics the City would like to address, including any major issues or challenges that should be explored in the survey. FM3 will then present a first draft of the questionnaire to the City for review and comment. After collecting feedback on the first draft, we will revise and refine the survey.
We foresee proceeding through several drafts, incorporating feedback from City staff before each revision, to develop a research instrument that will obtain all the information desired. Before interviewing commences, FM3 will secure approval from the appropriate City representative on the final version of the questionnaire. Once the questionnaire is approved, both the online and telephone version will be translated into Spanish.
Though the final survey questionnaire will be developed with extensive input and collaboration from the City’s team, FM3 anticipates that the final survey may assess the perceived quality of life in the City, the job being done by City government and key departments, the importance of and satisfaction with City services and programs, the most serious issues facing the City (including views on public safety, the local economy, homelessness, housing,
social and racial equity and inclusion, and other issues), residents’ priorities for the future, and if they feel their voice is being heard.
1.1.1.3 Contact Method Preparation
Because subsets of the sample will be invited to take the survey online, FM3 will create initial drafts of email, text, and (optional) postcard invitations for your review. Email, text, and postcard invitations will be tied to a specific address so that only one respondent per address can participate. (Email and text invitations will contain a unique link and postcard invitations will include a generic short link and a unique code.) Additionally, to elevate comfort levels in taking the survey, we recommend the invitations include your City seal or logo , with a note explaining you have hired an independent public opinion research firm to conduct a su rvey about local issues that are relevant to your residents, and electronically “signed” by a senior City official.
Please note that FM3’s use of text messages to distribute survey invitations has been vetted by independent legal counsel and determined to be in compliance with all applicable state and federal laws and regulations, including the U.S. Telephone Consumer Protection Act (known as the TCPA).
Telephone interviewers will also be prepped and thoroughly trained in the survey questionnaire's structure and design as well as in any unique or unfamiliar pronunciations.
1.1.2 Phase 2 - Data Collection
1.1.2.1 Interviewing (online and telephone)
Based upon the final sample specifications, FM3 will set a target number of interviews to be complete d online.
Next, we will estimate an email invitation response rate based upon your community's demographics and send out invitation emails to a random selection of email addresses. If the option to use postcards is selected, FM3 will design, print, and mail postcard invitations to a subset of addresses in the sample without telephone numbers or email addresses. Depending upon response rates, we may also send out reminder emails to recipients who have not yet taken the survey or new email invitations to addi tional addresses in the sample. FM3 will send out a number of text message invitations to help reach the overall target of online completes and complement the email and postcard methods.
Within a week of the initial distribution of email, text message, and (again, optional) postcard invitations, FM3 will compare the demographic and geographic characteristics of the online completes against our Census -derived quotas and begin conducting telephone interviews to balance out the sample and target underrepresen ted subgroups. (In many communities, those who opt to take a survey online tend to be younger and/or more affluent than the broader population, and typically have higher levels of education.) The telephone interviews are completed—and the online survey closed—when both the overall target number of interviews is reached, and the demographic and geographic quotas have been sufficiently filled. This means that sometimes more interviews than planned are completed in order to meet specific population quotas – at no extra cost to the City.
Throughout the data collection phase, researchers and statistical analysts assigned to your project will review responses daily to ensure data and sample integrity.
1.1.3 Phase 3 - Data Analysis and Reporting
1.1.3.1 Data Analysis
One to two days after interviewing has been completed, the topline survey results will be generated and FM3's initial analysis will begin. These results will show the overall percentage of respondents who chose each answer option for all survey questions.
Within two to three days, FM3 will complete a comprehensive set of cross-tabulated results. The cross-tabulated results include a table for each question or demographic variable in the survey, with a series of up to 180 columns indicating how various subgroups of residents responded to that question. The cross-tabulated results will make it possible to detect how responses differ, if at all, among various subsets of the sample. For example, it will be possible to compare answers provided by men and women; residents of various age categories and income levels; responses based on ethnicity/race, homeowners and renters, parents and non -parents, residents living in different areas of the city; and many more subgroups.
1.1.3.2 Reporting and Deliverables
FM3 will review the preliminary survey results and data with City staff and generate a detailed PowerPoint report of the survey results. The report will include a description of the methodology, executive summary, and detailed findings, and will be written to be understandable to a public audience without a background in survey research.
The report will make comparisons to any relevant research, including past Visalia surveys (if the methodology is sufficiently comparable), and will provide clear, actionable recommendations regarding how to use survey results to achieve your objectives – where appropriate. The PowerPoint report will first be presented in a draft format to City staff and then refined based on your feedback before submitting the final report for your review.
FM3 will also present the results to City staff , City Council and other key stakeholders requested at your convenience. Presentations can occur online and/or in person. Additionally, FM3 is happy to provide additional data analysis and develop summary memos or more-focused presentations or reports to clarify survey results after the initial presentation or to meet other City needs.
Upon the conclusion of the Resident Satisfaction Survey project, the City will have received from FM3 each of the documents listed below. All documents can be provided in hard copy or electronic form (or both), depending on the City’s preference.
✓ Final survey questionnaire(s)
✓ Topline results (the survey questionnaire(s) filled in with the percentages having chosen each response) ✓ A complete set of cross -tabulated results in an easy -to-read, comprehensive format (responses to every survey question, broken down by demographic, geographic and attitudinal subgroups of the electorate) ✓ Verbatim responses to any open-ended questions (if applicable) ✓ PowerPoint presentation report (an analysis of survey results, including a detailed report of topline and demographic results, key survey findings, conclusions, and actionable recommendations) ✓ Online and/or in-person presentation of results to City staff (preliminary and final), elected officials, and other stakeholders
✓ Raw data from the survey in electronic form
Finally, after FM3’s deliverables have been completed, we w ill remain available to answer follow -up questions and present results to additional stakeholders. We view the responses to the survey as an ongoing data resource; if needed, FM3 can conduct further analysis to provide answers to any follow-up questions that may arise.
2 PROJECT TIMELINE
FM3 is prepared to begin work immediately and would approach each survey in three distinct phases: research design, data collection, and data analysis and reporting. Below is a potential timeline for the Resident Satisfaction Survey with an address-based sample, with other surveys following a similar timeline. The entire process would take six to nine weeks from kick -off (voter -listed survey research can generally be conducted in less time), although we would be ha ppy to extend or compress the timeline to best meet your needs. At the conclusion of Phase 3, FM3 would remain available for ongoing consultation and any further analysis and presentation of the research as needed.
Phase 1: RESEARCH DESIGN (2-4 weeks)
• Kick-off meeting/confirmation of research specifications • Review necessary background materials
• Draft, refine, and finalize survey questionnaire
• Draft, refine, and finalize invitation email, text message & (optional) postcard • Finalize sample parameters and order/prep sample
• Translate survey questionnaire into Spanish
• Program and test survey questionnaires
• Prep, print and mail postcard invitations
Phase 2: DATA COLLECTION (2 weeks)
• Send email/text invitations and email/text reminders (as necessary) • Analyze demographics of online survey respondents
• Initiate and conduct telephone interviews
• Continually review responses and sample quotas
• Begin development of cross-tabulated report structure
Phase 3: DATA ANALYSIS & REPORTING (2-3 weeks)
• Generate topline survey results and cross-tabulated results • Conduct statistical analysis
• Draft and refine PowerPoint presentation report with City staff feedback • Present findings
3 COST FOR THE PROPOSED RESEARCH
FM3 does not bill using hourly rates but rather by project, utilizing costs that are informed by the specifications of the research being conducted. For example, our costs for survey research are informed by factors that include the number of interviews completed, the average interview length, the interviewing methodology used (dual-mode, telephone, online, intercept, etc.), the type of sample being utilized (such as likely voters, all registered voters, all adult residents, etc.), and the language(s) in which interviews are conducted. We provide all -inclusive costs indicated in Table 1 below.
Table 1 below contains four flat -rate, not -to-exceed options for your consideration. These prices are comprehensive and include all costs for questionnaire design; sample acquisition and preparation; Spanish translation; programming; email and text message invitat ions; online survey hosting (English and Spanish ); telephone interviewing (English and Spanish); data entry and analysis; and reporting and presentations. The price does not include travel and will be billed at cost if incurred.
Table 1: Estimated Survey Costs for Sample of 500
Survey Length No Postcard With Postcard
15-Minutes $40,550 $42,800
20-Minutes $44,500 $46,750
Of course, any number of alternative survey structures are possible, which may result in higher or lower costs.
FM3 is committed to working with the City of Visalia to tailor the research plan to fit your budget and meet your research needs.
We would welcome the opportunity to work with you on this research, and if you have any questions or if there is any further information we can provide, please do not hesitate to contact us. Thank you for your consideration and you may reach us as follows:
Richard Bernard, Ph.D., Partner
Fairbank, Maslin, Maullin, Metz & Associates (FM3)
12100 Wilshire Boulevard, Suite 350
Los Angeles, CA 90025
(310) 428-1809 (cell)
[email protected]