Friend, Family, and Neighbor (FFN)
Caregiver Support Network
Evaluation Report 2023
Prepared by
Attachement A
EVALCORP would like to thank and acknowledge San José Public Library for the opportunity to partner in conducting an evaluation of the Family, Friend, and Neighbor (FFN) Caregiver Support Network. Special thanks are extended to Araceli Delgado-Ortiz, Early Education Services Manager; Lisa Giannotti, Literacy Program Specialist; Hang Ngyuen, Literacy Program Specialist; and Debra-lea Olazaba, former Literacy Program Specialist, for their engagement with the evaluation and assistance with data collection.
We would also like to express gratitude for the current and past participants of the FFN Caregiver Support Network for taking time to share their experiences with us.
Moreover, this evaluation would not have been possible without the continued support and funding of the program by the San José Public Library Foundation, FIRST 5 Santa Clara County, City of San José, and David and Lucile Packard Foundation.
Acknowledgements
Attachement A
Table of Contents
Program Overview.......................4
Summary of Findings.................7
Evaluation Overview...................6
Overall Recommendations....15
Conclusion....................................18
Attachement A
On March 5, 2019, San José City Council approved an initiative related to childcare and early education and designated San José Public Library (SJPL) as the City Department to lead the implementation of childcare and early education workforce development training. To guide their work, SJPL conducted an environmental scan to identify what the current professional development opportunities and needs were for informal caregivers, otherwise known as Family, Friend, and Neighbor (FFN) caregivers, whom many parents rely on to take care of their children while they work or attend to other responsibilities. The information gathered from the environmental scan informed the program planning of the FFN Caregiver Support Network to deliver tailored, responsive training for the community of FFN caregivers in San José.
Through a collaborative partnership with the City of San José, FIRST 5 Santa Clara County, the Santa Clara Office of Education, and the Program for Infant Toddler Care, and the San José Public Library Foundation, Early Education Services at San José Public Library successfully launched the FFN Caregiver Support Network in January 2021 with its pilot cohort. The program provides tailored services and supports to accommodate caregivers’ varying needs and goals including learning and workforce development opportunities, a community of peers, and other resources. Moreover, the FFN caregivers are able to enroll in the program for multiple cohorts to successfully reach their personal goals that they set up with the program staff.
Program Overview
4 Attachement A
More than 200
Pediatric CPR/First
Aid certifications
More than 200 active
CA ECE Workforce
Registry accounts
More than 190
Trustline Registry
background checks
More than 7,500
professional
development hours
Weekly facilitated community networking
Professional development training and workshops in child development, health and safety, emergency preparedness, digital literacy, and self-care, mindfulness, and wellness
Digital devices, hotspots, and 1:1 digital literacy technical support Weekly one-on-one program guidance and support
Pediatric CPR and First Aid certification
Emergency supplies (fire extinguishers, first aid kits, fire alarms and carbon monoxide detectors)
Early learning curriculum ideas and supplies
Access to FIRST 5’s QUALITY MATTERS professional development community Financial and technical support necessary to register with TrustLine and California Early Care and Education (ECE) Workforce Registry
Ability to use professional development hours to qualify for stipends Assistance with obtaining a Family Child Care License for FFN caregivers interested in opening their own businesses
FFN Caregiver Support Network Program Components
Every year, the program continues to empower FFN caregivers through holistic programming that removes systemic barriers, fosters a diverse and inclusive environment, and facilitates caregivers’ personal and professional growth. Components of the FFN Caregiver Support Network include:
Since its launch, the FFN Caregiver Support Network has engaged more than 250 participants who have collectively achieved professional development milestones as listed below.
5 Attachement A
Evaluation Overview
Purpose
In March 2023, EVALCORP Research and Consulting (EVALCORP) was contracted by Early Education Services at San José Public Library to evaluate the FFN Caregiver Support Network. This baseline evaluation examined the current core strengths of the program and its impacts on participants. Additionally, the evaluation identified how program elements contributed to program outcomes. The program aspires to reflect and be responsive to participants’ needs; therefore, evaluation findings also illustrated areas for improvement. Throughout the process, the EVALCORP team worked collaboratively with Early Education Services staff to ensure that all evaluation elements reflected the participatory nature of the evaluation.
Methods
The evaluation employed a mixed-methods approach, including focus groups with current participants, a survey with former participants, and key stakeholder interviews with program staff. Development of all data collection tools and methodologies, as well as strategies for engagement with respondents and data analyses, were established in collaboration with program staff. The focus group protocol and survey were administered in Spanish and English to meet participants’ language and cultural needs. Administrative data provided by FFN staff was also analyzed as part of the evaluation.
Focus Groups with Current Program Participants: Focus groups were conducted virtually with participants from the current (2022–2023) cohort to understand immediate program outcomes, program strengths, and recommendations for program enhancement. Of the 126 FFN caregivers invited to participate, a total of 41 were engaged through five focus groups, two of which were conducted in Spanish.
Program Participant Survey with Former Participants: An online survey was administered to past program participants (2021 pilot cohort and 2021–2022 cohort) to gather data about satisfaction and long-term program outcomes. The survey also collected feedback regarding program strengths and recommendations for program enhancement. A total of 199 participants were invited to take part in the survey, and 45 responses (23% response rate) were collected. For the purpose of evaluating long- term outcomes, respondents who indicated that they didn’t participate in a past cohort were excluded from data analysis. A total of 38 responses were analyzed to inform the evaluation findings.
Key Stakeholder Interviews with Program Staff: Seven key stakeholder interviews were conducted with program staff to understand their perspectives on program strengths, impacts and barriers, as well as recommendations to strengthen the FFN Caregiver Support Network and replicate it for other communities of FFN caregivers. 6 Attachement A
Overview of Stakeholder Groups
A brief overview of the three stakeholder groups involved in the FFN Caregiver Support Network and who engaged in the evaluation is provided below.
Summary of Findings
Past Program Participants
Almost 70% of the individuals who responded to the past program participant survey indicated that they had participated in the program for two or more cohorts (Figure 3).
Half of the respondents were between the ages of 35–44 (Figure 4), and all identified as female.
Current Program Participants
Administrative data on the 2022–2023 cohort was provided by the program staff.
Current participants of the program ranged in age from 20–82 years (Figure 1), and a majority of them identified as female (97%). The two most preferred languages reported by participants were English and Spanish (Figure 2). 7 Attachement A
Program Strengths
Engagement with stakeholders highlighted many ways in which the FFN Caregiver Support Network provides valuable assistance to program participants. When asked about program components that were most helpful to participants, caregivers and staff unanimously pointed to the responsive staff who provide tailored support to the FFN caregivers through one-on-one development guidance. Staff mentioned that those who work directly with caregivers gain their trust by continuing to deliver support when and how it is needed. For example, the staff utilized multiple methods to connect with participants, including Zoom, phone calls, and texts. Moreover, when participants expressed that they needed resources that were outside of what the FFN Caregiver Support Network offers, the staff provided referrals to contacts who could support those needs. Program participants similarly expressed appreciation for the staff’s flexibility and patience as well as their ability to assist with each individual’s varying needs. The participants shared that they felt staff were readily available to support them. Additionally, staff emphasized that having a Spanish interpreter during all participant workshops and having materials translated into Spanish and Vietnamese further allowed them to provide responsive support for participants. "They are very supportive and responsive. The staff helped me get a hotspot given the internet is expensive...I have never felt like they have treated me less.
I have felt a lot of warmth and support."
– Current Program Participant "
Program Staff
The program staff members who participated in key stakeholder interviews fell into two categories: direct program staff (n=4) or program oversight staff (n=3). Direct program staff are those who engage directly with the FFN caregivers to provide support. Those we spoke with reported having been in their roles for a range of two months to four years. Program oversight staff, who provided additional insights into the program’s history, indicated that they had been in their roles ranging from 4-7 years.
Other top strengths of the program included opportunities for professional development and opportunities to build community with other participants.
Specifically, both participants and staff found the opportunities to complete college courses and achieve safety certifications valuable for FFN caregivers as they were able to obtain professional records that could support their future career prospects.
Meanwhile, engaging with other FFN caregivers facilitated a sense of belonging among participants. The participants appreciated the chance to connect with other caregivers in the community through shared experiences to reinforce that they were “not alone.” They emphasized that their work can be isolating at times, and the program helped foster a community that they could lean on for support. 8 Attachement A
Program Services % Satisfied or
Very Satisfied
Professional development training and workshops 97%
One-on-one program guidance and support 97%
Support with digital literacy (how to use computers and computer programs) 97% Pediatric CPR, First Aid, and Health and Safety training 97% Supply and resource giveaways 97%
In-person events (fire station visits, Children’s Discovery Museum, library playdates, etc.) 97%
Support with learning about various career opportunities within the early childhood education field 97%
Weekly facilitated community networking (Wednesday Zoom meetings) 95% Access to FIRST 5’s QUALITY MATTERS professional development community 95% Support that respects my culture and language 95%
College classes through Mission College 92%
"
To gain additional insight into the program’s strengths, past program participants were asked to rate their level of satisfaction with each program service (Table 1). A majority of individuals who completed the past program participant survey were satisfied or very satisfied with each of the services including professional development training and workshops, support with digital literacy, supply and resource giveaways, access to a professional development community, and support that is respectful of participants’ cultures and languages. Moreover, 97% of survey respondents said that they would recommend the program to others.
Table 1. Past Program Participants’ Satisfaction with Program Services (n=26-38) 9 Attachement A
Program Barriers
Along with program strengths, the evaluation assessed whether there were any barriers to program participation and implementation to ensure that the program could further deliver responsive support to FFN caregivers. While many caregivers mentioned that they had not personally experienced any barriers to participating in the program and felt that the program was accommodating and accessible, other participants and staff highlighted two primary barriers.
Scheduling Conflicts: Although the program was offered during a variety of times during the week including daytime and evenings on weekdays and weekends, some participants experienced scheduling conflicts when trying to partake in the program’s offerings due to their competing responsibilities as caregivers. Program staff also recognized scheduling conflict as a barrier for participants. They shared that while the program is tailored so that participants do not have to engage in all of the program components to successfully receive support, the participants were highly motivated and committed to participating in as many of the program’s offerings as possible. To overcome this barrier, participants suggested having workshops and college classes offered at a later time in the evening when they are typically free from other obligations. They also shared that it would be helpful to receive support with organizing their schedules and prioritizing responsibilities.
“Weekday events during the daytime were hard for me as I work during those hours… For myself, Saturdays 9-12 being actively on zoom was a challenge for me because I had to make breakfast and lunch on the weekend for the children.” – Current Program Participant
10 "
Attachement A
"
Virtual Support: Although all participants had access to in-person support with program staff, many engaged with staff virtually for convenience while they juggled multiple responsibilities. As such, some staff experienced challenges in providing support to participants virtually. They shared that being able to provide more in-person support would allow the staff to be more responsive to participants’ needs as well as overcome any technological barriers for FFN caregivers. “The staff worked really hard to do this. They did a lot of work with few staff members, and the program grew rapidly.”
– Staff "
“A lot is done virtually. It’s so convenient for people and is wonderful, but I wonder if something is lost at times doing things virtually… simple things like instructions… Trying to talk to someone about steps can be somewhat hard.”
– Staff
11 "
Language Barriers: The program was offered in Spanish and English, and translations in additional languages were provided [EM1] as requested by participants. However, some participants mentioned that having the program offered regularly in other languages like Vietnamese would be helpful especially for others in the community who felt like they couldn’t participate due to a language barrier. The staff also agreed that having services offered regularly in additional regional threshold languages as well as hiring staff who can translate for participants in their primary languages would help overcome such communication divides.
In addition to program participation barriers, the staff provided insight into the challenges they experienced in two main areas during program implementation as detailed below.
Limited Staff: The program experienced difficulties in staff recruitment and retention which contributed to a high staff-to-participant ratio and high burnout in staff.
Relatedly, the staff conveyed concerns regarding the program’s ability to continue providing support for participants due to limited staffing. Some staff suggested reaching out to community networks to enhance recruitment of additional staff members.
Attachement A
Program Impacts
The evaluation engaged with both current and past participant cohorts to assess the short- and long-term impacts of the FFN Caregiver Support Network on participants’ lives.
Short-Term Impacts
The current participant cohort highlighted three main short-term impacts from their participation in the program.
Increased Skills and Knowledge: Participants shared that they felt more knowledgeable about child development and caregiving practices which they now apply to their work as caregivers. Others mentioned that they were more aware of the resources available to them in their community. Moreover, program participants expressed that the program helped improve their caregiving practices, including juggling multiple life responsibilities, as well as increased their confidence in their role as caregivers. "
"It’s given me confidence for not only my own children but also in viewing others around me... It has made me want to better myself and seek greater understanding as my child grows older and at every age."
– Current Program Participant
12 "
Attachement A
"
"
Increased Sense of Professional Identity: FFN Caregiver Support Network participants reported that the program helped them to recognize the value of their work and feel a sense of pride about the care they provide to children. They shared that the program helped them to recognize their professional identity in the field of early childhood education and learn about potential career opportunities. Importantly, the participants felt validated and appreciated for the work they provided to their community.
“Everyone has their own trajectory, and
I don’t know where I’ll go with it. Now
that I know there is such a need, I may
volunteer somewhere where there are
kids at risk, gang prevention, anything
that would help in my neighborhood…
It put my thoughts into a whole
different perspective.”
– Current Program Participant
"
Increased Sense of Belonging: Many
caregivers emphasized that participation in
the program resulted in an increased sense
of belonging in their community. Participants
reported that the program helped them build
connections with other FFN caregivers in the
community through their mutual
experiences. In particular, this network of
FFN caregivers facilitated additional support
and information sharing for participants.
Some participants also shared that the
strengthened sense of belonging motivated
them to give back to their community by
volunteering at organizations or helping
other FFN caregivers.
“Hearing someone validate me as a caregiver is so important. It needs to be valued and appreciated. As a caregiver, a lot of people might just say you don’t have a job, you can just take care of the kids, or you’re just a babysitter. Hearing that we’re valued… and without us the community wouldn’t be able to go back to work… Hearing someone thank us and appreciate us, it feels really good.”
– Current Program Participant
13 Attachement A
Long-Term Impacts
Past program cohorts shed light on the long-term impacts of the program. A majority of caregivers who completed the past program participant survey agreed that the FFN Caregiver Support Network increased their knowledge about best practices for childcare and education, different digital products and platforms, and available community resources and supports (Table 2). Moreover, 100% of respondents agreed that they continue to utilize the information they learned from the program and that they are better caregivers overall after completing the program.
Table 2. Program’s Long-Term Impacts on Past Program Participants (n=26–38) Additionally, respondents were asked to share examples of what they had specifically acquired from the program that they continue to apply to their work as caregivers.
Responses included implementing what they learned about child development, utilizing their knowledge about available community resources, improving their communication with children and their parents, and developing professionally and personally. 14 Program Services % Satisfied or
Very Satisfied
I know more about best practices for childcare and education. 100% I am more aware of opportunities available to me in the early childhood education field. 100% I feel more confident about my ability to provide quality childcare and education. 100% I feel that the work I do is important. 100%
I feel more confident about advocating for myself and other FFN caregivers. 100% I continue to use the information I learned from the classes/workshops I attended. 100% I have shared the information I learned with others in my community. 100% Overall, I feel that I am a better caregiver to the children I care for. 100% I know more about different digital products and platforms. 97% I feel more confident about navigating different digital products and platforms. 97% I feel an increased sense of belonging and connection to the FFN community. 97% I feel more connected to the professional childcare system. 97% I know more about what community resources and supports are available to me. 94% Attachement A
Overall Recommendations
Recommendations for program enhancement were gathered from participants and staff to ensure that the FFN Caregiver Support Network can continue to adapt and provide responsive services for current program participants. Further, recommendations for program replication were gathered from staff to inform best practices for programs supporting FFN caregiver communities.
Recommendations for Program Enhancement
Program participants and staff were asked to share their thoughts on additional services that would be helpful for FFN caregivers. Their recommendations fell largely into the categories detailed below.
Additional Connection-Building Opportunities
Program participants mentioned that having more opportunities to build connections with other FFN caregivers, such as weekend events, playdates, and meet-and-greet functions, would facilitate a greater sense of community and information exchange.
Additional Accommodations for Workshops and Trainings
As well as having more networking opportunities, participants expressed that additional accommodations would be helpful to ensure that participants can receive the services that are offered. Examples included having services in additional languages such as Vietnamese, having access to technology to participate in virtual sessions, sharing of Zoom session recordings for participants who are not able to attend workshops, and having more in-person sessions for hands-on activities. Program staff echoed that additional in-person sessions would be beneficial and would help them to provide more hands-on support to overcome technological barriers. "
“It was challenging to be in the gatherings and classes… Communicating what occurred in missed meetings would be important to feel supported so everyone is up to date. For example, email or recording of meeting...”
– Current Program Participant
15 Attachement A
"
Caring for Children With Special Needs: Participants were interested in learning about different special needs that children may have, like autism or speech development, as well as learning ways to make them feel included with other children they care for.
Caring for School-Aged Children: Participants shared that they would like to receive support in caring for school-aged children. Providing guidance and lessons on caring for school-aged children would allow caregivers to continue caring for children as they age and support them with their schoolwork.
Additional Workshop or Training Topics
Program participants had multiple requests for additional workshop or training topics that would further support their work as caregivers. The most frequently mentioned topics are detailed below.
“As the children get older, understanding methods to introduce reading and math skills to the children would be helpful.”
– Past Program Participant
"
Managing Stress in Children: Participants expressed that they would like to learn techniques to help manage stress in children.
Career Guidance: Participants wanted additional career development guidance such as support with writing cover letters or résumés and interview practice.
Continuation of Support Post Program Completion
The past program participant cohorts shared that it would be helpful to be connected with additional support and resources after program completion. Similarly, the program staff described wanting to offer continued assistance for those who have completed the program. They communicated that the FFN Caregiver Support Network delivers unique and valuable support for participants and that they should consider ways to extend this support for FFN caregivers who are no longer in the program. "
"What’s going to be next for the participants who are caring for children who grow older? It would be good for them to stay in the alumni network and connect them to additional resources that they need at the time.”
– Staff "
Promoting Program Sustainability
Lastly, staff recommended obtaining sustainable funding sources to ensure adequate staffing as well as the program’s continued impact on the community. Some also suggested following up with past program participants to assess the program’s additional long-term impacts to inform potential funders and other stakeholders of the program’s importance in the community. 16
Attachement A
Recommendations for Program Replication
In addition to recommendations for enhancing the FFN Caregiver Support Network, program staff were asked to reflect on what they think are critical areas for successful program replication in other communities of FFN caregivers. The recommendations that were formed from emerging themes are detailed below.
17 “Consider if people are feeling
nervous about immigration status. Be
sensitive to that and understand it
will take a long time to build trust
and don’t ask for too much personal
information.”
– Staff
"
Know the Community That Will Be Served
It is important to know the community that will be
served, including their needs, beliefs, interests,
levels of agency, and best ways of connecting with
them. Putting in the work to understand the
community allows programs to provide flexible and
tailored programming that is responsive to the
community’s needs.
Form Partnerships With Other Organizations
Form partnerships with other local community-based organizations and agencies to leverage resources and funding. Partnerships with others will increase opportunities for supporting the community.
Have Appropriate Staffing
Ensure that the program has appropriate staffing in terms of numbers, passion for serving the community, and cultural competency. Having enough staff is important to provide adequate support for each participant. Currently, the literature does not indicate best practices for staff ratios for FFN caregiver support programs. It will be important to continuously monitor staff’s workloads and identify ways to balance the staff-to-participant ratio. Additionally, it is crucial to have culturally competent staff who can meet the language and culture needs of the community to gain their trust.
Provide Financial Support for Participants
Programs should be offered at no cost to the participants to overcome any financial barriers they may face.
Moreover, programs should provide supplies that
individuals would need to participate in the program such as laptops or internet connections. Financial incentives can be helpful in motivating participants to complete the program as they juggle multiple responsibilities.
Offer Professional Development Opportunities
Professional development opportunities should be
included in programs for FFN caregivers. The opportunity to obtain records of college course completion or training certifications are a valuable support for FFN caregivers’ career growth.
“To have partnership with
secondary education institutions.
That really lends validity and
carries weight especially for
people who thought they couldn’t
access anything at a college level.”
– Staff
"
Attachement A
As evidenced by the highlighted strengths and impacts of the program, the FFN Caregiver Support Network provides critical support and resources for caregivers in the city of San José. It is clear that the program is helping participants recognize the value of their role as caregivers in the community. Most notably, FFN caregivers are applying what they learned from the program to improve how they provide care for children in the community and support other informal caregivers like themselves. It is also important to note that the program participants who were engaged in the evaluation process took every opportunity to express their appreciation and gratitude for program staff. Many conveyed that the staff played a significant role in helping them feel heard, valued, and supported.
Program oversight staff likewise praised the direct program staff for their work and dedication that contributed to lasting impacts on FFN caregivers’ lives. Further, they urge other programs to serve communities of FFN caregivers given the great need and demonstrated positive impact of this program. Overall, the recommendations for program enhancement and replication gathered in this evaluation can inform best practices for providing responsive and tailored services, supports, and growth opportunities for additional communities of FFN caregivers. It is recommended that a separate evaluation is conducted to examine the long-term program impacts on its participants as the program hits its 5-year mark.
Given the FFN Caregiver Support Network staff’s commitment to meet the needs of their participants, they have already begun to address two of the program participation barriers expressed by caregivers (pages 10-11). The program has hired two additional full-time staff members to increase its capacity to deliver one-on-one guidance to the program cohorts and has enhanced its language capacity by hiring two Spanish interpreters to translate in real-time at the program’s weekly participant workshops. Program staff will continue to discuss ways to address remaining barriers and implement recommendations made by both program staff and participants. "I like the people who are in charge of the program. They are friendly, kind, and generous with their time. I think the staff put in a lot of work and made perfect arrangements, like the class topics, events, and give aways. Debra and Lisa have been there since day one. I look forward to going to meetings to see familiar faces and they’re rock stars. I appreciate their hard work."
– Current Program Participant
18 "
Conclusion
Attachement A