1 Council Study Session
City AI Initiatives and GovAI Coalition Summit Study Session | October 25, 2024 Information Technology Department
Khaled Tawfik, Chief Information Officer
Leila Doty, Privacy and AI Analyst
Namrata Batra Agrawal, Products & Projects Manager
Albert Gehami, Digital Privacy Officer
Angela Zhao, Privacy and AI Analyst
2 Agenda
• Introduction
• What is AI?
• Gaining Value from AI in the City
• Laying the Foundation for Responsible AI
• Support beyond the City (GovAI Coalition)
• Feedback & Questions
3 San José
Civic Innovation
& AI Hub
San José Youth
Empowerment
Alliance
4 Takeaways
Intentional investments in AI initiatives has provided
the City with more efficient and accessible services
for residents.
Building trust among the community and City staff
through technical education promotes inclusion and
moves San José forward for everyone.
Responsible AI governance maximizes the benefits
to residents while mitigating potential risks.
By founding the GovAI Coalition, the City advanced
its AI program while affirming its national leadership
in responsible and effective use of AI.
5 What is AI?
6 6
1950
1970
1990
2000
2010
2020
2030
Turing
Test
With Rapid Innovation, Associated Risks Increase
7 Modern “AI” Works By Finding Patterns
Source: Federal Reserve Bank of St. Louis, Housing Inventory: Median Listing Price in San Jose-Sunnyvale-Santa Clara, CA (CBSA), U.S. Dollars This simple idea
completed
trillions
of times
creates the
“magic” behind AI
$0.0M
$0.4M
$0.8M
$1.2M
$1.6M
2020 2021 2022 2023
San José Median home price
2023
8 AI Computer Vision is Identifying Patterns in Shapes
A
A
Data
Human
A
Algorithm
AI
4 4 A California
ABC1234
dmv.ca.gov
A
a
A A
A Aa
A
a
A4
ApplicationA
9 AI Comes in
Many Different
Types
Source: Center for Scientific Evidence in Public Issues Type of Output
Prediction AI
“forecasting”
Perception AI
“identifying”
Generative AI
“creating”
Example
California
ABC1234
dmv.ca.gov
This Photo by Unknown Author is licensed under CC BY-ND 10 ChatGPT:
Finding
patterns to
generate
the next
word
The San José Sharks are the best at hockey.
Find the pattern to generate the next word
11 Ask ChatGPT
• How will AI directly improve the quality of life for our residents? • What steps will we take to ensure AI solutions are accessible and equitable for all residents, including marginalized communities? • How will the City protect residents from potential job displacement or economic disruption caused by AI automation?
• What safeguards are in place to ensure AI systems are unbiased and do not unintentionally harm vulnerable populations? • How will we involve residents in decision-making about AI adoption, and how will we ensure transparency in its use? What 5 questions can we expect from a City Council study session on AI? “ ”
Generated in
5 seconds
Note: Emphasis via bolding was added manually
12 Prompt
Hi ChatGPT, I want you to
create a summary of item
7.1 from the October 29
City Council Agenda.
“Agenda link”
Use the attached memo.
Create a concise and
professional summary with
at most 5 bullet points.
Output
Input:
16-page memo
Ask ChatGPT:
Summarizing October 29 Council Agenda
13 AI in the City
14 AI Opportunities To Support City Goals
City Council Focus Areas
• ALPR, speeding cameras,
red light cameras
• Road safety detection
(road hazards)
• Data-informed
Decisions
• AI opportunities with
Housing and Planning
• Road Safety Detection
(graffiti & trash removal)
• Improve customer
satisfaction by proactively
detecting SJ311 services
• Streamlined & automated
services
• AI waste sorting solutions
• Efficient building permit
process
• Data-informed
Decisions
• Reducing bus travel time
• City AI partnerships with
SJSU
Increasing Community
Safety
Reducing Unsheltered
Homelessness
Cleaning Up Our
Neighborhoods
Attracting Investment
in Jobs and Housing
15 Staff
Upskilling
Programs
Data Upskilling
• Initiated in 2023
• Train City staff to acquire data skills
and promote data governance
processes
• Increase efficiency and quality of
services
AI Upskilling
• Initiated in 2024
• Train City staff to build AI tools to
achieve high efficiency
• Partnership with SJSU
“
”
I was able to transform
thousands of lines of data into
one singular visual that users
can appreciate.
16 Initial time savings with AI
Staff reported saving more than 50%
on specific tasks after receiving AI training
from the IT Department
Hundreds of staff trained in AI through ITD
Based on post-training survey of staff
Photo by Trisha Mendoza on Unsplash
17 AI Bridges
Language Barriers
• Two-way interpretation in 50+ languages at 25% of the cost of in-person interpreters for 2 languages
• 370% increase in online views due to Spanish captioning • City staff uses AI translate with San José 311 (SJ311)
18 18
Traffic Signals: Reduce Bus Travel Time by up to 20%
Optimized traffic at intersections using AI
Downtown San José bus lines 66 and 68
• Up to 20% faster
• 50% less time at red lights
Benefits
• 14% decrease in fuel consumption
• Reduction in emissions
• 20% reduction in commuting time
• Improves public transit experience
https://lyt.ai/customer-success-stories/delivering-faster-bus-rides-through-downtown-san-jose/ 19 Drafting Request For Proposal Solicitations
• Ask ChatGPT to create a draft in minutes to identify
questions that really matter
• Jumpstart laborious RFP development and save staff
expertise for refining business requirements
Prompt
“ ”
I can save about
8 hours per RFP .
20 20
AI Vision Traffic Cameras Support Vision Zero
How often does a
vehicle run a red
light here?
95% vehicle count accuracy and 82% pedestrian count accuracy informs traffic planning 24/7 intersection monitoring provides critical data to help reduce traffic fatalities Person at-risk
of getting hit
https://www.sanjoseca.gov/home/showpublisheddocument/88839/637975641014730000 View from AI traffic camera monitoring near miss Data answers key safety questions 21 Responsible AI
22 22
AI Risks Are Well-Documented
23 23
A
Data
A
Algorithm
4 4 A California
ABC1234
dmv.ca.gov
a
A A
A Aa
A
A4
AI Application
Reminder:
AI can be examined in three ways
24 Risk:
Algorithmic Bias
Data
Algorithmic Bias
Systematic and repeatable errors in a computer system
that create unfair outcomes
Dr. Joy Buolamwini uncovers racial bias in facial recognition tools (2018) dark-skinned face
not recognized by AI
white theater mask
recognized by AI
25 Risk:
Algorithmic Bias
Data
How are we mitigating this risk?
Trained Staff
Reliable Processes and Technology
• Data Upskilling Program
• AI Upskilling Program
• Testing before deployment
• Vendor disclosure
• Community engagement
Data Governance
• Data Governance Policy
• AI Readiness for Data
26 Algorithm
The Problem
Inputs OutputsRisk:
Lack of transparency
We see the inputs and outputs, but the processes in
between are hidden.
Black Box
???
27 Algorithm
AI FactSheet
Provided by vendors
Documents the technical
details of AI system
Ensures transparency of
AI systems
Risk:
Lack of Transparency
How are we mitigating this risk?
28 Risk:
Inaccurate Outputs
Real-World
Application
29 Risk:
Inaccurate Outputs
Real-World
Application
How are we mitigating this risk?
Field Testing
We learn and adapt by testing or piloting AI systems before fully deploying in the real-world to ensure effectiveness
30 Trailblazing the
Digital Privacy
Space
San José, which has taken a leading role in
defining responsible government use of AI
systems, appears to be the only city that requires
its police department to disclose accuracy data
for its gunshot detection system.
“
”
June 25, 2024
31 Policy
• AI Policy
• AI Policy Manual
• Generative AI
Guidelines
• Data Usage Protocols
• AI Meeting Notes
Guidelines
Processes
• AI Review
• Algorithmic Impact
Assessment Form
• AI Factsheet
• AI Standards and
Requirements for
Vendors
• Annual Usage Reports
People
• Community
Engagement
• Digital Privacy Office
• AI Working Group
• Digital Privacy
Advisory Taskforce
• GovAI Coalition
Responsible AI Governance Program
Who do you contact?
San José Digital Privacy Team at [email protected] 32 • Applies to all AI used by or on behalf of City
• Guiding Principles for responsible AI
• Roles and responsibilities surrounding
AI systems
• AI Review requirements for AI procurements
• Prohibited uses of AI systems
• Sunset procedures for AI systems
• Policy enforcement
The City’s Administrative AI Policy
governs the use of AI systems
Accessible via SJ City Policy Manual: 1.7.12
33 33
Responsible Use of AI
Acceptable Uses Prohibited Uses
As outlined in the City’s AI Policy
• Real-time and covert biometric identification
• Emotion analysis
• Fully automated decisions
• Social scoring
• Cognitive behavioral manipulation
• Designed to improve quality of life
• Expand accessibility for residents
• Enhance City services
• Respects residents’ privacy
Follows the City’s Guiding Principles
34 Leading community
engagement to guide
responsible use of AI
34 The Digital Privacy Team is actively educating and
engaging San José residents on technology
initiatives.
Education
AI 101
Deepfakes
AI voice-cloning scams
How to use ChatGPT
Engagement
Interactive sessions
Advisory Taskforce
Feedback on technology
initiatives
Next step: Building a
resident advisory board
Note: “Advisory Taskforce” refers to the Digital Privacy Advisory Taskforce 35 AI in the City:
Navigating use cases for high value
36 36
Deter Crime with Automated License Plate Reader (ALPR)
• ALPR assisted with solving 31+ notable
criminal investigations within four months
(robberies, stolen vehicles, hit-and-runs,
homicides, and shootings)
• Indispensable and widely used tool for
Special Operations, the Bureau of
Investigations, the Bureau of Field
Operations, and the Air Support Unit
167,014 license plates in active
investigations detected in 2023https://www.flocksafety.com/industries/law-enforcement 37 37
Automatic feed to SJ311
Proactive Services
Address issues before they are reported
Road Safety: Leverage AI to Detect Issues
38 38
Road Safety Conditions Pilot: Phase One
ThePhoto by PhotoAuthor is licensed under CCYYSA.
• Identifying information are blurred
including faces, out-of-scope
license plates, and home
addresses.
• Need more testing to ensure AI
doesn’t develop biases.
Digital Privacy Protections Accuracy of Detection
• Detection shows promise to make
City services more proactive to
resolve road conditions.
• The City continues to assess
other service areas.
Output to Increase City Services
• Phase One has shown AI can
detect potholes accurately.
• A more diverse test database is
necessary to confirm accuracy
of trash and road conditions.
39 39
Road Safety Conditions Pilot - Phase Two
Implementing privacy protection
Regular footage collection with two street sweepers
and four cameras
Partnering with and funded by
Toyota Mobility Foundation
RFI for vendor collaboration
40 Support from the City and Beyond
41 Empower public sector entities to harness the potential of AI for the public goodMission
Helped the City ensure responsible and effective
use of AI in San José
• Recognized as a national leader in AI policies and innovation • Over 150 agencies accepted the Coalition’s Open Letter, endorsing AI principles
• Collective best work was incorporated in San José’s program GovAI Coalition
Founded by the City of San José in November 2023
42 As of September 2024
Over 400 public agencies
Over 1,200 members
including:
Mayors
City Managers
Directors, CIOs & CISO
Non-profits/NGOs
AI Vendors
“ ”
The GovAI Coalition’s deliverables
are really an impressive set of
practical resources.
GovAI Coalition member
SJ leads the GovAI Coalition of 400 agencies committed to responsible AI 43 GovAI
Coalition
Founded by the
City of San José in
November 2023
• GovAI Coalition Board: TriMet, City of San Antonio,
Colorado Dept of Revenue, City of San Diego, City of
St. Paul, City of Long Beach
• White House Office of Science and Technology Policy
(OSTP)
• Department of Homeland Security (DHS)
• California Governor’s Office of Government Operations • Administration for Children and Families (ACF)
• National Institute of Standards and Technology (NIST) • San José State University (SJSU)
• Federal Trade Commission (FTC)
• California Privacy Protection Agency (CPPA)
Partnerships
44 Date: December 4-5
Location: San Jose Convention Center
Purpose: Grow responsible and effective AI in the
center of San José
SOLD OUT
500
Attendees
45 45
San José has received support to lead responsible AI in the public sector.
46 Summary
47 Takeaways
Intentional investments in AI initiatives has provided
the City with more efficient and accessible services
for residents.
Building trust among the community and City staff
through technical education promotes inclusion and
moves San José forward for everyone.
Responsible AI governance maximizes the benefits
to residents while mitigating the potential risks.
By founding the GovAI Coalition, the City advanced
its AI program while affirming its national leadership
in responsible and effective use of AI.
48 Join Our Efforts!
Partner to promote safe
and responsible use of AI
Invite us to engage with
neighborhoods on City
technology initiatives
Endorse policies that align
with the GovAI Coalition’s AI
Guiding Principles
49 This work was made possible with the collaboration
and support of the following City partners:
City Manager’s Office
Mayor’s Innovation Team
City Clerk’s Office
Human Resources
Office of Employee
Relations
City Attorney’s Office
City Council
Department of Transportation
Police Department
Human Resources
Parks, Recreation, and
Neighborhood Services
San José State University
50 Questions & Feedback
Council Study Session: City AI Initiatives and GovAI Coalition Summit Study Session Information Technology Department
Khaled Tawfik, Chief Information Officer
Leila Doty, Privacy and AI Analyst
Namrata Batra Agrawal, Products & Projects Manager
Albert Gehami, Digital Privacy Officer
Angela Zhao, Privacy and AI Analyst
Arti Tangri, Equity through Data Lead