1 City Fingerprint Project
City Fingerprint 2022
Mayor Barbara Halliday
“Hayward is a data-driven city that prioritizes
being clean and green. I’m looking forward to
seeing how the City Fingerprint Project can help us
develop policies and practices to respond more
quickly and efficiently to litter and illegal dumping
hot spots.”
INDEX
TOPIC PAGE
Executive Summary 5
Key Insights 12
Results 21
Considerations 29
Annex 31
Research methodology and execution
- Methodology
- Research locations
- Execution and limits
- Data Sources
35 36
37 45
57 Executive Summary
Overall context: Litterati will establish a litter fingerprint and baseline for each Study Area in the city.
Seasonality: To account for this, the baseline was carried out in Spring, Summer, and Fall.
The Study: Our research measured two aspects of litter in each area:
■ Litter Composition*:
● The general makeup of litter observed by our researchers. ■ Litter Density:
● A normalized measurement of litter per square meter in each study area. ● This measurement will be denoted as Identifiable Litter per Meter (iLPM) Executive Summary - Project Overview
*Litter composition as determined by our taxonomy
Study Objectives
● To determine the spatial concentrations of litter within each Study Area, we built a model that considers the effects of land-use type, proximity to places of interest, or demographic characteristics on litter density to extrapolate litter concentrations across the study area. ● Based on that, we utilized the Empirical Bayesian Kriging (EBK) method and Prediction tool in ArcGIS Pro to extrapolate a litter density map for each Study Area.
Executive Summary - Project Overview
Executive Summary - Project Overview
Study Timeline
● Week long collection period
● One week QA period
● 300 locations per city
Phase 1 Phase 2 Phase 3
April
11- 18
July
11-18
September
19-26
10 Zones 7 Zones 7 Zones
30 Locations
per zone
43 Locations
per zone
43 Locations
per zone
Executive Summary - QA Process and Results - Hayward*
Skipped /
Rejected
48 (16%)
Approved
252
(84%)
Total Number of Sampling
Locations: 300 per phase
Skipped / Rejected
● Locations not accessible
● Dangerous road/place
* Averaged approved/rejected over all 3 phases
CHECK
● Segment’s length
● Number of segments
● Comments
CHECK
● Quality of images
● Images in the correct
folder
● GPS signal
Executive Summary - QA Process and Results - All Cities* * Averaged approved/rejected over all 3 phases
Skipped /
Rejected
15 (5%)
Approved
289
(96%)
Norfolk
Skipped /
Rejected
83 (28%)
Approved
217
(72%)
Memphis
Skipped /
Rejected
48 (16%)
Approved
252
(84%)
Hayward
Executive Summary - City Fingerprint Dashboard*
* View Additional Statistics (e.g, Litter per Meter by Landuse/Demographics) on the City Fingerprint Project - Dashboard.
Key Insights
18 Overall Litter: Key Descriptive Statistics
* Identifiable Litter per Meter (iLPM) is the average amount of litter our ML can identify per segment length (meters) in ea ch city. ** Labeling confidence @0.78 - 0.86
● Norfolk had the most items overall
● Memphis had the most weight and volume collected
● Hayward and Norfolk had a higher average of Identifiable Litter per Meter (iLPM), due to smoking related litter ● Drink items being larger, this increased the weight and volume significantly in Memphis City Average
(iLPM)
#Items
(Identifiable)
#Items
(Unidentifiable)
Weight
(grams)
Volume
(milliliters)
Hayward 0.113 16,131 14,171 77,304 721,910
Memphis 0.105 12,713 16,429 137,746 1,246,921
Norfolk 0.113 21,073 21,183 107,031 1,160,556
Total 0.111 49,917 51,783 322,181 3,129,837
Key Insights - All Three Cities
Insights Span all Three Study Periods
● Norfolk and Hayward had an increase of iLPM
found over the three studies*
● For all three cities, the main categories found
were
○ Smoking
○ Drink
○ Personal hygiene
*iLPM - Only litter that was identified, pieces were not counted
Key insights - Hayward - Materials
● Non-Plastic recyclables: aluminum, glass, and metal were lower in numbers than other materials by a wide margin
● Plastics, cellulose acetate were highest
Key insights - Hayward - Categories
● Smoking, drink, household product items increased phase over phase (PoP)
● Food mostly stayed flat
● Personal hygiene items increased in the summer, but didn’t continue to increase moving into fall. This seems to track with mask mandates going away
● Smoking items seemed to jump markedly from summer to fall
Average Comparison
● Smoking was the highest amount of litter found, however Norfolk had ~38% more litter than Memphis
● Memphis had ~25% more drink litter than Norfolk
● Hayward and Norfolk had similar drink and household items found
● Personal hygiene items were more prevalent in Hayward and Memphis compared to Norfolk
Key Insights in detail
Across the 3 CFP cities
1. Smoking and Drink related categories were
significantly higher than other categories
2. Norfolk’s smoking related data is higher than all three cities
3. Memphis’s open container law could be contributing to higher levels of drink litter
Hotspot Concentrations* (All Litter over 3 studies)
NorfolkMemphisHayward
*Based value input on iLPM and the Mask on AOI Boundaries (without Exclusion Zones). Based on EBK analysis:
https://pro.arcgis.com/en/pro-app/latest/help/analysis/geostatistical-analyst/what-is-empirical-bayesian-kriging-.htm Hotspots - Hayward
Results
33 RESEARCHER TEAM - Hayward
Name Phase 1 Phase 2* Phase 3*†
Alexa Rivas* x x
Alfredo Coronado x
Allie Oilar x
Bert Manzo x
Isabella Casique x
Janie Chew x
Joanna Sizemore x
Juan-Carlos Martinez x x
Justin Oilar x
Leah Martinez x x
Mathew Humphreys-Martin x
Melissa Milleman x x
Michael Chang x
Nicole Nelli x x
Robert Martinez x x x
Roderick Robles x
Notes
● *Dropped from 10 to 7 zones
● †One researcher covered multiple zones
Click here to view the ArcGIS dashboard
Identifiable Litter per City - Hayward
Cigarette Butts and Bottle Caps are the most common identifiable pieces of litter
Most littered objects - Hayward
Classement Spring Summer Fall
1 cigarettebutt: 2923 cigarettebutt: 2651 cigarettebutt: 2445 2 bottlecap: 450 bottlecap: 421 bottlecap: 459
3 tissue: 313 tissue: 350 tissue: 308
4 cup: 235 bottle: 280 cup: 248
5 bottle: 231 cup: 267 bottle: 239
6 facemask: 205 facemask: 187 facemask: 159
7 label: 140 lid: 157 lid: 131
8 can: 125 straw: 148 straw: 121
9 straw: 122 box: 126 can: 103
10 box: 119 label: 113 label: 102
Most common brands - Hayward
Land Use Type - Hayward*
● Most hotspots were found in medium to high density
residential areas
● Some hotspots in industrial corridor
● More insights can be gathered from the ArcGIS Dashboard * View Additional Statistics (e.g, Litter per Meter by Land use/Demographics) on the City Fingerprint Project - Dashboard.
Heatmap Analysis - Litter by Points of Interest*
* View Additional Statistics (e.g, Litter per Meter by Land use/Demographics) on the City Fingerprint Project - Dashboard.
Heatmap Analysis - Litter by Census data*
* View Additional Statistics (e.g, Litter per Meter by Land use/Demographics) on the City Fingerprint Project - Dashboard.
Considerations
Go Forward Considerations
Researchers Considerations:
• Researcher safety should be considered above all else • Research vigilance is important
• Capturing all observations
• Taking time to review for quality
• Researchers need to remain consistent and not stray from their transect Seasonality:
• Date selection should occur in the middle or ⅔’s through a season, to allow for organic littering behavior to occur
• Too early a collection would potentially lead to observations more inline with the previous season
53 Annex
Segment QA process
* As of September 12th 2022
1 SEGMENT LENGTH - distance in all segments around 100m QA CHECKs
2 N° SEGMENTS - each location needs to have 2 segments
NOTE - if any important feedback from researcher3
4 5
PHOTOS
❖quality of images (not blurred, distance, light…)
❖correct folder (litter/bins/hotspots)
GPS signal - segment track is as much as possible a straight line
CHECK
● Segment’s length
● Number of segments
● Comments
CHECK
● Quality of images
● Images in the correct
folder
● GPS signal
RESEARCH IMPLEMENTATION STRATEGY
In order to perform the field activities Litterati engaged with:
● City Organizers: Many of the researchers were from the city organizer’s own personal network and were referred to the project.
● Educational Establishments: A few of our researchers were recommended through university postings
● Friends/Relatives of Researchers: Many of our later phase researchers came from direct recommendation from our phase 1 researchers
RESEARCHERS TRAINING PROGRAM
To ensure all researchers were trained in the use of the Litterati App and the Litterati methodology our participants went through various types of training:
● Online training and certification (Citizen Scientist Researcher) through Zoho Recruit ● Online meetings and training overview session through Google Meet ● Online training through video and corresponding slideshow review.
Research methodology and execution
57 Litter
Monitoring
Design
The resource to be monitored: Litter
What will be measured (indicator)?: Litter per m2
How it will be measured (respond design)?: See Data
Collection
Where it will be monitored (survey design)?: Spatially
Balanced Sampling (SBS) will be used to identify up to
300 sampling locations in each study area.
How frequently it will be monitored (time selection)?:
Quarterly (four-times per year)
How measurements will be summarized (population
estimation)?: See Analysis
Sampling Design: Determination of Survey Areas
“The set of geographic locations where measurements are taken is called a spatial sampling design. An efficient sampling design specifies sampling locations that allow the researcher to confidently estimate the value of the sampled variable, suchas pollution, at unsampled locations” (Link).
A Spatially Balanced
Sampling Design
➢ Spatially Balanced Sampling (SBS) is a sampling method that maximizes spatial independence among sample locations,
ensuring that every sample is distributed across the population (Theobald et al. 2007)
➢ Considers spatial heterogeneity and spatial autocorrelation ➢ SBS should be used when the spatial pattern of the response variable is unknown prior to sampling.
○ One of the objectives of the CFP is to identify explanatory variables influencing the amount and density of litter within each study area.
○ Following the CFP, it may be more appropriate to utilize a stratified random sample guided using multivariate clusters derived from the CFP.
➢ Lastly, SBS allows for oversampling (i.e. regeneration of additional sample locations if some are inaccessible).
○ This is especially important considering we do not yet know how many samples are required to correctly reject our null hypotheses.
➢ Literati uses SBS to select Survey Areas within the study area. ➢ Once determined, Litterati assigns trained researchers to systematically record litter along at least two transects (1x100m) within each Survey Area.
Determination of Survey Areas
1. Establish the Study Area Boundary and Exclusion Zone (Mask). a. Inputs: City Boundary, Census Blocks, Airports, and other excluded areas (polygon required).
b. Output: Study Area Boundary
c. Determine Optimum Sampling Size (n)
2. Create Inclusion-Probability Raster (IPR)
a. Clip SafeGraph CORE Places POI Dataset to Mask
b. Create a Distance Allocation Raster of POI Dataset.
i. Inputs: SafeGraph POI Data or OSM Data
c. Rescale the Values using the MSSmall Function to assign a higher inclusion probability score to the areas closest to all POIs.
3. Use the Spatially Balanced Sampling tool to distribute 300* points across the Study Area (*See Discussion on Determining Optimum Sample Size). a. Input: Reclassed Distance Raster (Areas with a score closer to 1 have a higher probability of being Sampled).
b. Output: Point feature class with 300 points .
4. Snap points to the nearest roadway (excluding major highways) or pedestrian footpath.
a. This step can either use city-provided data, data from the US TIGER Database, or OpenStreetMap.
b. The roads and sidewalks must also be clipped to the same Mask.
Here is an animated example of these steps being applied to Memphis, TN.
Identification of
Study Area
Boundary and
Exclusion Zones
Define Study Area:
➢The Study Area is the research boundary. Any area located outside of this area will not be surveyed by Litterati Researchers. Additional, exclusion zones within the study area will not be surveyed by Litterati researchers and must be identified early-on in the sampling design. ➢The term ‘exclusion zone’ refers to areas within each municipality that are inaccessible by our researchers because of denied access, safety concerns, or because the areas are otherwise difficult to access. For example, water features, military zones, construction sites, and airports are frequently excluded from the study area.
Create Mask:
➢ Once the exclusion zones have been determined, they are clipped out from the Study Area. The resulting polygon is used as a Mask in subsequent steps and analyses.
Example: Memphis, Tennessee
➢ The map on the left shows the mask for Memphis, TN created by extracting bodies of water (Census Blocks where the total of area of land equals zero) and removing the Memphis International Airport (surmised from city-provided land-use data).
Sample Exclusion Zones (Red X) in Memphis Tennessee: Airport
SBS is an unequal probability-based sampling strategy, meaning that some locations within the study area have a higher or lower probability of being selected. These probabilities are assigned via an Inclusion-Probability Raster (IPR) where each cell is assigned a value between 0 and 1. As long as the raster cell has a value greater than 0, it has a change of being selected.
How does Litterati assign a probability score?
➢ A raster cell is assigned a value of zero if it falls outside of the previously established mask – if a raster cell is located outside the initial study area or within one of the exclusion zones, it will be assigned a 0. It will not be sampled. ➢ While Litterati could assign a value of 1 to every other raster cell, to reduce bias in our sampling design, the IPR allows for the application of a priori knowledge. ➢ For example: Litterati assumes that litter is more likely to accumulate near walkable roads, pathways, and points of interest (PoIs). Furthermore, these are the places where researchers are most likely to have unfettered access. ○ Using the Distance Allocation and Rescale by Function Tools in ArcGIS Pro, Litterati is able to create a raster map with values between 0 and 1 with the highest sampling probability being assigned to the raster cells closest to any PoI.
○ Although this does increase the amount of bias in the Sampling Design, some bias is mitigated by allowing any PoI, regardless of category to influence the inclusion probability.
As we learn more about the relationship between litter and other explanatory variables, Litterati can create and more sophisticated IPRs.
Create the
Inclusion
Probability
Raster
Create
Spatially
Balanced
Points
The Create Spatially Balanced Points tool is then used to randomly distribute sampling locations across the study area.
Important Environment Settings:
➢ Mask: Study Area Minus Exclusion Zones
➢ Cell Size: Assigned 200 meters
○ Helps to ensure that sampling locations are spaced at least 200 meters apart (direct distance: as the crow flies).
➢ Spatial Reference: WGS 1984 Web Mercator (Auxiliary Sphere) ○ Units of Measure: 1 Meter
Snap Points to Nearest Walkable Path
➢ After the samples have been collected, they are then snapped to the nearest street or sidewalk to ensure that our researchers are able to access that location.
○ This helps to reduce the chance that a point is dropped on private property.
○ However, it does increase bias and may lead to some points being closer than 100 meters.
○ In cases like this, trained researchers should do their best to ensure transects for different survey areas do not overlap.
Data Collection: Assigning Survey
Area’s to Researchers
Spatial Criteria
➢ Uploading Survey Area’s to Litterati Partner Dashboard Application (see link) ➢ Evaluating the number of locations per person and the average distance (drive-time) between locations.
○ How do we give an accurate estimate to the cities to say you have 300 locations and X zones that have Y locations each?
○ You will need Z researchers to do this amount research in one time-frame.
Temporal Criteria
➢ Determination of Time of Day/Time of Week
○ During the week? Not on the weekends? Time of Day?
○ Other Concerns: Rain and other weather conditions, Street Cleaning Data Collection: Systematic Observations
of Litter at each Survey Area
Data Collection:
The goal of data collection is to be able to
accurately estimate the total litter per
square meter in the study area. Our toolkit
for data collection must be able to handle
various methodological approaches.
We have identified the ability to collect data
along transects as the lowest common
denominator amongst various
methodologies.
For the City Fingerprint Project, our
researchers will collect litter along two 1 x
100 meter transects.
Continued on next slide
Survey Areas:
At the start of each project, Litterati
researchers will be assigned a geographic
cluster of survey areas based on proximity.
The number of survey areas assigned to
each researcher is contingent upon the total
number of researchers available as well as
the the total number of survey areas for
each project*.
Once the researcher arrives at a survey area
(e.g. blue pin on map), they will be asked to
record litter using a specific data collection
methodology.
1. Transects are defined as 1 by 100 meter line adhering to the edge of a street, sidewalk, or curb. Each researcher will be asked to complete two transects per survey area. Each pair of transects must:
a. Be on opposite sides of a walkable street or path
b. Be in parallel to one another
c. Follow the same cardinal orientation (e.g. North, West, South, East)
d. And cover the same length (100 meters)*.
2. What information needs to be recorded before starting each transect?
a. The researcher should note the most predominant land-use types along that Transect (e.g. roadway, school, park, etc. – see example on right from CSIRO).
b. The researcher will also be asked to record a horizon photo at the start of each transect.
3. What happens if no litter is recorded in either transect? a. That’s alright! We will count that location as a zero.
4. Does it matter when a transect is completed?
a. Yes, each ANALYZE project should have a clear set of parameters for when a transect should be completed at a given sampling location.
b. This may be influenced by the research design, street cleaning schedules, inclimate weather, and other known
events.
Data Collection along Transects
Sample
Transect
Guides
➢ Create a 50m buffer around
each sampling location pin.
➢ Clip the streets and sidewalks
feature class to that buffer.
➢ Select street & sidewalk
features that do not intersect
with the sampling location and
delete those features.
➢ Create a copy of this feature
class.
➢ Run the CopyParallel python
function on the copy to create
two parallel lines 7.62m* away
from the street/sidewalk
centerline.
Data Collection
Recording Litter along Parallel 1 x 100 meter Transects Pro’s
➢ Easy to Complete in a Timely Manner
➢ Transects can be pre-assigned and easily
replicated as long as the same guide is
made available to subsequent researchers.
➢ Does not require much equipment, although
a 100 meter tape would help ensure that the
researcher is walking a straight line rather
than weaving in and out as they see litter.
○ A compass reading within the application
would also be useful, especially if it notified
the researcher when they were continuing in
the wrong direction based on previous
segments.
➢ It is difficult to extrapolate the average litter
per square meter in a given location*.
○ See Discussion in slide notes below.
➢ It is difficult to ensure that the researcher
has walked exactly 100 meters
➢ It is also difficult to ensure that the
researcher is creating parallel research
transects that do not overlap with each other
or transects from nearby survey areas.
➢ We do not currently have an adequate QA-
method in place.
Con’s
Data Collection Quality Assessment
➢Several steps are required to determine if a research transect was completed. ◦ Segment length meets requirements
◦ Minimum amount of required segments per location/task ◦ Quality of images organized correctly
◦ Quality of GPS coordinates, to accurate show researcher’s progression through transect Data Analysis: Report Deliverables
➢Establish a litter fingerprint and baseline for each city ➢Evaluate the relationship between litter and commercial corridors, storm drains, and illegal dumping hotspots
➢Extrapolate Litter per Square Meter along walkable paths in each city using Empirical Bayesian Kriging and Prediction (explored further on remaining slides) ➢Establish a litter fingerprint and baseline for each city ○ Raw COMB Breakdown (count, weight, and volume)
○ Normalized COMB Breakdown (count, weight, and volume) per m2* ■ Additionally, we ought to find a way to parse out Single-Use Plastics for Analysis ■ *These calculations are dependent upon knowing the number of 1 X 1 meter grid cells that have a walkable street or path passing through
➢This analysis will be made available in both the report (static) via the Dashboard(s) (dynamic).
Litter Fingerprint & Baseline
Developing Key Performance Indicators
(KPIs): Litter per Square Meter
a priori (theory-driven)
➢ This KPI threshold for could be pre-determined by
existing theory and research on how much litter is
too much litter.
➢ This version of the KPI ought to be the default for
a project.
➢ During Year 1 of a project, this KPI will inform the
Data Science Dashboard. After the baseline is
established, it is up to the client whether they
would like to continue with this KPI Threshold or
build a custom data-driven KPI Threshold.
a posteriori (data-driven)
➢ Once we establish a baseline for the amount of
litter per square meter in the study area, we can
operationalize how much litter to comparatively
too much litter.
○ Option A: Setting the KPI threshold has 1 or 2
standard deviations above the mean.
○ Option B: Using Natural Breaks to Classify
the amount of litter per square meter.
Geospatial Analysis of Litter and
Places of Interest
➢Evaluate the relationship between litter and commercial corridors, storm drains, and illegal dumping hotspots
○ Chart/ Map the correlation between litter and distance to nearest commercial corridor, storm drain, and/or illegal dumping hotspot.
Empirical Bayesian Kriging with
Prediction (Regression)
➢Extrapolate Litter per Square Meter along walkable paths in each city using Empirical Bayesian Kriging* and Prediction.
○ *See appendix for several slides illustrate how the results of EBK can be spatially joined with walkable paths to create an extrapolated litter heatmap.
➢Over the course of the City Fingerprint Project, we will need to devise a regression model that will help predict the litter per square meter in each study area with greater statistical power. Here are some steps we can take:
○ Consolidate a list of potential explanatory variables (spreadsheet in progress) ○ After Phase 1 Data Collection is Complete:
■ Perform an Exploratory Regression Analysis in ArcGIS Pro for each Study Area. ■ Compare the results
■ Test models across study areas
METHODOLOGY
This study evaluates the effects of introducing prevention signs in four locations, known to have litter issues.
In each location, the whole area was sampled. Using the Litterati App, researchers documented all pieces of litter larger than one centimeter. Organic waste was not
documented. Researchers also documented waste
infrastructures (bins, ashtrays, containers) and hotspots of litter located in the study area.
The research process was repeated a total of eight times, each Thursday between May 5th and June 23rd 2022 to
better understand the effect of these new signs on litter in the surrounding area.
Our Litterati mobile phone application enables researchers to record geotagged and time-stamped images of litter,
illegal dumping hotspots, and waste infrastructure.
Prior to data collection, researchers selected by Litterati underwent training (link) to review this protocol as well as how to use the Litterati app. The researchers were then assigned two research locations and asked to record litter over the entire area.
Data Collection
METHODOLOGY
Once collected, the data is then assessed to determine if all relevant locations were sampled, and all images were
correctly uploaded and registered according to their
specific location. Each image is then annotated by a human according to the Litterati taxonomy (link).
These results are then used to validate other data by the machine learning items and added other items that were
not defined or picked up by computer vision model, to
understand the performance of the models please refer to the computer vision model performance section.
Litter AIData Annotation
Litterati has a defined taxonomy of Category, Object, Material and Brand (link). As users consistently label their litter images with OBJECT, MATERIAL and BRAND, the LitterAI is trained to better predict these specific labels in future images. General Categories (ex. Drink, Food, Personal Hygiene) are derived from the different combinations of Objects and Materials. Where possible, a proxy weight and volume value is applied to each combination. The combinations of Objects and Materials are then mapped to international standards such as CSIRO, OSPAR, UNEP-IOC, NOAA etc., as well as data sheets from organizations like Break Free From Plastic (BFFP).
Litterati’s computer vision model has a mean average precision of 80% across 39 classes of Object-Material combinations such as Plastic-Bottle, Aluminum-Can, and Cellulose Acetate-Cigarette Butt. These 39 classes represent over 80% of the typical identifiable litter found in the urban environment by # of items.
STUDY LIMITATIONS
Research - Limitations
● Researcher Bias
○ As the project unfolds, it is possible that researchers became more attuned to the presence of smaller pieces of litter while collecting data.
○ Also, the orientation and quality of the image recorded by each researcher affects the ability of LitterAI to predict and classify the type of litter in that image. ● Litter annotation
Litter data recorded during this study was split into the two main groups:
○ Identifiable litter includes major objects that we classify with high confidence such as bottles, cans, and cigarette butts.
○ Unidentifiable litter includes small pieces of plastic and paper and are characteristically challenging to classify using models & human annotators.
Data Sources
DATA SOURCES
Secondary Data
External Datasets
• OpenStreetMap:
• Points of Interest (POI) – North America (link)
• Highways – North America (link)
• SafeGraph's Places (POI Data) (link)
• American Community Survey Datasets (link)
Litter Data
Dashboard:
● Click here to view the City Fingerprint Project - Data Dashboard (arcgis.com).
● Download filtered Litter Data by clicking the download button in the bottom left corner of the “Selected #Items out of Total” panel.
Project Data:
● Click here to download Litter Data spreadsheet
● Click here to download iLPM Spreadsheet
JOIN US.
WE’VE GOT A
PLANET TO CLEAN