City Council Memorandum
TO: HONORABLE MAYOR AND CITY COUNCIL DATE: JULY 19, 2022 FROM: INNOVATION AND TECHNOLOGY DEPARTMENT WARDS: ALL SUBJECT: MEASURE Z – CITYWIDE DATABASE DISASTER RECOVERY IMPROVEMENTS ISSUE:
Authorize the purchase of six VxRail servers and VMWare Software from Dell Marketing LP for Citywide database disaster recovery improvements, pursuant to Purchasing Resolution No. 23812, Section 602(f), in the amount of $392,499.18.
RECOMMENDATIONS:
That the City Council:
Authorize the purchase of six VxRail servers and VMWare Software from Dell Marketing LP under the NASPO ValuePoint Cooperative Agreement for Citywide database disaster recovery improvements in the amount of $392,499.18; and Authorize the City Manager, or designee, to execute all documents pursuant to this purchase including making minor and non-substantive changes.
BOARD RECOMMENDATIONS:
This report is published on July 7, 2022, for the July 19, 2022, City Council meeting.
On July 11, 2022, the Board of Public Utilities will hear this item for consideration:
That the Board of Public Utilities recommend that the City Council approve an expenditure in the amount of $196,249.59 for the utility billing system database disaster recovery improvements in accordance with Purchasing Resolution No. 23812, Section 602(f).
To obtain this pricing, the purchase needs to be made by July 31, 2022. Therefore, if the Board of Public Utilities approves the expenditure, staff requests that the City Council approve this item at the July 19, 2022, City Council meeting.
BACKGROUND:
The City's datacenters are responsible for storage, processing, and the distribution of large amounts of Citywide data (e.g., via databases, enterprise applications, billing, and financial systems, etc.). The amount of data processed and stored within the City is continually expanding. The City currently operates a database environment from its primary datacenter that needs a hardware upgrade.
Public Utilities’ (RPU) and Innovation and Technology (IT) staff developed a Technology Continuity of Operations Plan, by reviewing the impacts and requirements for technology availability for all RPU major technology systems and City systems that exist in the Oracle environment. The team found, in certain disaster scenarios, the utilities billing system has a recovery time that does not meet RPU’s operational expectations and needs. In the current architecture, data is backed up daily, which meets the needs, but a weakness exists in restoration time from backup for a complete data center failure (e.g., fire, flood). The time required to restore the current environment from a complete data center failure would impose significant business impacts to the City. Widespread, internal and public-facing, service disruption may occur if City Hall’s datacenter experiences an emergency.
In addition, the City’s central storage system and some servers in the overall City and RPU database environment are no longer supported by the manufacturer. Expanding capacity and resolving these deficiencies requires upgrades to the primary data center database server equipment, including the disaster recovery site, to improve the City' s continuity of operations and providing automated fail over of core critical services. The Budget Engagement Commission and the City Council allocated funding through Measure Z in 2017 to address this critical infrastructure gap, to maintain the City’s digital infrastructure, and improve resiliency to support City operations during emergencies.
The major citywide applications benefiting from this shared technology infrastructure include:
EnQuesta Utilities Billing system
Oracle UWAM Utilities and Citywide Asset Management and Work Order system 311 Siebel Customer Relationship Management system
Posse Citywide and RPU Permitting system
M5 Fleet Management system
DISCUSSION:
The new server architecture bundles servers, storage, and virtual machine technologies into one package. This creates the next generation of data center virtualization, called Hyperconverged Infrastructure. This would modernize outdated servers, storage and provide the City with Multi-Datacenter Active / Active resiliency. This project is imperative to the success of the City’s continuity of operations, providing additional redundancy, efficiency, cybersecurity, and increased processing speed of citywide applications for users.
In addition, the new hyperconverged infrastructure will provide the City with Cloud-like capabilities that will allow the City to consolidate isolated datacenters while enhancing security, governance, reduce server quantities and software license costs while accelerating service delivery to City operations.
This new hyperconverged infrastructure will support the existing Citywide IT database server infrastructure. Dozens of applications rely on this database server infrastructure, most importantly the utility billing system. Furthermore, this platform aligns with the City’s data center modernization efforts. These servers will provide the following benefits:
Replace central storage with faster and distributed solid state storage Enable full utilization of the City’s failover data center to mitigate the impact of a City Hall Datacenter outage Replace and consolidate older servers
Lower total cost of ownership
Enable better organizational agility
Enable faster resource provisioning
Improve cybersecurity by accommodating isolated virtual tenants, consolidate isolated department datacenters, improve efficiency, reduce duplicate hardware, software licenses and streamline infrastructure management across all departments Provide a scalable datacenter infrastructure, software, and security to support future City and Public Utilities’ smart cities technology initiatives Through research, it was discovered that the City could expedite the purchase by procuring the servers and necessary software required to operate this new infrastructure directly from Dell Marketing LP, through the NASPO ValuePoint Cooperative Purchasing Agreement. The NASPO multi-state cooperative purchasing program provides pre-negotiated discounts to participating agencies. This cooperative purchasing program has been competitively bid nationally. Purchasing Resolution No. 23812, Section 602(f), states, “Competitive Procurement through the Informal and Formal Procurement process shall not be required in any of the following circumstances:…(f) When Cooperative Purchasing is available and undertaken or when Goods can be obtained through Federal, State and/ or other public entity pricing contracts or price agreements." The Purchasing Manager concurs that the recommendation to approve is in compliance with Purchasing Resolution No. 23812, Section 602(f).
STRATEGIC PLAN ALIGNMENT:
The item contributes to Strategic Priority No. 6 Infrastructure, Mobility & Connectivity and Goal No. 6.2. - Maintain, protect, and improve assets and infrastructure within the City’s built environment to ensure and enhance reliability, resiliency, sustainability, and facilitate connectivity.
The item aligns with each of the five cross-cutting threads as follows:
Community Trust – Upgrading the data center improves the City’s continuity of operations, system reliability, and processing speed which all serve the greater public good.
Equity – This project benefits multiple public-facing systems such as Utility billing, 311 Customer Relationship Management system and the Posse Citywide and RPU Permitting system.
Fiscal Responsibility – This upgrade improves the City’s fiscal resiliency given a data center outage. In addition, the purchase is being made utilizing the NASPO multi-state cooperative purchasing program which provides pre-negotiated discounts to participating agencies. This cooperative purchasing program has been competitively bid nationally.
Innovation – This upgrade to a Hyperconverged Infrastructure creates the next generation of data center virtualization modernizing outdated servers.
Sustainability & Resiliency – This upgrade is in direct response to the technology continuity of operations plan. It is imperative to the success of the City’s continuity of operations, providing additional redundancy, efficiency, cybersecurity, and increased processing speed of citywide applications for users.
FISCAL IMPACT:
The total fiscal impact of this action is $392,499.18. Sufficient funds are budgeted and available as shown in the table below:
Fund
Program
Account
Amount
Measure Z Capital Fund
Data Center/Disaster Recovery Improvements
9914009-462308
$196,249.59
Electric Fund
Public Utility Business Support
6004000-421000
$196,249.59
Total:
$392,499.18
Prepared by: George Khalil, Chief Innovation Officer
Certified as to
Availability of funds: Kristie Thomas, Assistant Chief Financial Officer Approved by: Michael D. Moore, Interim City Manager
Approved as to form: Phaedra A. Norton, City Attorney
Attachment:
Dell Technologies Quote No. 3000121985807.1
Disaster Recovery Dell Purchase ● Page 2