HomeMy WebLinkAboutDS Canal Broadband Feasibility Study Final Report____________________________________________________________________________________ FOR CITY CLERK ONLY Council Meeting: February 3, 2025 Disposition: Received Report Agenda Item No: 6.a Meeting Date: February 3, 2025 SAN RAFAEL CITY COUNCIL AGENDA REPORT Department: Digital Service and Open Government Prepared by: Vinh Pham, Digital Infrastructure Manager Sean Mooney Director, Digital Service & Open Government City Manager Approval: ______________ TOPIC: CANAL BROADBAND FEASIBILITY STUDY FINAL REPORT SUBJECT: CANAL BROADBAND FEASIBILITY STUDY INFORMATIONAL REPORT RECOMMENDATION: Staff recommends that the City Council receive the report. BACKGROUND: In June 2019, the Marin County Board of Supervisors approved the development of a Digital Infrastructure Strategic Plan to address digital and broadband equity challenges in Marin County made evident by the COVID-19 pandemic. The strategic plan, known as Digital Marin, included a roadmap to advance broadband deployment and digital adoption across the County. Digital Marin was adopted by the Board of Supervisors in February 2022 and supported by the San Rafael City Council by resolution in September 2022. One of the identified projects for Digital Marin was the “Canal Neighborhood Wi-Fi Project,” a partnership between the City, County of Marin, San Rafael City Schools, Canal Alliance, and community partners to bring free public Wi‐Fi to the Canal neighborhood. In Fall 2020, the Canal Wi-Fi network was deployed and continues to provide free Wi-Fi to the neighborhood, with an average of 600 visitors per day. Despite the success of the Canal Wi-Fi program, public Wi-Fi is not deemed sufficient to meet the broadband connectivity needs of multi-family households in the neighborhood. A permanent broadband infrastructure is recommended by the Digital Marin plan to reduce costs and increase access to the Canal Neighborhood. In September 2022, the City of San Rafael, in partnership with Marin County, was awarded a Local Agency Technical Assistance (LATA) grant by the California Public Utilities Commission (CPUC) for $258,620 to begin the process of addressing permanent broadband infrastructure needs for the Canal neighborhood. The grant-funded project included data collection, market analysis, and strategic plan SAN RAFAEL CITY COUNCIL AGENDA REPORT / Page: 2 recommendations to analyze the Canal neighborhood, identify needs, quantify demand, calculate costs, estimate fees, and provide recommended project parameters and strategies to support the development of a shovel-ready infrastructure project. In May 2023, the City contracted with EntryPoint Networks to provide project leadership, management, design, and strategic planning to deliver the following work products: 1. Local data collection, including geospatial files with resident survey data for broadband availability, price, and performance. Resident and property owner surveys were developed and completed to measure broadband satisfaction, experience, and affordability for residents. This data was collected to inform project planning and activities. The City partnered with Canal Alliance and Voces del Canal to support engagement and outreach efforts to community members in the Canal, build trust, and assess community needs. The team generated 609 surveys that were collected both in person at community events (Movies in the Park) and distributed online in coordinated outreach with community partners and trusted messengers in the community. 2. Local broadband market data and analysis related to availability, price, and performance for residents, particularly those residing in multi-dwelling living units, were collected to inform project planning and activities. 3. A high-level design, including material and labor cost estimates for a fiber to the living unit solution, and feasibility modeling, was developed that can inform municipal broadband funding applications. 4. Internet measurement data collected using technology developed by the University of Chicago and branded as NETRICS in the Canal Neighborhood. The methodology used for collection involved sampling randomized speed tests over a period of time to inform potential funding activities. 5. A Final Report providing explanatory narrative, findings, outcomes, and recommendations to inform municipal policies, strategies, and potential activities capable of supporting broadband improvements for the San Rafael Canal Qualified Opportunity Zone. ANALYSIS: Figure 1 San Rafael Qualified Opportunity Zone. The Canal neighborhood in San Rafael has significantly lower household incomes than surrounding neighborhoods. The demographics of the neighborhood at the focus of this study (Census Tracts 1122.03 and 1122.04) are designated as a Qualified Opportunity Zone area, with a median household income of SAN RAFAEL CITY COUNCIL AGENDA REPORT / Page: 3 $49,000 compared to a median household income for Marin County of $121,700. These two census tracts have a population of 8,024, representing 13.6% of the City’s total population of 59,000. 89% of residents in this area identify as Hispanic or Latino. Figure 2 Internet availability in the Canal Neighborhood. What We Learned about Canal Neighborhood Internet Availability Despite federal and state broadband map designations identifying the neighborhood as adequately “served” by incumbent provider Comcast, data collected via online and hard copy surveys for this report found households that were not provided with the level of service commiserate with federal standards. Comcast has announced technical upgrades intended to improve the area’s online subscriber experience, however, our study discovered: • 50% of unserved respondents would use the Internet for education • 91% of homes with children do not experience modern internet speeds of 100 megabits per second (Mbps) download speed and 20 Mbps upload speed or greater • 44% of unserved households indicate that the monthly cost is the main reason they are unserved • Unserved, underserved and served: o 62% of households are categorized as ‘unserved’ o 28% of households are categorized as ‘underserved’ o 10% of households are categorized as ‘served’ • 88% of respondents with Internet service want more providers in the Canal • 59% of unserved people listed some barrier involving cost to be the reason they are unserved SAN RAFAEL CITY COUNCIL AGENDA REPORT / Page: 4 “Last Mile” Grant Opportunities for Broadband Funding The term “Last Mile” (in the context of broadband telecommunications) refers to the final leg of a broadband network that connects local, city, or county-owned resources to the state-provided broadband network (referred to as the “Middle Mile).” Current federal and state funding opportunities primarily focus on supporting “Last Mile” connections to the “Middle Mile” fiber network provided by the State of California. The Federal Funding Account, administered by the California Public Utilities Commission (CPUC), is a $2 billion grant program for “Last Mile” broadband infrastructure projects to connect unserved Californians. Applications are open to cities, counties, tribes, non-profits, and broadband service providers. The 2023 application window closed on September 29, 2023, and the program received 484 grant applications within the application window. At least two applications were received for every county in the state, with a total of more than $4.6 billion in requests to fund broadband infrastructure projects to connect unserved Californians. AT&T submitted 250 projects totaling an estimated $2.6 billion for funding, some of which specifically included areas of Marin County and San Rafael. However, applications from AT&T and other broadband providers excluded any improvements in the Canal neighborhood, as shown on the map below, most likely because the Federal Communications Commission (FCC) has deemed this area “served” by the existing service provider, Comcast. These findings support the conclusion that a strategic municipal plan coupled with public funding opportunities is advisable to narrow the digital divide for members of the Canal neighborhood. SAN RAFAEL CITY COUNCIL AGENDA REPORT / Page: 5 Figure 3 Map of AT&T FFA project areas outlined in red. The Canal neighborhood was excluded. From 2023 to 2024, the primary “Last Mile” funding opportunity for underserved communities was the Broadband Equity, Access, and Deployment (“BEAD”) Program which provided $1.8 billion to expand high-speed Internet access in California. The Canal Neighborhood was deemed not eligible for BEAD funding because the Federal Communications Commission has determined that each property address in the neighborhood is adequately “served’” by existing broadband providers. “Served” broadband connections are defined as download speeds of at least 100 Mbps and upload speeds of at least 20 Mbps. The Challenge Process Resident, market, and internet measurement data from the City’s Canal Broadband feasibility project were collected and submitted through the CPUC’s BEAD Program Challenge Process by The City of San Rafael and EntryPoint in coordination with the County of Marin’s Digital Marin team in an attempt to overturn the FCC’s determination that the Canal is adequately “served” and make broadband funding available for the area. The neighborhood’s incumbent cable operator (Comcast) submitted a rebuttal to the City’s challenge prompting the California Public Utilities Commission to overturn all submitted challenges, concluding that existing services provided by Comcast are sufficient, and successfully blocking any potential funding from the BEAD program for the Canal neighborhood. SAN RAFAEL CITY COUNCIL AGENDA REPORT / Page: 6 Figure 4 Photos taken of the Coaxial Plant during data collection in Q3 of 2023. The challenge process likely benefitted area residents because the local cable operator performed a technology upgrade in the Canal neighborhood to improve reliability and upload speeds just before the challenge window opened and data collection commenced. Other funding opportunities may become available soon, and the City is looking into options for local connectivity to the state’s “Middle Mile.” The Final Report prepared by EntryPoint provides an explanatory project narrative, key project findings, activity outcomes, and recommendations to inform strategies, and potential future activities capable of supporting broadband improvements for the San Rafael Canal neighborhood and other underserved areas with broadband gaps. Additionally, as funding becomes available, EntryPoint developed a proposed broadband network design and map that proposes how to build out a City-owned fiber infrastructure in the Canal neighborhood connected to the state’s “middle-mile” fiber network. This tool will assist in future grant applications and potential opportunities to connect multi-family properties once a City-owned network is in place. Next Steps: The Digital team has outlined the following next steps from this effort to help address digital equity concerns in the Canal neighborhood. 1. Pursue Grant Funding Opportunities – We will continue to pursue grant funding opportunities to connect to the state’s “Middle Mile” network and build out infrastructure to support a fiber network in the Canal neighborhood in coordination with existing Capital Improvements in the neighborhood. 2. Citywide Broadband Planning – We are currently working in coordination with the Department of Public Works to inventory all City-owned fiber assets in tandem with a Citywide asset management project. We will be scoping a Citywide broadband plan to provide guidelines and roadmaps for City-owned broadband infrastructure. 3. Expand and Promote Canal Wi-Fi Public Wi-Fi Network – In 2024, the City took ownership of the Canal Wi-Fi public Wi-Fi network (built in 2020 with the County of Marin) and is currently developing a plan to upgrade existing equipment, expand the network, and promote it to the community. FISCAL IMPACT: This is an informational report, and as such, there no direct fiscal impact associated with the report. The California Public Utilities Commission (CPUC) awarded the City a total of $258,620 in Local Agency Technical Assistance grant funds to conduct the feasibility study. OPTIONS: The City Council has the following options to consider on this matter: SAN RAFAEL CITY COUNCIL AGENDA REPORT / Page: 7 1. Receive the report. 2. Take no action. RECOMMENDED ACTION: Staff recommends that the City Council receive the report. ATTACHMENTS: 1. San Rafael Canal Broadband Feasibility Study and Market Report 2. San Rafael Canal Neighborhood Network Design San Rafael Canal Broadband Feasibility Study and Market Report January 2025 THANKYOU This Broadband Feasibility Study and Market Report was commissioned by the City of San Rafael and funded by a Local Agency Technical Assistance grant from the California Public Utilities Commission. We would like to thank all of the partners that contributed to the project. We especially appreciate the invaluable collaboration and support provided by our local partners. Their commitment to improving our community's broadband access and affordability has been instrumental in collecting the data used in this report. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 1 Canal Broadband Feasibility Study and Market Report Contents Overview .......................................................................................................................................... 2 Problem Summary ...................................................................................................................... 4 Fiber Optic Infrastructure ........................................................................................................ 4 Affordability ................................................................................................................................... 4 Speed & Reliability ........................................................................................................................ 5 Speed Test Data Detail ............................................................................................................... 6 Comcast’s Mid-Split Upgrade ................................................................................................... 7 Competitive Options for Internet Service Providers .......................................................8 MDU Broadband Barriers in the Canal neighborhood ...................................................8 Demographics and Housing .................................................................................................... 11 Data Analysis ................................................................................................................................ 13 Solution ........................................................................................................................................... 13 Financing ....................................................................................................................................... 15 Financial Feasibility.................................................................................................................... 15 Projected Monthly Cost to Subscribers .............................................................................. 16 Take-Rate ....................................................................................................................................... 16 Network Management and Operations ............................................................................. 17 Key Project Findings .................................................................................................................. 18 Project Outcomes ....................................................................................................................... 21 Recommendations .....................................................................................................................23 San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 2 Canal Broadband Feasibility Study and Market Report Overview In September of 2022 the City of San Rafael was awarded a Local Agency Technical Assistance (LATA) grant from the California Public Utilities Commission for an amount not to exceed $258,620. LATA provides funding support for local agencies to expand broadband service to unserved and underserved Californians. The LATA project in San Rafael focused on gathering data to inform the development of a municipal approach to improve broadband service in areas where additional funding or competitive forces are needed. The Canal neighborhood in San Rafael was strategically selected as the project focus area because, as a historically disadvantaged community, the neighborhood has been subjected to a pattern of disinvestment on the part of the area’s incumbent broadband providers. The broadband market in the Canal neighborhood is a microcosm of the nationwide market failure that has resulted from providers concentrating their broadband investments in more affluent neighborhoods. Disadvantaged areas, like the Canal neighborhood, are often left to languish with a single terrestrial provider, aging legacy infrastructure, and service levels that do not meet regulatory requirements in terms of speed or reliability. Most broadband policy today assumes that the private market model can address existing market failures, bridging the digital divide, and achieving the goals of ubiquitous and affordable broadband internet by incentivizing improvements using targeted public subsidies with “light touch” regulation. Public broadband programs focus public infrastructure investments and subsidies on "unserved" locations flrst, with any remaining program funds devoted to "underserved" locations. Locations considered "served" by incumbent providers are excluded from virtually all infrastructure funding programs, either explicitly or because these areas are simply not feasible within program guidelines. The Broadband, Equity, Access, and Deployment (BEAD) program exemplifles how the categorization of locations as "unserved", "underserved", and "served" will infiuence broadband outcomes for the foreseeable future. BEAD is the single largest public broadband investment in United States history. The program provides $42.5 billion in broadband infrastructure funding focused on installing broadband directly to unserved and underserved addresses. Each location’s funding eligibility was initially based on data submitted to the Federal Communications Commission (FCC) by incumbent providers and provided for public review in a National Broadband Map. Based on the data submitted in 2023 by phone and cable companies, of the 16,865 they reported in San Rafael, only 170 scattered addresses were eligible for any level of BEAD funding. Challenge processes were implemented to allow local governments, non-proflts, and broadband service providers to review and contest the designation of locations as "served" or "unserved" on the National Broadband Map to improve the data being used to allocate BEAD funding. Because most local governmental agencies do not have either the funding or expertise to challenge industry data or to develop broadband solutions capable of leveraging BEAD funding, the LATA program was established to assist local agencies to meet this need. With the expressed goal of developing a BEAD grant fundable broadband infrastructure improvement project for the Canal neighborhood, this LATA project focused on (1) collecting data to inform a BEAD challenge process to make the area eligible for funding, and (2) designing a solution aligned with BEAD San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 3 Canal Broadband Feasibility Study and Market Report requirements, the following LATA work products were completed: 1. Data Collection: a. Deliverable: Survey development, execution and flnal report in a statistically relevant sample size organized in Geographic Information System (GIS) or mapped dataset with authoritative broadband demographics, satisfaction, experience, and affordability information. b. Purpose: Inform future city actions, such as BEAD challenge processes, funding eligibility and affordability rates for municipally supported solutions. 2. Market Analysis: a. Deliverable: Detailed broadband market report for the Canal neighborhood and the larger San Rafael area. b. Purpose: Document and provide current market availability, including availability, service levels, reliability, and costs to inform municipal policy and solution development. 3. Improvement Design: a. Deliverable: Fiber optic to the living unit infrastructure design, including unit counts for materials and labor, and feasibility modeling for a municipal flber to the home or living unit infrastructure improvement. b. Purpose: Provide an adequate level of design and projected costs to inform a municipal flber optic improvement that could be funded through a grant award. 4. Internet Measurement Data: a. Deliverable: Performance and reliability data for the Canal neighborhood that complies with BEAD challenge processes. b. Purpose: Provide data to inform a BEAD challenge proving that Canal neighborhood service locations should be changed from ‘served’ to ‘unserved’ based on collected data. 5. Final Report / Strategic Plan: a. Deliverable: Final summary report describing the tasks performed, flnal deliverable summaries and project outcomes. b. Purpose: Report and document flnal project flndings for municipal leadership while also highlighting how project deliverables could be used to inform potential city actions to improve broadband access and affordability. This report summarizes the key factors contributing to the lack of digital access, network reliability, and broadband affordability for residents in the Canal neighborhood to inform municipal policy, infrastructure, and business model recommendations capable of solving the broadband disinvestment dilemma. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 4 Canal Broadband Feasibility Study and Market Report Problem Summary Residents in the Canal neighborhood of San Rafael, CA face the following barriers to digital access: 1) Residents in the Canal neighborhood do not have access to flber optic infrastructure 2) Average speeds sampled from Aug 1, 2023 – July 31, 2024, do not meet the current FCC guidelines of 100 x 20 Mbps. 3) Existing options are expensive 4) Residents do not have competitive choices for internet service providers. 5) Each of the above problems is more pronounced for residents living in apartment buildings, which represents a majority of residents in the Canal neighborhood. Fiber Optic Infrastructure Analysis for this report did not identify any residents in the Canal neighborhood that have access to flber optic connectivity. Comcast is the dominant internet service provider (ISP) in the Canal neighborhood and the company connects individual living units with coaxial cable. Only 26% of last mile subscribers in the City of San Rafael have access to flber optic internet connectivity and that is through AT&T. Those residents with access to flber typically live in single-family detached homes and duplexes in medium to high-income census tracts. A limited number of business addresses have access to flber. Affordability In San Rafael’s Canal neighborhood, most residents and businesses subscribe to wireline internet services from the cable operator (Xfinity/Comcast) or from the incumbent telephone operator (AT&T) for DSL. Residents in the Canal neighborhood will pay just under $70 for Xflnity’s lowest speed option at 150 x 23.37 Mbps. Many communities in the country where there are multiple flber providers are paying at or near $70 for Gig speeds. For example, Google Fiber prices their Gig product at $73. Sonic Fiber in nearby cities advertises symmetrical Gig speeds for $59.99. Xfinity Residential Xfinity advertises the following residential ISP services in San Rafael. Note that all rates, except for Standard Pricing, are available exclusively to new customers. Comcast is the primary high-speed internet provider for 18,295 of San Rafael’s 24,701 housing units, and 96% of all multi-family units with eight or more units. This analysis presumes most residents in these homes are existing Comcast customers paying Standard Pricing for “internet only” service. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 5 Canal Broadband Feasibility Study and Market Report Speed (Mbps) [Down / Up] 1-12 Mo Rate [Contract Required] 13-24 Mo Rate [Contract Required] 24-36 Mo Rate [Contract Required Standard Pricing [+ Taxes and Fees] Pro Install [Fee] 150 / 23.37 $29.99 $67.00 $67.00 $67.00 $100.00 300 / 23.37 $40.00 $40.00 $79.00 $79.00 $100.00 500 / 23.37 $60.00 $60.00 $89.00 $89.00 $100.00 *800 / 23.37 $70.00 $70.00 $99.00 $99.00 $100.00 1000 / 23.37 $75.00 $75.00 $109.00 $109.00 $100.00 1200 / 40.25 $80.00 $80.00 $119.00 $119.00 $100.00 2000 / 306.2 $130.00 $130.00 $130.00 $130.00 $100.00 Taxes and Fees often represent an additional (20%) of Standard Pricing Shared Network – Speeds are “Up To” and are not guaranteed. Speeds are not Symmetrical. Modem with Managed WiFi - $15.00 per month / $25.00 per month after 24 months. Data Caps apply. * Larger families with school-aged children living in apartment buildings often require higher speeds to meet their in- home computing demands, particularly during peak hours. 800/20 Mbps would not be an unrealistic benchmark: $124 monthly with modem. AT&T Residential AT&T advertises the following residential ISP services in Northern San Rafael: Speed (Mbps) [Down / Up] Standard Pricing [+ Taxes and Fees] Equipment [Fee] 300 / 300 $60.00 $150.00 500 / 500 $70.00 $150.00 1000 / 1000 $85.00 $150.00 2000 / 2000 $155.00 $150.00 5000 / 5000 $255.00 $150.00 Taxes and Fees often represent an additional (20%) of Standard Pricing Shared Network – Speeds are “Up To” and are not guaranteed. Speed & Reliability According to mLAB data, the average broadband service in San Rafael is not meeting the FCC Deflnition of Broadband at 100/20 Mbps. Xfinity/Comcast’s average speeds in San Rafael are 106.03 Mbps download / 13.21 Mbps upload. AT&T’s average speed in San Rafael is 70.75 Mbps download / 34.27 Mbps upload. August 1, 2023 – July 31, 2024, Sample Size 280,136 Information in the table above is from Xfinity’s website. Information in the table above is from AT&T’s website. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 6 Canal Broadband Feasibility Study and Market Report Speed Test Data Detail mLAB is an academic group that provides authoritative data from speed tests on a city-by-city basis across the United States. Academic and scientific research organizations rely on mLAB data. Every time an individual runs a speed test through an affiliate of mLAB, the data is saved in Cloud Storage hosted by Google and made available to the public. The data below is the speed test results in the City of San Rafael from August 1, 2023, to July 31, 2024. The sample size for this 12-month period includes 280,136 speed tests. FIGURE 10 Speed Test Data Sample Size FIGURE 10 shows that there were over 280,000 download and 0ver 264,000 upload speed tests completed inside the City of San Rafael over 1 a one-year period using the mLAB’s platform (between August 1, 2023, and July 31,2024). The average download for all speed tests performed over that time was just under 100 Mbps with the uploads averaging around 15 Mbps as shown in figure 10. The average speed delivered by Xfinity/Comcast in San Rafael is 106.03 Mbps download / 13.21 Mbps upload. AT&T’s average speed in San Rafael is 70.75 Mbps download / 34.27 Mbps upload. Overall cumulative average speed in San Rafael is 94.48 Mbps download / 15.01 Mbps upload. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 7 Canal Broadband Feasibility Study and Market Report FIGURE 11 Speed Test Data Comparison between AT&T & Comcast FIGURE 11 shows the average results of the over 250,000 tests completed between August 2023 and July 2024 by provider. AT&T maintained an average download of approximately 70 Mbps and an upload average of approximately 30 Mbps. Comcast maintained an average download of just over 100 Mbps and an upload average of approximately 12 Mbps. It is important to note that this data represents the results for the entire City, not just the Canal neighborhood. There is currently no publicly available tool capable of collecting speed test data for just the Canal neighborhood. Some testing devices, using technology developed by the University of Chicago and branded as NETRICS were installed in the Canal neighborhood to collect a sampling of randomized speed tests over a period. Information specific to the data set collected by this activity is included later in this report. Comcast’s Mid-Split Upgrade On July 1, 2024, Comcast announced a “mid-split upgrade” in the San Rafael Canal neighborhood, seven days before the BEAD Challenged opened. The mid-split upgrade allowed for more spectrum to be dedicated to the upstream path, which means customers were able to access faster upload speeds. This is particularly beneflcial for activities like video conferencing, online gaming, and uploading large flles. Improved Network Performance: By using 1.2 GHz of spectrum within the coaxial portion of the link, the mid-split upgrade enhanced overall network performance, making it more reliable and efficient. When San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 8 Canal Broadband Feasibility Study and Market Report Comcast notifled Marin County of this mid-split upgrade, they did not explain why this upgrade had not been made earlier for the residents of the Canal neighborhood. On 8/6/24, representatives of Marin County submitted speed test challenges in the California Challenge portal on behalf of San Rafael, based on Netrics data provided by Merit Networks, Justin Campbell. Marin County flled six challenges based on Netrics speed tests, which showed Comcast service falling below the FCC standard of 100/20 Mbps. Marin County is now evaluating the full Netrics dataset in collaboration with Justin Campbell (Merit) before commenting further on the impact of Comcast's mid- split upgrade. More information related to this data collection and analysis will be provided as a part of a future strategic plan that will utilize the flndings of this market analysis to inform a broader municipal strategy. Competitive Options for Internet Service Providers Options for internet access in the City of San Rafael are limited – and this is particularly the case for residents of the Canal neighborhood. As stated earlier, Comcast is the sole internet provider to 66% of all San Rafael living units and the sole ISP to 96% of all MDUs with 8 or more living units in the City of San Rafael. The Canal neighborhood consists mainly of multi-family dwelling units. As a result, Comcast (Xflnity) is the sole internet service provider to 100% of the greater Canal neighborhood, including Bahia and Spinnaker. Residents and businesses in the Canal neighborhood that are dissatisfled with cable offerings have no current options for terrestrial technology capable of delivering even a 25/3 Mbps experience, let alone 100/20 Mbps service. To obtain a flber optic connection, the new connection must be within a city block of an existing flber connection, or the resident or business must be willing to pay the amortization cost for the entire flber extension in a contract requiring monthly payments of $500 to $3,000 for two to flve years. MDU Broadband Barriers in the Canal Neighborhood Because the Canal neighborhood largely consists of Multifamily Dwelling Units (MDU’s), the existing infrastructure presents a speciflc challenge to residents due to performance and network reliability gaps. These gaps may stem from a number of sources including aging infrastructure, links which are shared by multiple users, jury rigged building wiring, unreliable wireless systems which are tethered to the coaxial cable or are independent of the cable network, as well as too many endpoints sharing a common hub that creates congestion during times of high utilization as shown in the following images taken during a recent site visit to the neighborhood. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 9 Canal Broadband Feasibility Study and Market Report Example of Infrastructure Neglect San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 10 Canal Broadband Feasibility Study and Market Report Additionally, there are a number of complications related to deploying new flber optic infrastructure into existing Multi-family Dwelling units (MDU’s). These include: 1) Some owners of multi-tenant environments are incentivized to block competitive offerings into their rental units because they receive revenue sharing agreements from incumbents for exclusive marketing rights to MDU residents. In April 2024, the FCC issued its Open Internet Order, which adopted bright-line rules that prohibit blocking, throttling, and paid prioritization. Prior to taking effect, the 6th District Court overturned the order, stating that the FCC overstepped its authority without authorization from Congress. Had it not been overturned; the rule would have prohibited exclusive access and service contracts in residential and commercial multi-tenant environments. 2) It may be difficult to deploy new flber infrastructure into old buildings because of construction barriers. 3) The economics of maximizing return on investment for private ISPs dictate that installation needs to be a relatively quick process, using relatively inexperienced technicians, and straight-forward installation methods. 4) The owners of multi-family units do not have strong incentives to invest in last mile flber for their existing facilities even though existing cable providers may offer expensive packages and/or poor reliability. There are 24,701 living units in the City of San Rafael, of which 4,427/18% are in apartment buildings with 8 or more units. There are 24,701 living units in the City of San Rafael, of which 4,427/18% are in apartment buildings with 8 or more units. Some owners of multi-tenant environments are incentivized to block competitive offerings into their rental units because they receive revenue sharing agreements from incumbents for exclusive marketing rights to MDU residents. It may be difficult to deploy new flber infrastructure into old buildings because of construction barriers. The economics of maximizing return on investment dictate that installation needs to be a relatively quick process, using relatively inexperienced technicians, and straight-forward installation methods. The owners of multi-family units do not have strong incentives to invest in last mile flber for their existing facilities even though existing cable providers may offer expensive packages and/or poor reliability. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 11 Canal Broadband Feasibility Study and Market Report Summary of Relevant Demographic & Broadband Data The Canal Area - Relevant Community Data The Canal Area comprises two neighborhoods of San Rafael, California, designated by the city as the "Canal Waterfront" and the "Canal." The Canal Area is bounded on the east by San Francisco Bay, on the north by the San Rafael Canal and on the south and west by Highways 101 and I-580 and by San Quentin Ridge. Land use is split almost evenly between residential and non-residential (commercial/industrial) uses.[1] San Rafael’s Canal Area represents a substantial portion of the low-income housing in Marin County. Further, the Canal Area is highly segregated with roughly a 90% Hispanic population and less than 5% white non-Hispanic population. According to the Canal Alliance 2023 Annual Report, as much as 72% of the population lives below the federal poverty line. The population density of the Canal neighborhood in San Rafael is 58,409 people per square mile. California's urban areas have a population density of 4,790 people per square mile and the state has a population density of 253.9 people per square mile. Demographics and Housing According to a 2021 San Francisco Chronicle article, the Canal Area represents a substantial portion of the low-income housing in Marin County, one of the most affluent counties in the United States. The Canal area is also one of the most segregated neighborhoods in Marin County, with more than a 90% San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 12 Canal Broadband Feasibility Study and Market Report Hispanic population and less than 5% white non-Hispanic population. The Chronicle article suggests that most of San Rafael's low-income residents live in the Canal Area. This area provides a location for workers who hold low-income jobs that are essential to the functioning of the community to live close to their place of employment. Although most residents in this area are low-income workers, most of the apartments in which they live are neither subsidized nor less expensive than comparable apartments elsewhere in San Rafael. Article: This neighborhood in Marin is the most segregated in the Bay Area (sfchronicle.com) The following demographic data was included in the County’s 2024 Speed Test Challenge: Canal Neighborhood Street Corner – San Rafael, CA (SF Chronicle Article: This Neighborhood in Marin is the Most Segregated in Bay Area, October 10, 2021 San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 13 Canal Broadband Feasibility Study and Market Report Data Analysis The broadband market data summarized in this report suggests that the Canal Neighborhood in San Rafael, California suffers from the effects of near monopoly status for the broadband service provider. There is only one terrestrial provider capable of delivering close to the 100/20 Mbps target and actual data provided in the report indicates that average speeds are below the FCC target. The lack of competition is likely contributing to lower service levels and higher consumer costs for broadband. Digital infrastructure in the Canal neighborhood is a critical necessity for full participation in the economy, education, community life, and healthcare. Reliable and robust digital infrastructure is a basic requirement for the functioning of City services and operations, from flnance to transportation to emergency services. Similarly, businesses require reliable and fast digital infrastructure to connect with customers, protect their supply chains and ongoing operations. Education and healthcare systems require digital infrastructure to connect with students or patients, to communicate between facilities, and ensure timely and appropriate services. Connecting to individuals from disadvantaged groups, either because of income, race, age, or language abilities, is even more critical to ensure these groups have full access and availability to beneflt from today’s digital applications. Service delivery models which optimize proflts rather than affordability and accessibility will inevitably create a digital divide. Due to the critical nature of digital infrastructure, ensuring a reliable and affordable network is a clear public policy concern and decisions focused on remedying the problem can have a far-reaching impact on all the systems that are important in a city for businesses and people. Solution Building a Sustainable Model that is Affordable and Available Improving Affordability The dominant national ISPs have developed a business model that is “rent seeking ” and sustained by controlling the infrastructure. Network control allows incumbents to impose premium pricing on network rents (ISP fees). The actions listed below can effectively overcome these “rent seeking ” practices and drive down the cost of access in a meaningful way. These include the following: 1. Separate and optimize the key cost components of digital access into the three main network categories: (1) Capital Infrastructure Investment, (2) Monthly Maintenance & Operations Expenses, and (3) Monthly Internet Access from ISPs and other digital services. 2. Apply for state and federal grants targeted to offset the cost of deploying new flber-optic infrastructure. 3. Fund digital infrastructure with public money and keep the infrastructure in the public or non-proflt domain. Services can be for-proflt and privately owned and operated. 4. Apply established utility operational models for funding, construction, operation, and fees and leverage established municipal utility powers, tax exemptions, and liability beneflts to drive costs down. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 14 Canal Broadband Feasibility Study and Market Report 5. Put downward pressure on price by enabling dynamic competition between service providers via an open access network model. 6. Allow households in multi-tenant buildings to share the infrastructure, maintenance, and operations costs. 7. Allow subscribers to pay off any cost for infrastructure and eliminate that line item once the infrastructure debt has been retired. 8. Leverage automation to lower operational expenditures. Unbundled Infrastructure and Services The dominant national ISPs bundle the infrastructure and services together to insulate the infrastructure owner from outside service providers. An open access model depends on unbundling or separating the primary functions and network costs into three buckets: 1) Infrastructure Capital Deployment, 2) Ongoing Network Operations, and 3) Services. To optimize each function and to enable the city to become a neutral host, it is important to unbundle the key network functions and costs. Public Infrastructure Persistent barriers to universal internet access, availability, affordability, and adoption are now public domain concerns. Internet access is a necessary feature of modern life—like other utility infrastructure—and can no longer be considered a luxury item. The incentives for private industry are not aligned toward resolving persistent gaps and the solutions advanced by private industry have not addressed these critical public needs or provided effective sustainable solutions. Informed public policies coupled with targeted public investments are needed to provide lasting solutions. These public policies must be informed by the fact that reliable internet is now necessary for access to educational systems, economic activities, healthcare, public safety systems, and many other cultural and societal interactions. Fiber-Optic Infrastructure as a Public Utility Managing Fiber-optic networks as a public utility makes sense because of the essential nature of the infrastructure. Utility frameworks, such as roads, water, sewer, storm drains, and electricity, exist to support essential functions critical for societal success. Providing digital access as a public utility will result in maximum service at the lowest possible cost for residents, businesses, and anchor institutions. Treating this as an amenity rather than a utility affects affordability, availability, and quality of service. Increased Competition Through Open Access Open access is a model that divides the infrastructure and services into two separate systems and then shares the infrastructure between multiple service providers, like road systems and airports. A key goal of an open access system is to lower costs and improve service by increasing choice and competition. For an open access system to realize its potential, it is critical for the infrastructure owner to be a neutral host of the infrastructure. The role of a neutral host is to control and manage the infrastructure without privileging one service provider over another. A true open access network depends on enabling robust shared infrastructure that is operated on a non - discriminatory basis. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 15 Canal Broadband Feasibility Study and Market Report Alignment with Users Residents, business owners, and visitors in the Canal neighborhood should receive maximum value for minimum cost, following traditional utility models. Goals should be established for leveraging this infrastructure to enhance livability, increase economic development activity, enable important anchor institutions like healthcare and education, and care for natural and human resources. As digital infrastructure becomes increasingly important to each of these things, the signiflcance of alignment between the network owner/operator and users also increases. Local Control Over Pricing and Reliability Local control over critical infrastructure allows for the needs of residents and business owners to infiuence policy and regulations. Today’s dominant ISPs are nationwide companies that are not organized to align the network with local needs and interests. Digital infrastructure can increase local value when it is owned and controlled by a local neutral host. The digital divide, education, economic development, public safety, and healthcare are all examples of variables that can best be understood and addressed locally. Control over network infrastructure will allow residents in the Canal neighborhood to leverage the power of the network. Economic Development Historically, economic development has followed investment in infrastructure for all major systems including transportation, water, sewer, and communications. Until now, municipalities have mostly remained independent of a governance role over digital infrastructure, allowing private companies to decide where they will build, what they will build, the cost of services, and the kind of innovation that will happen on these systems. However, due to gaps in affordability and availability, municipalities are increasingly taking a more active role in the governance of this infrastructure. Financing The most meaningful opportunity to improve affordability in the Canal neighborhood will come from accessing state and federal grants to offset the cost of deploying new flber-optic infrastructure and then to pass the savings on to subscribers in the form of lower monthly subscription payments. For this to work , the infrastructure would need to be in a governance structure that is either a public or non-proflt entity with a primary objective of improving the affordability and reliability of internet access for residents. Financial Feasibility The Canal neighborhood is an area with high urban density, allowing for a combination of both underground (buried) or above ground (aerial pole) infrastructure construction. An accurate projection of infrastructure costs requires a low-level design and an analysis of both an aerial flber and buried flber implementation. Aerial construction can be much faster but requires pole make-ready work. Underground construction can be more resilient to weather events and other accidental damage but takes more time to deploy and is typically more expensive. Ultimately, feasibility will depend on the quality and effectiveness of community engagement to educate residents on the value proposition of a locally controlled municipally sponsored network. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 16 Canal Broadband Feasibility Study and Market Report The cost summary below unbundles the core components of the flber infrastructure capital stack (1. Infrastructure, 2. Maintenance & Operations, 3. ISP Services) and is a realistic projection of costs on a per premise basis. The $9.99 cost for services assumes a competitive open access marketplace and is derived from experience with ISPs operating on other open access networks which have expressed a desire to provide similar pricing in new open access networks. Projected Monthly Cost to Subscribers Projected Monthly Subscription Cost Projected Residential Services Monthly Costs 100% Aerial 70% Buried /30% Aerial 100% Buried Infrastructure $28.00 $32.00 $35.00 Maintenance and Operations $25.00 $25.00 $25.00 ISP Services (Dedicated 1 GB Symmetrical) $9.99 $9.99 $9.99 Monthly Total $62.99 $66.99 $69.99 This next projection illustrates the savings to subscribers of accessing state or federal grants to cover the cost of infrastructure and passing those savings on to subscribers. The industry is accustomed to accessing public funding and then capturing the majority of the beneflt of that funding rather than passing the beneflt through to subscribers. Projected Monthly Subscription Without Infrastructure Cost Projected Residential Services Monthly Costs 100% Aerial 70% Buried /30% Aerial 100% Buried Infrastructure $28.00 $32.00 $35.00 Maintenance and Operations $25.00 $25.00 $25.00 ISP Services (Dedicated 1 GB Symmetrical) $9.99 $9.99 $9.99 Monthly Total $34.99 $34.99 $34.99 Take-Rate Take-rate refiects the percentage of premises which opt-in to taking a service and is critical to project success because the operational sustainability of a project depends on crossing a certain take-rate threshold to spread the common broadband infrastructure costs across a broad number of subscribers, therefore translating into a sustainable operating cost per premise. Achieving a viable take rate is primarily a function of creating value and then communicating that value to subscribers. Community engagement and digital literacy training will be important components for enabling successful outcomes. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 17 Canal Broadband Feasibility Study and Market Report Network Management and Operations The work required for network operations includes network monitoring, network management, outside plant repairs, and new customer installations. Network operations can be provided by a public body or by a third-party partner. The recommendation is to utilize third-party partners at least until the network is stabilized with an established and sustainable take-rate. At that time, the public body or non-proflt can evaluate whether it makes flnancial and operational sense to move operations. Outsourcing logical network responsibilities (customer support, network operations center (NOC), monitoring, and troubleshooting) to a network management partner generally is preferred. The enclosed model includes a budget of $25.00 per subscriber per month to cover the total cost of maintenance and operations. Key Project Findings San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 18 Canal Broadband Feasibility Study and Market Report Key Project Findings Market Broadband provider data submitted to the Federal Communications Commission (FCC) for the Canal Neighborhood reveals only one option capable of meeting the FCC ‘Broadband’ deflnition which requires a terrestrial connection with a minimum download speed of 100 megabits per second (Mbps) and a minimum upload speed of 20 Mbps. The entire City of San Rafael suffers from a lack of broadband market competition with the problem being particularly pronounced in apartment buildings: • There are 24,701 living units in the City of San Rafael, of which 4,427 (18%) are in apartment buildings with 8 or more units. • AT&T provides flber service to 6,406 (26%) units, of which 165 (4%) are in apartment buildings with 8 or more units. • Comcast is the sole internet service provider to: • 66% of all San Rafael living units, • 96% of all MDUs with 8 or more living units, and • 100% of the greater Canal neighborhood, including Bahia and Spinnaker. Conclusion: The San Rafael broadband market, including the Canal neighborhood, is subject to a virtual monopoly that is very pronounced in the Canal neighborhood. This lack of competition is contributing to lower service levels and higher consumer costs for broadband. Market data indicates that there is consumer demand for more choice, enhanced performance packages, upgraded reliability, and improved affordability. These market conditions are a direct result of the existing infrastructure investments. Private investments to address these gaps are highly unlikely in the absence of a municipal broadband plan that identifles potential gap funding and strategic planning capable of supporting choice, equity and affordability. Infrastructure Because the area largely consists of Multifamily Dwelling Units (MDU’s) supported by coaxial cable internet access, the existing infrastructure presents a speciflc challenge to the residents living in the Canal neighborhood because of performance and network reliability gaps. Coaxial cable installations commenced in Marin County in the 1970’s. The cable plant serving the Canal neighborhood today is a hybrid system that utilizes multiple types of conductors through various segments of the network to modern technologies which offer higher data rates. Onsite review of the area’s cable installation found coaxial cables which were antiquated and poorly maintained. Using substandard cables, networking San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 19 Canal Broadband Feasibility Study and Market Report technologies, and electronics used to transmit signals across the cables will result in subpar and unpredictable performance and reliability. Conclusion: The Canal neighborhood suffers from a pattern of disinvestment resulting in unpredictable performance and reliability. Meaningful, long-term solutions should include addressing area infrastructure needs in a sustainable manner. The Lived Experience Initial internet speed measurements and a resident survey of Canal neighborhood residents were collected in 2023. San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 20 Canal Broadband Feasibility Study and Market Report Survey speed data were placed into one of the following categories based on federal and state deflnitions: 1. Unserved: Below 25/3 Mbps, 2. Underserved: Between 25/3 Mbps and 100/20 Mbps, and, 3. Served: At or Above 100/20 Mbps. According to these deflnitions, most households in the Canal Neighborhood that participated in this 2023 study were not properly served: • 60% of households are ‘unserved’ • 29% of households are ‘underserved’ • 11% of households are ‘served’ Broadband is critical to a community’s ability to thrive and remain competitive in terms of education, economic development, talent retention, employment opportunities and population growth. The 2023 survey found that Canal neighborhood residents without broadband desire affordable service, and connected citizens believe that more provider options are needed. • 90% of homes with children do not experience modern Internet speeds of 100/20 Mbps or greater. • 56% of unserved respondents would use the Internet for education. • 90% of respondents with Internet want more providers in the Canal neighborhood. 68% of these also agree that the cost of Internet access is too high. • 53% of residents with no Internet access responded that they would pay $25 or less per month for service, indicating a need for a low-cost option in the Canal Neighborhood. Conclusion: Broadband is critical to the community’s ability to thrive and remain competitive in the areas of education, economic development, talent retention, employment opportunities and population growth. The data collected in 2023 found that the majority of residents were not receiving internet service levels that met the San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 21 Canal Broadband Feasibility Study and Market Report FCC’s broadband deflnition. Residents without broadband desire affordable service and connected residents believe that more provider options are needed. Project Outcomes Over the next couple of years, billions of dollars will fiow to states through the Broadband Equity and Deployment (BEAD) program to build the internet infrastructure that reaches communities without adequate (or any) access. This money will fiow through state broadband offices. The broadband industry, particularly internet service providers (ISPs) and related organizations, have demonstrated a “love-hate” relationship for BEAD and other public programs. Incumbent organizations “love” that they can potentially access public funding to extend their private investments into new areas or collect a subsidy for delivering services in their established markets, but they “hate” that these programs could potentially fund competition in the form of new infrastructure or service subsidies. This is of particular concern for them when a program could fund a new infrastructure where they are currently providing inadequate or unreliable internet service using legacy technologies and dilapidated infrastructure. The FCC's National Broadband Map was created to provide a comprehensive and accurate picture of broadband availability across the United States, with the immediate and primary goal of focusing BEAD and other public broadband programs on areas lacking broadband access and guide policy decisions to address the digital divide. The industry’s reaction to the development of the map becomes evident when reviewing the map’s history. In May of 2023 the map identifled more than 8.3 million U.S. homes and businesses that lacked access to high- speed broadband. A November 2023 map update adjusted this number to just over 7.2 million locations, and a May 2024 version shows 8.8 million locations that need better broadband service. There is no single reason for these up and down changes. A key factor that has driven these changes is that local stakeholders have challenged the availability of reliable internet services that meet the FCC deflnition for broadband. These challenges have resulted in entire areas that were previously shown as ‘served’ based on data provided by the area’s incumbent provider being changed to ‘underserved’ or ‘unserved’, making BEAD or other public funding available for area broadband improvements. This project’s primary goal was to demonstrate eligibility of the Canal neighborhood for BEAD funding by the submission of a successful BEAD challenge and designing a municipal shovel project for a subsequent BEAD grant application. Comcast/Xflnity adjusted their network to eliminate underserved addresses. This undermined the project’s key objectives but led to some positive outcomes for residents. Incumbent Network Upgrade Internet testing performed before Comcast’s mid-split upgrade shows that most addresses were not receiving adequate broadband internet service levels prior to the upgrade. Tests performed after the upgrade show that most of those same addresses are now receiving “served” broadband internet most of the time. While Comcast has not provided any reason for the timing of the upgrade, it is logical to conclude that this improvement, along with numerous other improvements being made across the country, were necessary for Comcast to defeat the BEAD challenges being submitted for addresses in their service territory. For this reason, one positive outcome from this LATA project is the network upgrades being performed by Comcast to improve service in disadvantaged areas that have been subjected to a pattern of deferred maintenance and disinvestment, like the Canal San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 22 Canal Broadband Feasibility Study and Market Report neighborhood. Although this does not address the lack of competition and the legacy infrastructure problems, it has resulted in an improved ‘lived experience’ for the residents, even though it enabled Comcast to successfully defeat the BEAD challenges submitted to the State by the project. Public Data A second positive outcome from the project is the collection, documentation and analysis of important broadband consumer data and experience information. Historically, the industry has shaped broadband investments and service level expectations based on the market. As broadband evolves from an amenity service to an essential service, the consumers experience, service level needs, and a household’s monthly available internet budget should inform leadership, shape policy, and identify targeted public funding needs that are appropriately coupled to measurable public outcomes. The city now has a repository of historical consumer and internet measurement data with some market analysis and recommendations that should be used to shape municipal broadband policies and public funding support going forward. Infrastructure Design A third positive outcome is the development of a flber-to-the-living-unit infrastructure design that could support a future city-sponsored project. The city now owns this design, including geospatial data flles complete with routes, distances, and unit counts. This level of design can be used now or in the future to submit a grant application for a public or private flber project, to inform public policy with a better understanding of the costs associated with a flber optic infrastructure improvement, the levels of public support that could incentivize a private investment, and the value of public assets that could be contributed to a broadband project or program. FIGURE 12 Canal Neighborhood Municipal Fiber to the Living Unit Design San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 23 Canal Broadband Feasibility Study and Market Report Recommendations 3 Public Engagement City leadership has served as the public champion for the broadband disadvantaged in San Rafael with this LATA project. Although the stated project goal of making the Canal neighborhood eligible for BEAD funding and applying for grant funding to build a municipal open access flber installation in the area using BEAD funding was not achieved, service levels were improved, relevant broadband data was collected and analyzed, making city leadership much better informed and prepared to continue in this role. This flnal report recommends that the city maintain regular updates to the data collected by this project, including monitoring potential funding opportunities for broadband improvement projects. Much of the data collected was submitted by the residents of the Canal neighborhood with the help of Marin County, Voces Del canal, and the Canal Alliance. The City should maintain a regular broadband engagement cadence with these organizations to monitor progress and inform municipal policies and actions. A brief annual review which summarizes the state of broadband in the Canal neighborhood posted publicly for review by area residents and elected leadership will help keep broadband access and affordability as a priority consideration. Industry Engagement Broadband industry leaders have historically viewed their data as authoritative. They have been quick to dismiss consumer challenges as something the market would address if it were a real need. The data collected from the Canal neighborhood and nationwide highlights that this cannot be expected when the market is concentrated in areas with high returns while other areas are plagued by a pattern of disinvestment. San Rafael is something of a posterchild of this broadband disparity because multiple gigabit internet service providers are available on one side of a road at costs as low as $50 per month for symmetrical (upload and download) 10 gigabit internet access, while the other side of the road may only have one Internet Service Provider (ISP) capable of offering 1 gigabit internet access (1 gigabit download with 20 megabits upload) for $120 per month before taxes, fees, modem rentals, and other fees. By using and maintaining the data collected in this report, the city can encourage new broadband investments and help incumbent operators understand and address the market failures in San Rafael. As broadband continues to evolve into an essential service, the authoritative data driving public broadband policy and funding should not be based on the theoretical maximum for their technology but by actual data collected at the consumer’s premise. The City can help San Rafael’s incumbent operators shift their maintenance and upgrade schedules to refiect this new reality by clearly establishing in policy what the municipal expectation from providers is when they are making use of the public Rights of Way (ROW). The City should also institute municipal policies that require broadband improvements whenever possible, such as when an area is redeveloped, or other roadway or utility infrastructure improvements occur. Conclusion This project has highlighted the disparity in access and affordability that exists in San Rafael. Effectively closing these gaps will require a multi-pronged approach that focuses on infrastructure improvements, affordable access, and digital literacy. While this report focused on delivering an infrastructure improvement for the Canal neighborhood, the data collected, analyzed, and documented in the various reports paints a clear picture of what San Rafael, California – LATA Report – Produced by EntryPoint Networks Page | 24 Canal Broadband Feasibility Study and Market Report investments, upgrades, and improvements are needed to address the broadband market failure in San Rafael. Using this information and maintaining a focus on desired outcomes will help the City to implement effective policies to help close the gaps, evaluate and apply for any future municipal broadband funding opportunities, and participate in the larger State and national broadband discussions as broadband internet becomes an essential service. THANKYOU For Your Consideration www.entpnt.com San Rafael Canal Neighborhood Network Design Report January 2025 Page 1 of 13 Canal Neighborhood Network Design Contents Executive Summary ......................................................................... 2 Purpose........................................................................................................... 2 Key Findings .................................................................................................... 2 Solution Challenges ........................................................................................ 2 Landlord and Renter Misalignment ................................................................... 3 Fiber Feasibility Modeling................................................................................. 3 Multi Dwelling Unit (MDU) Considerations ........................................................ 4 Apartment Building Interior Network Wiring Considerations .............................. 9 Property Access Considerations .................................................................... 10 Aerial and Underground Considerations ......................................................... 10 Wireless ........................................................................................................ 10 Canal Neighborhood Design ........................................................... 11 Geospatial Desktop Design ............................................................................ 11 Design Data ................................................................................... 12 Conclusion .................................................................................... 13 Page 2 of 13 Canal Neighborhood Network Design Executive Summary Purpose This report uses lay person terms to document and summarize the key assumptions, decisions, and resulting flber optic plant design used to provide objective capital and operational cost data to inform a municipal broadband strategic plan for the City of San Rafael, California. This design report provides the basis for architecture decisions and the cost ascertainment and estimation methods used to model costs. These outputs have been developed to inform extensible municipal solutions capable of closing the digital divide for communities suffering from a lack of private broadband investment. Key Findings • Fiber optic is the optimal network media for metro or suburban broadband infrastructure investments. • An area in the Canal neighborhood containing 1,100 living units located inside 449 separate dwelling structures which range from single family dwelling units to duplexes, fourplexes, and multi-unit apartment buildings was designed using a desktop geospatial design tool to inform cost estimates. • 39,879 feet of cable would need to be installed within the public rights of way and utility easements in order to provide a flber optic connection for each living unit, resulting in an average of 36.25 feet per living unit. (39,879 feet divided by 1,100 living units equals a 36.25 feet per living unit average.) • Underground construction options capable of achieving a per foot unit cost of $59 or less would be required for capital recovery feasibility based on available market data. • The resulting average per passing cost of $2,138.75 (36.25 feet multiplied by $59 per foot equals $2,138.75) would result in a total mainline or passing construction cost of $2,352,625 ($2,138.75 per passing multiplied by 1,100 passings in the design area). • The total drop cable required to connect the 449 separate dwelling structures is estimated to be 41,221 feet, resulting in an average drop length from the easement or right of way to the dwelling structure of 92 feet (41,221 feet of cable divided by 449 installations equals an average of approximately 92 feet per drop installation). • Additional materials and labor will be required to extend flber from the demarcation point where the flber cable from the street terminates at the dwelling into the interior of each dwelling’s living units. • Based estimated construction costs from the design, achieving a monthly out of pocket cost of $50 or less for consumers subscribing to this municipal flber optic solution will require at least 7 out of every 10 living units passed by the flber line to connect and pay for a connection. Alternatively, grant funding that could be used to reduce the amount of funding that will need to be recovered through monthly fees could help the network achieve feasibility if less than 7 out of every 10 living units passed by the flber line to connect and pay for a connection. Solution Challenges An effective multi-unit dwelling or apartment rental property solution will need to address two challenges: 1. The incentive misalignment for landlords to invest in property broadband improvements for tenants, and, 2. The nuances associated with applying typical flber to the home feasibility models to multi dwelling units can create modeling inaccuracies. Page 3 of 13 Canal Neighborhood Network Design Landlord and Renter Misalignment Natural incentives for landlords are focused on maximizing the return on their property investment while renters naturally focus on maximizing the value they receive from their rent payments. While these incentives are not mutually exclusive or diametrically opposed, they are often misaligned when it comes to broadband. Many landlords have entered into bulk billing arrangements with broadband providers. Under such an arrangement, a company agrees to provide service to every living unit in the building. Renters are then billed a prorated share of the total monthly cost. Under these arrangements, tenants may be billed by either the landlord or the service provider and the landlord may receive a portion of the revenue generated. Although the Federal Communications Commission has outlawed exclusive agreements that prevent competitors from offering service in multi dwelling units, landlords and providers are still utilizing non- exclusive agreements to proflt and flnding new ways to effectively limit a renter’s options as a broadband consumer. For example, landlords and providers can require a competitive provider to install new wiring in the building by blocking access to any existing network cables, thereby driving up costs, limiting competition, and protecting their revenue. Market forces also typically require a renter to select properties based on prioritizing affordability, “must have” elements, and amenities, in that order. This often results in a renter accepting expensive and unreliable broadband service in order to meet their rental needs within their existing budgets. Fiber Feasibility Modeling Fiber-to-the-home (FTTH), also known as flber-to -the-premises (FTTP), can deliver high speed internet access when optical flber is installed and connected directly to every home, apartment building, or business. This approach for delivering high-speed internet services and greater bandwidth costs less to build today and offers superior performance compared to non-flber coaxial cable and DSL (Digital Subscriber Line) connections. Feasibility modeling for FTTP projects require flber plant costs to be divided into two categories: 1. Passing costs: A flber passing is a location that can be connected to a flber optic network, usually because it's close to flber infrastructure like utility poles, manholes, or flber pedestals. The deflnition of a flber passing can vary slightly by operator, but generally it's considered to be any location that can be connected to flber running along a main road. Passing costs are all of the costs required to make a direct flber connection available to every serviceable location along a route or in an area. The cost per passing for flber includes the cost of the following: a. Core and distribution network equipment: Located inside active facilities. b. Backbone flber route: Aerial attached to poles or underground direct buried or in a duct or conduit. c. Passive optical splitters: Part of the cost per home-passed if used. Factors that infiuence passing costs include: d. Distance: The further the distance between service locations, or the average passing distance, the more expensive it is to construct, resulting in higher rural passing costs. e. Type: Attaching aerial cables to existing poles is typically less expensive than installing underground depending on the condition of the poles and their readiness to accept a flber attachment. Page 4 of 13 Canal Neighborhood Network Design f. Terrain: Challenging terrain like rocks, rivers, mountains, or forests can increase costs. g. Location: Installing flber under and around hard surfaces such as concrete and asphalt can cost more than placing it along the edge of a rural road, resulting in higher costs for metro or suburban areas. h. Labor: Labor costs can account for over 65 – 90% of the total cost. i. Materials: Material costs can account for 10 – 35% of the total cost. 2. Drop costs: A flber drop installation consists of connecting a small, single strand, flber optic cable to a strand inside of an aerial or underground passing cable located near the property in a public right-of- way or easement and extending that flber into the living unit and connecting it to a premise device where the services are terminated. The access port, or the upstream electronics that communicate with the premise installed electronics is also part of the drop costs, as the number of required access ports is based on the number of drops and premise devices connected to the system. Fiber drops can be installed overhead or underground, depending on the need. The installation process involves splicing the flber strand from the main line to the home, apartment building, or business. There are several types of flber drop cables, including fiat, indoor, and outdoor self-supporting. Factors that infiuence drop costs include: a. Distance: The further the distance between the distribution or passing cable and the dwelling or living unit, or the average drop distance, the more expensive it is to install. b. Type: Aerial installation, which involves running cables from existing poles to the dwelling unit, is typically less expensive than underground installation, which requires trenching and possibly conduit placement. c. Location: Installing flber under driveways and parking lots costs more than underground installations in softscape areas requiring restoration such as yards and landscaped areas. d. Premise equipment: The type of premise equipment and the cost of the upstream access port can vary signiflcantly depending on network design, equipment type, manufacturer, and supported functionality. e. Labor: Labor costs can account for over 65 – 90% of the total cost. f. Materials: Material costs can account for 10 – 35% of the total cost. Multi Dwelling Unit (MDU) Considerations Traditional Fiber-to -the-premise (FTTP) models only include the costs required to terminate a single flber drop on an exterior wall of the apartment building. Modeling in this way does not provide for any costs associated with improving the building’s interior wiring or network cabling to assure that each living unit has flber access. This gap creates a challenge that is often expected to be the responsibility of either the landlord or the tenant(s), neither of which may be qualifled nor prepared to address the need as shown in the following design graphics. Page 5 of 13 Canal Neighborhood Network Design FIGURE 1 Graphic showing the primary components required to construct a fiber to the premise infrastructure . Figure 1 shows the primary components of constructing an underground flber to the premise infrastructure. These components include the installation of underground duct, vaults, flber optic cable between active locations and the installation of network electronic equipment inside network facilities with power and environmental controls such as buildings, huts, cabinets, and the premises where the service is terminated. Underground flber optic infrastructure is used for modeling in this report because it is actually cheaper to construct than other wireline infrastructures such as copper DSL and coaxial cable. There is also a growing speed chasm between flber and other alternatives, including wireless options. Additionally, this report assumes that a solution capable of supporting 30-year cost recovery models, which require a ‘future-proof’ infrastructure capable of handling the growing bandwidth demand well into the next 50 or more years. Underground installation was selected rather than aerial for these same reasons, including an emphasis on flxed predictable costs and maximizing the life span on the digital infrastructure. The next several graphics are intended to provide a basis for understanding the speciflc design aspects used to model costs. They are presented in logical order from what would be assumed to be highest cost to the potentially lowest cost based on the level of design performed for this report and anecdotal data collected from similar projects. Page 6 of 13 Canal Neighborhood Network Design FIGURE 2 Graphic showing the 1:1 port and fiber to premise ratio required for an Active fiber network. Figure 2 shows some speciflc design aspects associated with an Active flber to the premise infrastructure. Key metrics contributing to costs are the requirements for one access port and one flber strand for every connection as shown. Put simply, in an Active installation, each customer has their own dedicated flber optic strand within a cable from their premises to the next upstream location with active network equipment. Active network equipment is hardware that connects devices to form a network and is responsible for processing, amplifying, and transmitting data signals. These devices are powered by an electrical network and require some level of environmental control to protect the equipment from moisture and extreme temperatures. For these reasons, the requirement for a dedicated flber strand to an active network access location and a dedicated access port for each customer is the primary factor contributing to cost to construct estimates. In lay person’s terms, an active network’s primary costs come from the high strand (large cable and duct sizes and counts) and port counts (signiflcant network racking space) required to construct a flber-to-the-premise infrastructure. Distribution areas typically connect 2,000 to 3,000 customers to a single location with active electronics. This need is typically better served by a small flber shelter or hut when compared to a cabinet because the total cost of ownership over 20 years is often less for a structure when compared to a cabinet due to the high recurring monthly costs for cooling and maintenance. Page 7 of 13 Canal Neighborhood Network Design FIGURE 3 Graphic showing the 1:64 port and fiber to premise ratio required for a Passive fiber network. Figure 3 shows some speciflc design aspects associated with a passive flber optic to the premise infrastructure. Passive installations make use of splitters that allow customers to share flber and access ports. For passive installations a key bandwidth consideration is the number of customers sharing a single flber and access port. This report assumes that 64 premise endpoints or possible customers will share a single flber and access port at the access cabinet or flber hut. This report utilizes XGS-PON as the network standard to model costs. XGS-PON is an updated standard for Passive Optical Networks (PON) that can support higher speed 10 Gbps symmetrical data transfer. The “X” in XGS represents the number 10, and the letter “S” stands for symmetrical, XGS-PON = 10 Gigabit Symmetrical PON. Key metrics contributing to capital and operational cost reductions for a passive design come from reduced strand counts (smaller cable and duct sizes reduced by a ratio of 1:64) and reduced port counts (reduced by a factor of 1:64). Typical distribution areas of 2,000 to 3,000 connected properties are well served by small cabinets with simple heat exchangers that are easy to maintain for environmental controls. The splitters used in this design architecture do not require any power or environmental controls and may be easily housed inside various physical locations such as splice enclosures inside underground vaults due to their small size. Page 8 of 13 Canal Neighborhood Network Design FIGURE 4 Graphic showing the Access ports installed in the apartment as required for an Active MDU fiber network Figure 4 shows some speciflc design aspects associated with an active flber optic to the apartment infrastructure. Apartment design modiflcations typically move the access ports for the living units to the apartment building as this signiflcantly reduces the cost to get flber connectivity to the building. This modiflcation essentially treats the port at the hut or cabinet and the flber used to connect that port to network access equipment located in the apartment building as a backbone or distribution connection. While this solution does signiflcantly reduce the cost to connect an apartment building, it relies on racking, power, backup power, and environmental controls at the apartment building itself in order to provide the port count necessary to connect all of the living units. Access to these facilities will also need to be provided to the network operator at all hours in order to support operations. The dynamics associated with these design adjustments tend to improve costs to connect the apartment building by moving the drop infrastructure and costs to the building. Shifting drop costs in this manner presents a number of challenges, including: • Improving an interior building space to house network access equipment (with power and HVAC), • Improving or installing interior network cabling to connect the building’s access equipment to the premise equipment installed inside each living unit, and, • Providing the necessary economic and operational functions to support these changes. Page 9 of 13 Canal Neighborhood Network Design FIGURE 5 Graphic showing 2x distributed splitters installed for a Passive MDU fiber network Figure 5 shows some speciflc design aspects associated with installing two splitters along the flber route to an apartment. This type of architecture is called a distributed split flber network. This method uses cascading stages to distribute connectivity flrst to the building and then to the living units inside the building. This type of design change for apartments typically provides the following beneflts: • Distributed splitters can further reduce the required flber strand count. • Distributed splitters provide fiexible split ratios and locations. • Distributed splitters eliminate the requirement for locating active equipment at the apartment building for access ports. Apartment Building Interior Network Wiring Considerations The existing apartment buildings in the Canal neighborhood have legacy interior network wiring installed. Copper phone lines that would typically support Digital Subscriber Line or DSL connections were installed many years ago. Site visits and interviews with residents indicate that this phone wiring has not been maintained and is not in use for virtually all of the apartments in the area. For these reasons, a solution that relies on using existing phone cables should not be considered. Cable coaxial lines are prevalent throughout the area. Site visits and interviews with residents indicate that coaxial cable supporting dissimilar standards and solutions has been disparately strung around and through the various apartment buildings and complexes over the years. While this has resulted in an inconsistent and Page 10 of 13 Canal Neighborhood Network Design sometimes incompatible wireline building infrastructure, there are solutions capable of using these cables as an alternative interior wiring solution as part of a broader broadband infrastructure strategy. Aside from the technical challenges associated with locating and testing a coaxial connection capable from a demarcation point on or in the building to each living unit, ownership and access challenges should be expected as the landlord or incumbent cable operator may or may not reserve ownership of these wirelines by contract. For these reasons, no solution that relies on existing coaxial cable has been modeled as a part of this report. This report assumes a flber all the way to the living unit solution that utilizes a distributed split XGS-PON architecture in an effort to maximize cost efficiencies while also guaranteeing a working solution from end to end without relying on legacy components that may or may not be available to support any portion of the solution. Property Access Considerations Constructing broadband infrastructure solutions requires the installation of infrastructure on public and private land. Performing these installations requires Rights-Of-Way (ROW) permits, public utility easements, and licenses or agreements to provide access to the private properties where living units are located. • Permits provide access to the public rights of way, such as streets, sidewalks, trails, or highways, and allow broadband operators to place infrastructure within and to access the right of way for purposes of construction or maintenance. • Easements are typically created as a part of suburban development and acquired from private property owners to grant the right to use and access private property for a specifled purpose. • Private property access agreements are also needed to place infrastructure from a ROW or easement to a private residence or business, including the right to potentially access the interior of a multi-tenant dwelling in order to install or maintain the infrastructure. Such agreements may be embedded in a service agreement or contract for retail service(s). Aerial and Underground Considerations Fiber and other wired infrastructure may be placed on poles owned by telephone and electric companies (aerial installation) or buried underground in the ROW or public utility easements. • Aerial installations involve a preparatory process known as “make-ready” work, in which the poles are surveyed to determine if they need repair or replacement to support the new broadband cabling. Aa desired aerial broadband improvement may not be feasible due to the costs, delays, and disruptions associated with completing these processes. For this reason, the modeling for this report utilized an underground installation. • Underground installations of flber lines and other cables that support broadband run through conduit or duct that protects the cables from damage. Vaults containing splice enclosures where splitters may be located, and the flber cable can be accessed to extend lines along diverse routes or directly to the properties, buildings, and apartments served. Wireless Wireless is not considered a broadband solution for any suburban or metro area by the FCC. This limits federal and state funding initiatives to flber with a preference for underground installation as providing the best long- Page 11 of 13 Canal Neighborhood Network Design term return for any public investment. For these reasons, wireless is not contemplated as a solution or component for any portion of this report. Canal Neighborhood Design Geospatial Desktop Design An area in the Canal neighborhood containing 1,100 living units located inside 449 separate dwelling structures which range from single family dwelling units to duplexes, fourplexes, and multi-unit apartment buildings was designed using a desktop geospatial (GIS) design tool to inform cost estimates. This software design tool utilizes advanced algorithms and data-driven approaches to create highly optimized flber optic network designs, signiflcantly speeding up the planning and construction process for telecommunication networks by generating efficient designs based on various input data like geography, regulations, and existing infrastructure, allowing for faster deployment and cost reduction compared to traditional methods. Essentially, this translates complex network planning into a user-friendly platform that generates detailed designs with minimal manual intervention. The resulting design is available to view online at https://fond.biarrinetworks.com where it can also be downloaded in standard GIS flle formats such as SHP and KML flles. A screenshot of the online application is shown in the following graphic. FIGURE 6 Figure 6 shows the area designed to inform model inputs such as total route distance, average passing distance, average drop distance, and property and living unit counts. Page 12 of 13 Canal Neighborhood Network Design Design Data The design outputs show that 39,879 feet of flber cable would need to be installed to 449 buildings and extended into 1,100 living units to provide a true flber to the living unit multi-tenant solution. An order of magnitude cost estimate based on anecdotal information for similar installations results in an estimate of $2,352,625 to construct the passing infrastructure in the ROW and public utility easements. An average drop length cost to extend flber to each of the 449 buildings would be $650 per extension resulting in a total cost of $291,850 to terminate flber and place a splitter at the building. Combining these costs results in an estimated total cost of $2,644,475 before extending flber inside of the apartment buildings or installing network devices in the apartments or living units. Fiber extensions and equipment installations for the individual living units is estimated at approximately $650 per living unit. Combining all of these costs results in a total estimated project cost of $2,858,975 if 30% of the living units passed sign up for service, $2,930,475 if 40% of the living units passed sign up for service and $3,001,975 if 50% of the living units passed sign up for service. The reason costs increase as the ratio of signups per unit passed, or take rate, increases is because more interior building wiring, access ports, and premise devices will be required. The average cost per connected customer improves as the take rate increases because the cost to connect each building is now shared between a larger customer base as shown in the graphics below. FIGURE 7 Figure 7 shows how total project costs increase as more subscribers request an install. 0 100 200 300 400 500 600 700 800 900 1000 $2,800,000.00 $2,900,000.00 $3,000,000.00 $3,100,000.00 $3,200,000.00 Total Project Cost by Subscriber Take Rate Page 13 of 13 Canal Neighborhood Network Design FIGURE 8 Figure 8 shows how the cost per subscriber is reduced as take rate increases and the common or passing costs are shared among more rate payers. Take rates in excess of 70% (7 out of every 10 living units, or more, with access to the infrastructure connect and subscribe to services) or grant funding capable of reducing the total amount of capital required to be recovered through monthly fees will be required to achieve desired affordability targets (less than $50 per month out of pocket) based on market data. Conclusion Take rates in excess of 70% or grant funding capable of reducing the total amount of capital required to be recovered through monthly fees will be required to achieve desired affordability targets (less than $50 per month out of pocket) based on market data and current municipal bond rates. These conclusions and the data inputs used to develop the proforma model will be documented as part of a flnal report. $0.00 $1,000.00 $2,000.00 $3,000.00 $4,000.00 $5,000.00 $6,000.00 $7,000.00 $8,000.00 $9,000.00 $10,000.00 330 440 550 660 770 880 Passing Costs by Take Rate THANKYOU For Your Consideration www.entpnt.com