- Published on 09 August 2022
Tens of billions in infrastructure funding are being distributed to close the digital divide, but will certain short-term changes result in problems down the road for all parties?
Aiming to provide enterprise customers with expanded and enriched broadband networks to reach new sites and home workers during challenging times, carriers and their vendors are reworking longstanding practices to skirt weak supply chain links.
Tens of billions in federal funding are earmarked for the telecom industry’s largest ever infrastructure undertaking. The focus is on closing the digital divide and introducing high-speed services using next-generation fiber technologies.
But supply chain woes, chip shortages, and more have some carriers changing the way they acquire broadband infrastructure from equipment suppliers to advance their high-speed Internet service rollouts during a time of mounting frustration.
Current delay situation
When you combine delays in fiber optics, the resin that surrounds them, and the electronics for fiber broadband, carriers are experiencing 52-week lead times, according to Gary Bolton, President of the Fiber Broadband Association (FBA). The group held its annual conference in June and turned away over one hundred exhibitors.
Bolton said 52 weeks is workable for companies that plan their fiber deployments carefully. By the time they apply for a government grant, design their networks, and do other planning, they are going to need at least 52 weeks, anyway.
Rush to cash in
The allure of the largest ever broadband breakout has caught the attention of the telecom industry and enterprise IT. Carriers hope – sooner rather than later - to interest enterprises in expanding and upgrading their corporate networks to reach new offices and support work-from-home efforts as the telecom industry tires of rolling delays from supply chain issues. The waiting is the hardest part.
“Investment in next-generation fiber technologies has skyrocketed over the last four years,” explained Jeff Heynen, VP of Broadband Access and Home Networking for Dell’Oro Group, a market research and advisory services firm. “Broadband service margins remain incredibly strong, providing additional incentives for multi-gigabit service rollouts. Fiber plans have been pushed forward to help (upgrade) access networks.”
Shortening the acquisition process
Non-traditional acquisition processes have taken root. Instead of first apprising vendors of their gear needs in "forecasts," to vendors, Consolidated Communications has dropped forecasts and replaced them with actual purchase orders to shorten the process, explained Tom White, CTO of the carrier, which operates in over twenty states from coast to coast. "A purchase order out trumps a forecast all day, every day." Breaking practices helps deal with a 9-12 month wait on electronics and other required components.
Ziply Fiber, a carrier providing business and residential services in parts of the Pacific Northwest, has taken Consolidated’s approach a step farther. “We own a lot of real estate and yards behind buildings,” began John Van Oppen, VP of Network for the company. "We fill it up with a parts supply of about 12 months and keep ordering on the back end." Submitting early purchase orders has also helped the carrier, which also uses the process for acquiring fiber. However, both carrier technology executives admit this changed process is not guaranteed to always work, as equipment vendors often receive de-commit notices from the makers of components used in equipment because of their production challenges. This can add three to six months to the availability of broadband networking – everything from fiber CPE to outside plant equipment such as cabinets.
The carrier executives – and others - discussed numerous issues in the Broadband Technology Summit, a multiday event created by Fierce Telecom and the Dell’Oro Group.
Concerns with overbuying
Dell’Oro’s Heynen questioned the long-term merit of overbuying. "This makes me nervous because the overcommitment is likely going to turn into an overshoot once the supply chain issues are resolved. It is called the Snapback Effect, and we are going to see some of that next year. But even if the providers are overbuying to move up the queue for equipment, the vendors still cannot ship to those commitments, which is reflected in the growing backlogs these vendors have reported."
Low inventories, which have been commonplace during Covid-19, provide a potential snapback effect for the economy.
The road ahead?
Heynen is concerned that recent interest rate increases will slow down purchases and whether a percentage of the vendor backlogs would simply go away as providers canceled projects and waited things out. He is convinced that there will be some operators “who just slow roll their buildouts or cancel some fiber overbuild projects that just aren’t strategic enough.”
Targeting small rural carriers that have landed RDOF funding but that typically lack global supply chains, Nokia in June introduced a network-in-a-box package that lets them provide high-speed fiber services to towns with up to 1,000 homes.
Each Nokia Broadband Relief Kit is comprised of the necessary FTTH equipment, software licenses, support, and state-of-the-art in-home Wi-Fi gateways required to serve a typical town of 1,000 households, according to the vendor. The kits support GPON and XGS-PON over a single port and fiber using Nokia’s Multi-PON-Module (MPM) technology.
The kits can support 25G PON today or when the need arises, according to the vendor. The kits are available now and include expedited delivery. Nokia has created twenty-five of these kits.
Households in unserved or underserved areas have been hardest hit by global supply chain shortages of telecommunications equipment as operators planning new gigabit broadband networks have been unable to complete their builds.
While both public and private funds are available in unprecedented quantities, many small operators have found themselves unable to secure the necessary materials from their established supply chains to meet their self-imposed construction schedules or regulatory-imposed milestones.
Global cable maker Corning has asserted that manufacturing capacity is a large part of the problem, especially in North America.
CEO Wendell Weeks put it on the company’s Q4 2021 earnings call in January “if we could make more, we could sell more…the real bottleneck is not resin, is not raw materials, is not labor. It is just us being able to get into place capacity that is more appropriately balanced to the demand that we are experiencing.”
To that end, Corning announced plans to build a new optical fiber manufacturing facility in Poland, and in September 2021, said it would spend $150 million to expand its optical cable manufacturing operation in North Carolina.
The new manufacturing facility in Poland is set to become operational this year. That will free up U.S. fiber capacity for U.S. demand, according to Corning. The company’s execs believe Corning and others expanding manufacturing capacity will improve lead times for their carrier customers.
- Putting Broadband Internet on the Map
- What Enterprises Can Expect as Carriers to Spend Billions on Broadband Rollouts
- Telecom Industry Takes the Workforce Challenge to Staff Internet-for-All Broadband Rollout
- Published on 26 July 2022
If the FCC rules interference is a concern, spectrum sharing systems could provide a solution for all parties by maximizing the use of the 12GHz band.
In the latest spectrum challenge, Elon Musk's SpaceX claims DISH Networks wants to use the 12 GHz band with its terrestrial 5G network, which would cause harmful interference to his emerging Starlink satellite Internet service. DISH disagrees.
For its part, DISH already has a 5G network and hopes to expand it by moving into the 12 GHz band, which it uses to provide satellite TV service.
In January, the FCC voted to issue a Notice of Proposed Rulemaking (NPRM) seeking comment on ways the 12 GHz band might be better used. The notice sought input on methods for allowing new uses in the band while protecting incumbents.
Both operators have submitted to the FCC technical studies that support their arguments regarding interference. SpaceX’s Musk has successfully urged customers to contact the agency supporting the company’s claim.
DISH continues to claim a 12GHz 5G network can coexist with satellite internet services without generating interference.
Both operators, and much of the telecom industry, await a ruling from the FCC, which has not said when it will act.
Enterprise IT impact
Enterprise IT, however, may not feel a significant impact from the resolution of the challenge. That is unless they seek broader competition in the private 5G wireless network sector or are looking to connect mostly rural sites using Low Earth Orbit (LEO) satellite services.
The broader impact could reach farther if the FCC finds no inference and/or recommends a spectrum sharing approach that could maximize the use of the 12GHz band by all interested parties without fear of interruption.
This appears to have worked well with the Citizens' Broadband Radio Service (CBRS) creation, auction, and diverse spectrum winning bidders, including many enterprises, most looking to build private 5G networks.
Building a spectrum-sharing infrastructure will take time, and that is once the FCC rules on the 12GHz band dispute.
A recent history of 5G service interference concerns
For veteran IT staff, this is but the very latest in a lengthy list of disputes over potential interference caused by shared use of the same spectrum. Most recently was last year’s spike in concern about 5G services offered in the C-band near U.S. airports interfering with altimeters, which are used to gauge elevation in airplanes.
The concerned parties expressing serious concern about harmful interference included the Federal Aviation Administration, CEOs of several airlines, the Department of Transportation, and plane makers. AT&T and Verizon were in opposition to slowing the deployment of 5G near airports. But the carriers eventually agreed in January to delay rollouts by six months while the key players explored the issue.
In June, the FAA announced that key stakeholders in the aviation and wireless industries have identified a series of steps that will continue to protect commercial air travel from disruption by 5G C-band interference while also enabling Verizon and AT&T to enhance service around certain airports.
- The phased approach requires operators of regional aircraft with radio altimeters most susceptible to interference to retrofit them with radio frequency filters by the end of 2022. This work has already begun and will continue on an expedited basis, according to the agency.
- At the same time, the FAA said it worked with the wireless companies to identify airports around which their service can be enhanced with the least risk of disrupting flight schedules.
During initial negotiations in January, the wireless companies offered to keep mitigations in place until July 5, 2022, while they worked with the FAA to better understand the effects of 5G C-band signals on sensitive aviation instruments.
Based on progress achieved during a series of stakeholder roundtable meetings, the wireless companies offered Friday to continue with some level of voluntary mitigations for another year, according to the FAA.
Despite the collaborative efforts to work out a solution, a permanent fix has not yet been created and agreed on.
The CBRS sharing solution
If the FCC and its tech experts are concerned about interference between DISH 5G and SpaceX birds, the situation could be addressed so that neither is shut out of the 12 GHz band. That could be enabled using spectrum sharing, which was used to let carriers, enterprises, and others to concurrently use the 3.5 to 3.7 GHz space alongside longtime users. The 2020 auction made available space to 228 winning bidders and brought the FCC $4.58 billion.
DISH, bidding under the name Wetterhorn Wireless, came in as the second largest total winning bidder, spending over $900 million.
A wireless industry ecosystem, including the FCC, equipment vendors, and those building systems necessary to enable greater spectrum sharing, have been hard at work to prepare the CBRS for a wave of new users. The band is valued at over $15 billion.
The initial commercial deployment of CBRS service was approved by Spectrum Access System (SAS) administrators Amdocs, CommScope, Federated Wireless, Google, and Sony.
- Published on 30 June 2022
Demand for skilled staff is sky high, but not supply, for the largest-ever U.S. infrastructure effort. Associations, carriers, and vendors look to fill the staffing gap as spending on fiber broadband climbs.
With supply chain challenges aplenty in expanding broadband networks, carriers are now facing a workforce shortage in deploying the fiber equipment that they have purchased, especially for the Fed-funded Internet-for-all program.
For those IT managers that were not yet aware of the workforce shortage, which could complicate network expansion plans, Charter Communications CEO Tom Rutledge sounded the alarm last month at a Moffett Nathanson investor conference.
“There is no labor pool there. For all the construction that must be done, there is no skilled labor force that is currently out there doing it that can be repurposed,” warned Rutledge. “It has to be built and trained.”
“It's going to be challenging,” he continued. “We have thousands of unfilled positions.” Charter serves over thirty-two million business and residential customers in forty-one states.
Why now? How did we get here?
The deluge of broadband funding, along with private expansion efforts, is driving a major spike in demand for those services and for workers. Many have begun to land large grants, with some launching their own training and certification programs which offer a big boost in open jobs in local communities and far beyond.
Telecom industry responds to the needs
Despite the doom-and-gloom outlooks borne out of supply chain breakdowns, planning, and component shortages, the networking industry has begun fighting back, with an ambitious fledgling training and certification program from the Fiber Broadband Association (FBA) in Wilson, North Carolina, and a vastly expanded undertaking by partners AT&T and fiber manufacturer Corning. Do not forget training programs created by individual communities.
Just how broad is the shortage of tech workers needed to help carriers deploy broadband infrastructure funded by the Biden Administration's Infrastructure and Jobs Act? Consider this:
Carriers have come from far-flung locations to poach students from a nascent fiber broadband certification program launched as a pilot in early March to educate and train attendees – from the FBA in conjunction with Greenlight Community Broadband at Wilson Community College in Wilson, N.C.
The FBA has been engaged with twenty-three states about rolling out this fiber optic technician training program with their community college systems and fiber optic broadband service providers. “We look to reach all 50 states and the U.S. territories by the end of the year," said Deborah Kish, Vice President of Research and Workforce Development at the FBA, when the undertaking was launched.
With equal parts classroom and hands-on instruction, the Optical Telecom Installation Certification (OpTIC) program was designed by fiber broadband experts to quickly scale fiber technician education, fill the existing fiber skills gap, and accelerate fiber deployments across North America.
The need for skilled fiber optic technicians will significantly impact each state’s ability to deploy broadband. The FBA’s OpTIC program teaches the knowledge and skills required to professionally install, test, and maintain high-speed fiber broadband networks.
"When we saw the need for an expanded fiber workforce in order to keep up with broadband demand and growth opportunities, we began development of this intensive training program to ensure that no state is left behind in the digital equity gap," said Kish.
Not singing the Blues in rural Louisiana
LUS Fiber, a city-owned telecom in The Bayou state, was awarded $21 million of a federal grant earlier this year and is asking for a $19 million helping of the state’s $180 million program to expand in other rural Acadiana communities.
LUS is working with South Louisiana Community College (SLCC) to launch a new fiber-optic install technician program this summer to meet the expanding workforce needs of the region and help residents develop skills to launch their careers.
"We've been working with the industry now for just a little over two years to design a program that is versatile enough to produce entry-level employees into each aspect of this industry," SLCC's Director of Transportation, Distribution, & Logistics Charlotte LeLeux told the Lafayette Advertiser newspaper in June.
The school's new fiber-optic technician program, an 18–20-week course, is expected to launch at SLCC’s Crowley campus in July.
It will cover how to splice fiber optic cables, how to hang cable on telephone poles, how to operate installation equipment, and other skills. The goal will be to cover everything from construction to putting fiber in the home, LeLeux said, “so that when they're hired on by these companies, their training with them would be very minimal.”
AT&T, Corning train technicians and network specialists
Targeting workforce development, AT&T and Corning have joined forces to create a new training program focused on equipping thousands of technicians and network specialists across the industry with the skills crucial to design, engineer, install, and manage a growing fiber broadband network across the U.S.
Steve Mitchell, senior vice president, Carrier Networks at Corning Optical Communications, told me: "As the industry is currently experiencing a shortage of technicians and installers, this training will support future needs and help build the skilled workforce of tomorrow,” wrote Jeff Luong, President Broadband Access, and Adoption Initiatives at AT&T in a blog on the undertaking.
"Highly trained workers are needed and needed quickly," emphasized Luong. He predicted the program will be available in time to support the historic government investments outlined in the Biden administration's Infrastructure Investment and Jobs Act.
What’s in the program?
The Fiber Optic Training Program was launched in May and will is taught by experts across the industry, housed in Corning facilities in North Carolina, and serve needs across the country, according to the duo.
“The program includes training on optical fiber and networking, network design, hands-on splicing, connectorization, field construction for cable deployment, testing, and system turn-up,” explained Luong.
The training program will also include network system lab visits and technician ride-alongs. Upon completion, trainees will be ready to fill needed roles at carriers, construction firms, and broadband providers.
The road ahead
Success with staffing will determine if carriers meet their already stated deployment deadlines.
“Collaborating with Corning, the largest manufacturer of fiber optic cable in the U.S., will help AT&T get closer to attaining our goal of reaching 30 million locations with fiber by 2025,” said Luong in the blog post.