Analysis: Does the internet have a back-up plan?

At first it looks like the result of a spending spree at Go Outdoors. There's a table piled high with sleeping bags, tents, ration packs, dental kits, steel toecap boots, deckchairs, UHF/VHF radios, Savlon, water purifying tablets and even a bog-in-a-bag, but behind all of this is nothing short of a global telecoms network on wheels.

Seven trailers parked-up in a huge warehouse – some on wheels, others designed for 747 cargo holds – together create a network node capable of handling 15 Terabits per second. This is AT&T's top-secret Global Network Disaster Recovery centre somewhere in the south of England (we can't say where), and its US$16 million custom-built contents can get an entire city of smartphone users back online after a major disaster.

High stakes

The stakes are high because AT&T's network, while not publicly visible in the UK, constitutes a big slice of global telecoms and internet traffic. Worth $140 billion, it's made-up from over a million miles of network fibre at 155 sites in 60 countries. It transmits 67 petabytes of data every day. It's got over 116 million wireless customers, 649,000 Wi-Fi hotspots and 38 internet data centres.

If even one node of that network goes down during a disaster it's a big problem both for AT&T and for the people in the affected area, who are cut-off just when they most need to communicate. Ready to protect all of this – and keep everyone online and in contact with each other post-apocalypse – is a dedicated team on-call 24/7 for disaster-deployment.

The team – many of them volunteers from within the company – contains veterans of 70 'network events' ranging from the Colorado and California wildfires (2013), the Oklahoma tornado (spring 2013), Super Storm Sandy (autumn 2012), Hurricane Irene (2011), the Alabama/Tennessee tornados (2011) and the Santiago, Chile earthquake (2010).

This vast warehouse can turn a disaster zone into a communications hub. This is the tech behind tragedy.

The biggest threats

"Most of the problems that we see are power-related," says Justin Williams, Network Disaster Recovery – International, AT&T Network Emergency Management, Preparedness & Response, who took TechRadar Pro on a tour of the facility, one of six that AT&T has around the globe (one in Europe and four in the US).

"Self-sufficiency is everything, so we take our own power with us," he says, explaining that his NDR team takes generators, fully charged batteries and everything else needed to restore the network in any given scenario.

We're not just talking about natural disasters; unpredictable social and political events are also problematic. Impromptu demos and protests can bring down a node in a network if the call activity goes catastrophically high, though AT&T is constantly probing its network for potential problems.

Communications is a major challenge after an 'event'. "You can't rely on on local people on the ground because in a disaster zone they will have their own family issues," says Williams. It's a story of constant network overload. "Calling your loved ones on New Year's Eve is difficult, but try it in an area where the network is toast and you've got no chance," he says.

The NDR team takes its own satellite phones (checked and charged every week) and its own managed data links via satellite because the main problem – as in Chile after the devastating earthquake of 2010 – was a network that was bursting. "In Chile the main problem was congestion levels, and it was very unreliable – and there were aftershocks, too, the impact of which we had to monitor."

The gear

The mobile trailers at AT&T's disposal include Trailer 365 (home to all that survival gear), a five-tonnes command centre and Trailer 180, a dense wavelength division multiplexing (DWDM) truck stuffed with servers, air-con units, generators (and the occasional tea-urn and microwave) alongside an endless number of FSI cables, each with 72 fibre connections.

What's in the warehouse is constantly connected to the AT&T network and constitutes a PoP site in its own right – and one of the bigger ones, at that. After the exact configuration of the lost network node is copied to it, it's all driven or flown-in, to the disaster zone, then slotted-in to exactly replicate what was at the 'lost' site post-disaster. The entire fleet numbers 320 vehicles. "If you're talking about a 'smoking hole' scenario,: says Williams, "this is the only game in town."

Could terrorists bring down the internet?

Absolutely – it's known as a 'smoking hole'. "The World Trade Center at 9/11 is the classic example," says Wiliams. "We obviously monitor the media, but with something like the World Trade Center you see the call volumes around it just go bananas and you know instantly that there's been some major issue. That's when we deploy – it's so sudden.

" Since terrorist attacks can occur at any time anywhere on the planet and can produce almost any scenario, NDR is all about being prepared and ready to go. "If a government raises its threat level – as the UK did recently – then that triggers some activity for us," says Williams.

"We were linked to the local authorities and agencies that were alerting around Newport recently for the NATO Summit, and in most countries we have engineers on the ground constantly reporting back."

Without reliable communications humanitarian response teams can't function, but it's not just far-flung places that the NDR team heads out to in an emergency.

In the UK, it was activated and in deployment mode for the Buncefield fires of 2005, the 7/7 bombings and this year's floods, but this global hub is in the south of England for multiple reasons; as well as being a low-risk area it's geographically important because it's close to airports and roads. No wonder, then, that it's responsible for Europe and the Middle East.

"We'll drive these trailers as far as Turkey and the Middle East before considering flying," says Williams. The facility's location also means that the team is based in an accessible time zone for the Asia Pacific region, so that, too, also falls under its brief. It's currently on alert because of the heightened activity and protests in Hong Kong, with AT&T field engineers feeding back reports daily.

Next-gen networks

As AT&T moves towards an IP-based network, the NDR program is continuously innovating. The fleet now includes nine trailers dedicated to the recovery of the core, high-capacity routers that send and receive all of the network traffic from one network office to the next.

These trailers can support data traversing at rates as high as 100 Gigabits per second. When fully-equipped, the IP trailers can scale up to a capacity of over 15 Terabits per second.

"However you cut the future of networking you're still going to have large amounts of fibre and copper and access points that terminate onto hardware somewhere," says Williams. "If you're a data centre or you're hosting then you can move that data around very quickly," he adds, insisting that AT&T doesn't have that choice.

One of the biggest innovations of late is simple; charging stations for mobile phones. "After Superstorm Sandy in 2012 we developed charging stations, and now we rollout smaller vehicles with generators with plug sockets for people to charge their phones," says Williams. "It means they can phone their relatives who they may not have spoken to for several days."

In disasters, as in everyday life, communications is everything, but the sheer scale of AT&T's operation is impressive; the internet's back-up plan is always ready to roll.

from Techradar - All the latest technology news http://ift.tt/1rJSnv3