Most of the ancients’ business was conducted not with coin or even paper but with electric tokens of money sent flying through the air. When the Apocalypse overwhelmed them, their devices failed and their wealth vanished.
The Second Sleep, Robert Harris
I just finished reading The Second Sleep, the latest book by Robert Harris. Last year every fiction title I rated 5 stars was by Harris, so I was looking forward to it.
What is different about The Second Sleep is how it looks into the future rather than being historical fiction like most of his other books. Indeed, it is set almost 1,000 years after an unspecified apocalypse where the church has been revitalised as the dominant organisation at the centre of everyone’s life.
There is historical precedent for civilisation collapse. Pretty much every society has failed for various reasons. Examining past civilisations shows an average lifespan of 336 years with most disappearing entirely. Some such as the Chinese and Egyptian transformed into different societies and others like Rome were revived into other countries.
Early in the book the main character, Fairfax, comes across an old letter written by a scientist predicting 6 scenarios:
Imperial College, London, 22 March 2022
Dear Colleague
…
Three months ago I formed, with a group of like-minded colleagues, a working party to consider what contingency measures should be taken to prepare for a systemic collapse of technical civilisation. If you do not share this concern, and especially if you regard such a notion as alarmist nonsense, please discard this letter now! We have broadly identified six possible catastrophic scenarios that fundamentally threaten the existence of our advanced science-based way of life:
1. Climate change
2. A nuclear exchange
3. A super-volcano eruption, leading to rapidly accelerated climate change
4. An asteroid strike, also causing accelerated climate change
5. A general failure of computer technology due either to cyber warfare, an uncontrollable virus, or solar activity
6. A pandemic resistant to antibiotics
…
We regard our society as having reached a level of sophistication that renders it uniquely vulnerable to total collapse. We regard our society as having reached a level of sophistication that renders it uniquely vulnerable to total collapse. The gravity of the threat has increased vastly since 2000, with the transfer of so much economic and social activity to cyberspace, and yet there has been no corresponding contingency planning at government level.
…
The Second Sleep does not explain what caused the apocalypse but instead suggests several possible triggers. Despite the letter, what these scenarios might be are unknown to the characters. Concepts such as cyber warfare and antibiotics are as foreign in the time the book is set in as they would have been before they were invented.
The angle Harris considered is somewhat unusual due to how far in the future it is. The nature of these apocalyptic triggers remain a mystery to the characters because so much knowledge has been lost. His contention is that we rely so much on electronic means of knowledge that should we lose access to our computer systems, we will have lost all ability to exist as a the civilisation we know.
In this post I wanted to consider how much we actually rely on computer technologies and what the risks might actually be.
Risks and contingency planning
Flooding. Car accidents. Crime. Governments have extensive contingency plans in place for a wide range of different scenarios.
Just researching where I live, London has 13 detailed planning documents for situations ranging from pandemic to building collapse. The London Risk Register plots out many more scenarios and explains their likelihood, impact, and the mitigation planning that is in place.
Although several of the scenarios in the letter appear on the risk register, Harris is actually writing about something a bit more subtle: the loss of knowledge due to our reliance on computer systems, all of which require electricity.
As interesting as they are, the specific scenarios that cause the apocalypse are less relevant. The real, long lasting damage is actually inflicted by the loss of electricity which follows. It is “the transfer of so much economic and social activity to cyberspace” which introduces this un-planned-for fragility into the global system.
H41: Systems Failure – National Electricity Transmission
Examining the London Risk Register, the key risk highlighted by The Second Sleep is is H41: National Electricity Transmission.
Consider how much of your life requires electricity and you will realise why this is categorised at the highest impact level in the register. Just thinking about my usage of electricity today explains why this is categorised at the highest impact level in the register.
My phone charged overnight to instruct my watch to buzz to wake me up (which I then charged after getting out bed). I used an electric kettle to make tea and ate yoghurt that was stored in an electric fridge. The water in the shower was heated through a boiler that requires electricity to ignite and run the electronic control board. I checked my email using a laptop and router connected to fibre equipment in the street cabinet, all powered by electricity. I read a book on a Kindle that was charged and was then reminded about a school meeting by my phone. I had to cross the road using electric traffic lights and the school security system is powered by electric door locks. My mobile phone connected to a cell tower, connected to the internet backhaul to a central data centre, all powered by electricity.
And this only brought me to lunch time!
We all rely on the electricity grid to provide electricity on demand, any time of the day or night. It’s a crucial aspect of our lives and we have zero control over it.
So what could we expect in a system failure of the electricity grid? What is scenario H41?
H41: Systems Failure – National Electricity Transmission
London Risk Register 2019
A total national blackout due to the loss of the GB National Electricity Transmission System caused by damage to or technical failure of the transmission network. The technical recovery process (Black Start) could take up to 5 days; however, there is the potential for wide area power disruptions for up to 14 days, potentially affecting millions of consumers.
This is assessed as a 3 in the likelihood category which means the chances of it happening in the next 5 years are > 0.5% i.e. there is a 1 in 200 chance. To put this into perspective, the threat of conventional attacks on the transport system are considered to be in category 5 of likelihood i.e. there is a 1 in 2 chance of such an attack happening in the next 5 years.
What does history tell us about major electricity failures?
The UK has never seen a country-wide electricity failure. The National Risk Register stated in 2017:
a national blackout has never happened. A much less severe incident did occur between 22 and 28 December 2013 when, as a result of two severe winter storms and consequent damage to the distribution overhead line network, around 900,000 UK customers suffered a loss of electricity. 876,000 customers had power restored within 24 hours, however 16,000 experienced disruption for longer than 48 hours. Severe flooding in some areas has also led to instances of local power loss, such as in Lancaster during the 2015-16 floods.
National Risk Register Of Civil Emergencies 2017 edition
Since then there have been several other major electricity outages, such as the 2019 blackout which affected over 1m people:
Large parts of England and Wales suffered power cuts on Friday night after a major network failure, affecting around 1m people and causing widespread disruption to trains and roads ahead of a busy weekend for holiday travel. The loss of power lasted for under an hour before it was restored but the disruption continued into the evening. The National Grid electricity system operator said: “This evening we had an unexpected and unusual event, the loss of two generators that connect to the GB transmission system, which led to a fall in the frequency of the electricity system.
Power outage hits large parts of England and Wales, Aug 9 2019
Although these incidents are inconvenient, they generally last just a few hours for most people. Blackouts in other countries have not been resolved so quickly. New York saw several days of problems following Hurricane Sandy in 2012:
Every age gets the metaphorical crises it deserves, and New York’s came in 2012, when Hurricane Sandy hit the city and caused power outages across half the city. I was there, and at first everyone was really nice to each other. Within a few days, a Mad Max vibe began to creep into daily interactions. The lights came back on in time to get the city more or less back to normal, though not everywhere.
Why New York City Is On the Verge of Disaster
The storm was the immediate cause of the blackout, of course, but the storm took advantage of an electrical infrastructure weakened by years of poor investment choices. We know this because a few months after the storm, the Utility Workers of America, the union negotiating with Con Edison, released a report on the company’s operational practices, alleging that “Con Edison appears to operate its electric distribution system based on a policy of “run it until it fails.’”
And the US suffered an outage affecting 50m people in 2003.
More recent outages have lasted between a few hours and a whole day. But what happens if there is a national outage which lasts longer?
Black start
Clearly we have been able to survive short outages but when so much of society runs on electricity, what happens if it is no longer available for prolonged periods of time?
In the event of a major electricity outage, you would be unable to connect to the internet because your router would be down and power to the street cabinets that run the fibre network would also be offline. The old-fashioned telephone network is resilient to failure for up to 5 days but the mobile phone network would fail within 2 hours.
Within 2-6 hours all public communication except BBC Radio 4 would stop and within 12-48 hours, the water supply would also fail (London Power Supply Disruption Framework Oct 2018 p8).
Consider how much of your work involves services you have no control over. You need electricity for your laptop to turn on, to connect to a wireless network which provides internet access to servers hosted in a data centre somewhere in the cloud. How much do you actually have stored locally?
An entire generation’s correspondence and memories had vanished into this mysterious entity the antiquarians called ‘The Cloud’. The few records that remained were mostly to be found in the ordinary parish churches – buildings made of stone and intended to last, which had continued to stand even as the newer settlements around them crumbled into ruins.
The Second Sleep, Robert Harris
The worst case scenario is a nationwide outage that requires a cold start of the national power generating infrastructure:
if the whole of the network, including the power stations supplying the national high voltage electricity distribution network (the grid), loses power, you would need to restart those individual power stations before they can get the grid powered up again.
The challenge? Normally, all power stations need electrical supply to start up. But with a total electricity blackout, there’s no electricity to restart the system.
That’s why the reboot procedure is called ‘Black Start’ – and it’s one the most important, yet little-known back up plans in the UK.
Black start: the most important back up plan you’ve never heard of
Such a scenario may require up to 7 days to restore full service with days 2-7 involving rolling blackouts (3 hours of power):
UK Power Networks have set out the phases to restore the national power network from the time of the outage to the full restoration of the network. The actual restoration timeframe could vary, depending on the cause of the outage, but it could be up to 7 days. Designated Black Start power stations will become operational in the first two hours and will provide power to progressively restart the rest of the network. In this scenario, London could be one of the last regions to be reconnected.
London Power Supply Disruption Framework
A national black start has never been required but it was used regionally in 1987:
In October 1987, there was a regional Black Start in the wake of the powerful hurricane that hit the south of the country. The storm damage left Kent and Sussex disconnected from the National Grid – but thanks to Black Start contingency plans, most people barely noticed. Kingsnorth Power Station restored power to the area and it ran independently, cut off from the rest of the Grid, until repairs enabled it to be connected up again.
Black start: the most important back up plan you’ve never heard of
The fact that a national outage has never happened should provide some reassurance. In London, the power network has seen improving reliability year over year:
But it is not business as usual that we need to worry about. It’s exceptional events that pose challenges for contingency planning.
Storm David with wind speeds of 71-93MPH in January 2018 is the most recent example of weather-based disruption and power was restored to 99% of customers in 24 hours. UK Power Networks (who are responsible for the electricity distribution in London and the South) deployed 1,000 additional staff and answered over 16,000 calls with an average time to answer of 5 seconds, 10 times the number of calls received in a normal day (source). This shows that contingency planning is in place and has been successfully deployed in the recent past.
But what would be the impact of a longer duration of power outage, or permanent loss?
How do you access knowledge without electricity?
History was a patchwork of voids. The great university libraries and public archives had mostly rotted away or been used as fuel in the Dark Age.
The Second Sleep, Robert Harris
It seems sensible to digitise everything. I read non-fiction on my Kindle because I like to highlight and save notes (also electronically, in Apple Notes). All my records and files are kept on encrypted storage locally, sync’d via iCloud Drive and backed up to B2 using Arq. And using WhatsApp and email means all my communication is electronic as well (although they auto-delete after a time).
Consider how you look up information on a daily basis. Whether it is Wikipedia or another site from web search, everything starts electronically. Even if you end up reading a book or a research paper, how often do you use the services of your local librarian to help research? And what do they use to search on your behalf?
Library usage is declining (although that trend may be slowing), but that doesn’t mean investments aren’t being made. The British Library is a great example of a long term vision being put into action as the largest and oldest copyright library in the world.
The British Library is the custodian of one of the largest cultural heritage collections in the world. With more than 150 million items in both physical and digital formats, the collection grows rapidly, with 8km of physical items and 150Tb of digital items for long-term preservation being added every year.
BL North
Whether it is improvements to the famous British Library London St Pancras site or the expansion of BL North where a large volume of items are stored, libraries can be considered not just sites for research and engagement but also a crucial physical backup of knowledge.
Other organisations, such as the Long Now Foundation, are attempting to tackle the problem of durable long term storage.
Think about the technology that was in use for file storage over the last 10-20 years. VHS – players no longer easily available (production ceased in 2016). Zip disk – a proprietary format no longer in mainstream usage. CD – laptops are rapidly ditching drives (I’ve not owned a computer with a CD drive for at least 5 years now).
Not only do we need to think about the format the data is stored in but also the media it is stored on. There’s a reason why all UK legislation was printed on vellum until 2015 (now on archival paper).
Space weather
Since the last major space weather event (The Carrington Event of 1859), the world has become significantly more reliant upon digital storage of information. Electrical failure generally means that systems are unavailable right now, but they will come back online soon when the electricity supply is restored. Space weather poses an additional risk of permanent data loss.
Space weather is listed as a medium risk (3) and high impact (4) on the London Risk Register, also being present in similar categories on the UK National Risk Register. Caused by solar flares, solar energetic particles and coronal mass ejections, it can cause major disruption to electricity grids:
H56: Space Weather
London Risk Register 2019
Disruption to the electricity grid, resulting in two rural/coastal sub-station disconnections each effecting communities of approx. 100,000 people, with loss of power for 1 month or more and rota-disconnections for a further 1 month or more. Voltage instability may also result in local blackouts, most likely in urban areas lasting a few hours.
It can also cause problems with key systems like GPS, communication networks and even cause data loss to sensitive storage equipment.
The Carrington Event in 1859 is the largest space weather event on record. Telegraph systems were heavily disrupted and the Aurora Borealis was sighted as far south as Mexico, the Caribbean and Sub-Saharan Africa.
Other significant space weather events have been recorded since then. A Carrington-scale CME was measured by the Stereo-A satellite in July 2012. The path this took from the Sun narrowly missed the Earth. A smaller storm in 1989 tripped the equipment protection systems of the Hydro-Québec electricity network in Canada, resulting in a loss of power for nine hours across the province.
Another storm in 2003 caused the UK aviation sector to lose some GPS functions for a day.
National Risk Register Of Civil Emergencies 2017 edition
The UK Met Office provide space weather forecasting services and the electricity industry already use shielding to protect equipment. There is a detailed government preparedness strategy even though the annual probability of a Carrington level event is low at just 1%.
Practical conclusions
It is sensible to take personal precautions to the same degree that well run technical projects plan for system problems. In cloud infrastructure this means deploying across multiple zones and regions, regularly testing backups and failover plans and building for failure.
Deploying across multiple regions is essential to mitigate the risks discussed here. Geographies matter. Your regional backup needs to be sufficiently diverse to be outside of localised (earth and space) weather as well as resilient to electricity grid failure. This typically means deploying in different countries on different continents.
The largest companies do extensive disaster planning, such as Google simulating the loss of its HQ and how that impacts the operations of the rest of the company. And emergency services also run large scale simulations for training.
These are quite extreme for individuals, especially when the likelihood of being impacted by such an event is very low. Instead, small scale planning is prudent for individuals.
Considering the risks discussed above, dealing with electricity loss for several days (up to a week) is a reasonable worse case scenario. The US Government has a whole website with practical advice such as keeping a supply of water and food and the London website also suggests things like having a grab bag ready, which should include non-digital essentials like a map of your local area.
This is not about building a nuclear bunker (and some do think it is scaremongering) but taking simple and sensible precautions to ensure you don’t run out of critical supplies – water being the most important.
Once you have taken care of the essentials, you can turn to your digital assets.
The beauty of online document products like Google Docs are that all the complexity of sync and storage is abstracted away. Do you understand how the critical systems you rely on actually work? Are you satisfied that your online documents are properly replicated across multiple regions? Remember that backup is not the same as sync and files that appear on your computer may actually just be links to the cloud version (as is the case with Google Drive).
Where are your photos, files and documents? Does this include statements for your new, mobile-only bank? Do you understand how those files are represented and synced? Do you have backups? How can they be accessed if the service is down? What about all those messages from family that have sentimental value?
What is your black start scenario plan?
Of course this is all very unlikely but the difference is that instead of relying on a government that is legally required to make (and publish) contingency plans, all our digital assets tend to be held by private entities. The cloud and SaaS has many benefits, and this is not an argument against either, but it is prudent to consider whether you really understand how these systems actually work. What are your assumptions about their availability? How much are you relying on them?
Hancock said, ‘What manner of people would pamper themselves with heating glass?’ ‘A decadent people,’ responded Shadwell, ‘which I conjecture was part of their undoing, as it ever has been in the history of civilisations. The Romans depended on slaves, the ancients on science. They made their lives too luxurious and in the end rendered themselves helpless.’
The Second Sleep, Robert Harris