Maersk
In 1940 Maersk Had a Standing Instruction. In 2017 It Had an Outage.
The day before Denmark was occupied, Maersk's founder told his captains which orders from head office to stop trusting, and whom to ask instead. Seventy-seven years later, controls that the company's own identity specialist had been requesting for two years were still waiting when the network went down, and the directory was rebuilt from one copy that happened to be offline. Why the same company met the same problem by design once and by luck once.
On 27 June 2017, a piece of malware later named NotPetya spread through the network of A.P. Moller - Maersk, then the world's largest container shipping company. [7] By the account of its technology chief, about 55,000 devices were infected within seven minutes. One of the first things the malware took was the credential of a domain administrator, an account with administrative rights over the Windows machines joined to a company's domain. [8]
Maersk kept roughly 150 copies of its directory, the record that tells a network who each person and each machine is and what each may do. The copies were spread around the world and kept in step with one another, so that losing one would not matter. [7] That day the company believed it had lost them all. [8]
One copy was found. An office in West Africa had lost its power, so its copy was offline when the attack came. The hard drive was flown out, and Maersk's technology chief called it the yeast from which the rest of the network was grown again. [7] [8] By its chairman's count, the company installed 4,000 servers, 45,000 PCs and 2,500 applications in about ten days, a job he said would normally be quoted at six months, and still moved about 80% of its volume in that time, by hand. [10] Maersk forecast a hit to its results of USD 200–300 million. [5] Its revenue for the year still grew by 13%. [6]
A weakness in administrator access had been pointed out. Inside Maersk, a specialist had been asking for two years for controls on what administrator accounts could reach. By another account a wider redesign of the network's security, proposed in 2016, had been approved and given a budget. Neither was done. [7] [13]
One reading stops at the cost, and the lesson it draws is to patch, to segment and to back up. Banseog's answer is different. Maersk had once faced the same question, in another material, and answered it in one telegram: when an order arrives in head office's name, who outside head office can confirm it, and what is kept out of head office's reach? In 1940 Maersk wrote the answer down and sent it to its ships. In 2017 a power failure supplied half of it.
A playbook written long before the fork
Maersk traces its own beginning to a loss. In 1883 the ship that Captain Peter Mærsk Møller sailed, VALKYRIEN, was wrecked off Scotland, and he was left without an income. Three years later he bought a small steamship, convinced that steam was the future. [3] He put a white star on its funnel, to remember an evening when he had looked for one star in a grey, overcast sky and it had appeared. It became the emblem of the fleet. [2] His son, A.P. Møller, founded the shipping company in 1904. [2]
The Second World War cost Maersk 25 ships and the lives of 150 seafarers. At its end the fleet was down to 21 ships, seven of them under Danish control. Within three years the tonnage was back to its pre-war level. [1] In March 2018 a fire broke out aboard MAERSK HONAM. By September the company had reviewed how dangerous goods are stowed and put new rules into force across more than 750 vessels. [11]
What this record shows is a company that is very good at the day after. When something is lost, it rebuilds faster than anyone expects, and it writes the rule that the loss taught. That is our reading, not Maersk's description of itself, and it is a strength. Its other side is that the day before has fewer entries. Care taken before a loss competes with nothing visible. Nothing is missing yet.
The founder seems to have seen this in his own house. The sentence he left to it reads: "No loss should hit us which can be avoided with constant care." [3] Maersk still lists constant care first among its values. [4] A company rarely writes into its values the thing it does without effort.
There is one entry for the day before that stands out, and it is exact.
Permanent Special Instruction One
On 9 April 1940 German forces occupied Denmark. The Maersk fleet numbered 46 ships, and 36 of them were outside Danish waters. A.P. Møller had sensed what was coming, and feared that the occupier would try to take control of the whole fleet. The day before, he and his son sent a telegram to the ships abroad. [1] [12]
Maersk's own history summarises it as an order to make for neutral ports and to stop following headquarters. [1] The text, preserved in the founder's biography and reprinted in a Danish maritime museum's yearbook, is more exact than the summary, and more interesting. [12] It ends with its own name, PERMANENT SPECIAL INSTRUCTION ONE, and it does five things.
It names a condition: if Denmark becomes involved in the war. It names a person and an address outside Denmark, the founder's cousin and partner Hans Isbrandtsen at 26 Broadway, New York, and tells each captain to report his position, voyage, cargo and fuel to him. It tells captains not to load new cargo or begin a new voyage until he has given directions. It leaves to each captain's "best judgement" whether to make for a neutral port and stay there, and says plainly that insurance may lapse and that either side may seize the ship. And it asks each captain to confirm receipt.
The centre of it is one clause. Captains were not to follow new directions from Copenhagen "unless confirmed by him".
Read that as a design. Møller did not tell his captains to ignore head office altogether. He told them that, once Denmark was in the war, a new direction from Copenhagen would count only when a named man in New York had confirmed it. The clause reads as the work of a man who expected that orders might soon leave his own office that were not his. So he put a second key outside the building while he was still free to do it, he gave the ships somewhere else to report, and he asked for proof that the message had arrived.
The instruction did not save the ships. PETER MÆRSK reached Hong Kong the day after the invasion and was seized there by the British, and the ships abroad were, in time, requisitioned by the Allies. [1] [12] What it provided was narrower and mattered more later: an authority for the ships abroad that did not run through a head office under occupation. The founder's son went out to the United States, and Maersk credits him with a central part in rebuilding the fleet after 1945. [1]
The fork: two requests, neither done
By 2017 Maersk's network was built the other way. The copies of the directory kept one another current. A domain administrator's credential carried authority across the Windows machines joined to it. No deliberately isolated copy of the directory was found when it was needed. And the recovery plans for the directory, by one insider's account, had only ever covered the loss of a site or a data centre. Nothing in them covered losing everything at once. [13]
That is efficient, and it is reasonable until the thing that is lost is the centre itself. A centre that can no longer be trusted is the case the 1940 instruction provided for.
Was Maersk careless? There are four accounts of the company before the attack, and they do not say the same thing.
Gavin Ashton was responsible for Maersk's identity and access management service at the time, and wrote under his own name in 2020. He describes two years of pushing, without result, for controls on privileged access, the controls that limit what an administrator's account may do. He describes IT run as a cost to be minimised, and several security functions with no clear lead and limited funding. [13]
Wired, drawing on unnamed people inside the company, described a wider proposal. In 2016 a group of IT executives pressed for a redesign of the security of the whole network. It was approved and given a budget. But its success was not among the measures that decided senior IT managers' bonuses, and it was not carried through. [7]
Maersk's technology chief said in 2019 that the company had not been unusually weak, and that it had been patched for months against the flaw NotPetya used. [8] And its chairman said at Davos, seven months after the attack, that it had shown Maersk to be "basically average" at cyber security. [10]
These accounts describe different requests and should not be merged. Wired points, by role and not by name, to the most senior managers overseeing IT; for Ashton's request no decision-maker is identified. But the accounts share a shape. The work had been asked for well before the attack. And both describe what came after: Wired's sources say that nearly every security feature staff asked for was promptly approved, and Ashton says that within days two access controls were being deployed, and that the following months saw the kinds of measures he had been looking to deploy get the green light. [7] [13] Whatever had been missing before 27 June, it was not a proposal.
This is why the surviving copy matters so much to this reading. Wired places it in Ghana; Maersk's executives have said Lagos. [7] [8] On the point that counts they agree. A power failure kept it clean. It was not a design. Maersk's later head of security put it in three words: "We were lucky." [9]
The ten days that followed showed what Maersk does best: by its chairman's count, thousands of machines rebuilt in what he called a heroic effort, and about 80% of normal volume handled manually. [10] The people earned everything they did in those ten days. The power failure gave them somewhere to start.
The people at the fork
In 1940 the second key was placed outside the building by the man who held the first: A.P. Møller, who led the company, with his son beside him. [1] Before 2017 the requests came from a specialist and from IT executives, and a control that limits the centre's own credential is hard to carry from below the centre. That is our reading; we have seen no assignment record.
In 2017 the chairman was Jim Hagemann Snabe, who had taken the chair that spring and came from the software industry. He was the one who said "average" in public. [10] But the week was carried further out. By one account a 23-year-old IT support employee from the African office flew with the drive. [8] In offices and terminals around the world, people took bookings and kept cargo moving without their usual booking and terminal systems.
It could have been read at the time
Scroll the table horizontally to read all columns.
| Factor | April 1940 | June 2017 |
|---|---|---|
| Warning in hand | Occupation feared; the fleet could be seized | Two years of requests to limit administrator access |
| What the plan assumed could be lost | Head office itself | For the directory: a site or a data centre, never everything at once |
| Whose word counted | Head office's, in ordinary times | The administrator's credential, across the Windows domain |
| Who could confirm a new order | Hans Isbrandtsen, New York, named in advance | No one named, in the accounts we have |
| What was held out of reach | A named authority for the 36 ships abroad | No deliberately isolated copy found; one office offline by accident |
| Proof asked for | Each captain asked to confirm receipt | The surviving copy was discovered during the crisis |
Be exact about what could be read, and by whom. The accounts behind the right-hand column were published only after the attack. The left-hand column was public: the episode is in Maersk's own published history, and the full text is in the founder's published biography. So those accounts were not available to an outside reader in 2016. The 1940 instruction was. With it he could have asked the question: which of your orders needs a second confirmation, and what do you keep out of reach? Nor can a record like this predict which Maersk would turn up, the one that acts the day before or the one that acts the day after. It does something more useful. It shows what the company already knew how to do, and in what form.
The path that was open to Maersk
What could be read could have been acted on. But the path was not to become a model of security practice. Security work had been asked for, in two different forms, and it had waited. The path that was open lay in what Maersk had already done once.
Instruction One did two separate jobs, and it helps to keep them apart. The first was a limit on the centre's own word: no new direction from Copenhagen counted until Isbrandtsen had confirmed it. In a network, the nearest job is the one Ashton was asking for, limits on what any single administrator's credential may do, so that one stolen key reaches less. The second was continuity: a line of authority that would still be there, with somewhere to report, if head office was lost. In a network, that job belongs to a copy of the directory kept offline on purpose, and to a test, on a date, that it can be restored. The power failure supplied a copy by accident. The first job was still waiting.
What the two jobs share is a decision that belongs at the top of a company in either century: what, in advance, must stay independent of head office. In 1940 that was settled in an evening and sent in one telegram. We read it as possible because it came from the top. It was the centre agreeing to be checked.
We cannot show that such a decision would have been taken in 2016 if someone had framed it this way. We can show that the security work actually proposed waited. A company's alternative seldom lies in someone else's best practice. It lies in something the company has already done well once. Maersk's was in its founder's biography, numbered One.
Afterwards: luck turned back into design
By 2019 Maersk's technology and security chiefs were describing a different doctrine. Assume the network will be taken down, and build everything around recovering and running it. Out of some 1,200 critical applications, know the fifty without which the business stops. Keep backups offline, and keep a regime that proves they work. Cut privileged access sharply. [8] [9] In our reading, that is both jobs of the 1940 instruction written a second time: something kept out of reach with a way to prove it works, and a limit on the centre's own word. This time the plans assume that the centre is what is lost.
Ashton also records resistance. The first controls began going in within days. But when it came to his own plan for the rebuilt systems, he writes, people were "quickly falling back into the old reluctance to change". [13] He later left the company. In our reading, the day after is vivid for a while. Then it becomes the day before again.
If head office sent one last message
Many companies have something that everything else takes its orders from. It may be a system, a plant, a customer or a person. Efficiency comes from keeping everything in step with it. The warning that it could fail, or be made to speak in someone else's voice, is often visible. Often the response has already been asked for, and sometimes it has been approved. Then it waits, because nothing is missing yet.
Maersk's chairman called its security average. We would put it this way: companies adapt to what they could not fix, and after two years the unfixed thing is no longer a problem. It is the way things are done.
Banseog starts by finding the way a company has chosen at its past forks. Then we look for two things in its own record: the step that was asked for and never taken, and the time the company once took a step like it without being forced. Many companies have an Instruction One somewhere in their history. Few remember where.
So the first question Banseog asks is simple. If head office sent one last message before going silent, what would it tell your people to decide on their own?
Scope: Banseog's interpretation, based on the public record of the company's choices and the order in which they were made. The description of Maersk's way of choosing and the comparison of 1940 with 2017 are ours, not Maersk's. The account of security work before the attack rests on Wired's reporting from unnamed insiders and on one named former employee's personal account; they describe different proposals, and Maersk executives have described the company's readiness differently. Sources differ on where the surviving copy was (Ghana or Lagos), who carried the drive, how long the recovery took and what it cost. The 1940 telegram is quoted from a 2011 reprint citing the founder's biography; Maersk's own summary of it is shorter and less conditional. Banseog has no relationship with A.P. Moller - Maersk.
Sources
- A.P. Moller - Maersk, "Explore our history" (sections on 1904–1912 and on the Second World War, including "The night before the occupation")
- A.P. Moller - Maersk, "A star is born", 22 December 2022
- A.P. Moller - Maersk, "The founding family"
- A.P. Moller - Maersk, "Our core values"
- A.P. Møller - Mærsk A/S, "Interim Report Q2 2017", announcement, 16 August 2017
- A.P. Møller - Mærsk A/S, "Annual Report 2017", announcement, 9 February 2018
- Andy Greenberg, "The Untold Story of NotPetya, the Most Devastating Cyberattack in History", Wired, 22 August 2018
- Dan Swinhoe, "Rebuilding after NotPetya: How Maersk moved forward", CSO Online, 9 October 2019 (Adam Banks and Andy Powell at the Gartner risk summit, London)
- Riviera Maritime Media, "Protect and survive: how Maersk learned from the NotPetya cyber attack", 26 June 2019 (Andy Powell at the Maritime Cyber Risk Management Forum, London)
- Jim Hagemann Snabe, panel on securing a common future in cyberspace, World Economic Forum, Davos, January 2018, as reported by iTnews and The Register, 25 January 2018 · The Register
- A.P. Moller - Maersk, "Maersk implements new guidelines on dangerous goods stowage", 26 September 2018
- Christian Peter Baird, "En hændelse i udeflåden", Handels- og Søfartsmuseets årbog 2011, pp. 43–45, reprinting the telegram from Ove Hornby, "Ved rettidig omhu…": Skibsreder A.P. Møller 1876–1965, p. 163
- Gavin Ashton, "Maersk, me & NotPetya", 21 June 2020