Microsoft said the corporate account of one of its engineers was hacked by a highly skilled threat actor that acquired a signing key used to hack dozens of Azure and Exchange accounts belonging to high-profile users.
The disclosure solves two mysteries at the center of a disclosure Microsoft made in July. The company said that hackers tracked as Storm-0558 had been inside its corporate network for more than a month and had gained access to Azure and Exchange accounts, several of which were later identified as belonging to the US Departments of State and Commerce. Storm-0558 pulled off the feat by obtaining an expired Microsoft account consumer signing key and using it to forge tokens for Microsoft’s supposedly fortified Azure AD cloud service.
The disclosure left two of the most important questions unanswered. Specifically, how was a credential as sensitive as the consumer signing key stolen from Microsoft’s network, and how could it sign tokens for Azure, which is built on an entirely different infrastructure?
On Wednesday, Microsoft finally solved the riddles. The corporate account of one of its engineers had been hacked. Storm-0558 then used the access to steal the key. Such keys, Microsoft said, are entrusted only to employees who have undergone a background check and then only when they are using dedicated workstations protected by multi-factor authentication using hardware token devices. To safeguard this dedicated environment, email, conferencing, web research, and other collaboration tools aren’t allowed because they provide the most common vectors for successful malware and phishing attacks. Further, this environment is segregated from the rest of Microsoft’s network, where workers have access to email and other types of tools.
Those safeguards broke down in April 2021, more than two years before Storm-0558 gained access to Microsoft’s network. When a workstation in the dedicated production environment crashed, Windows performed a standard “crash dump,” in which all data stored in memory is written to disk so engineers can later diagnose the cause. The crash dump was later moved into Microsoft’s debugging environment. The hack of a Microsoft engineer’s corporate account allowed Storm-0558 to access the crash dump and, with it, the expired Exchange signing key.
Normally, crash dumps strip out signing keys and similarly sensitive data. In this case, however, a previously unknown vulnerability known as a “race condition” prevented that mechanism from working properly.
Members of the Microsoft Security Response Center wrote:
Our investigation found that a consumer signing system crash in April of 2021 resulted in a snapshot of the crashed process (“crash dump”). The crash dumps, which redact sensitive information, should not include the signing key. In this case, a race condition allowed the key to be present in the crash dump (this issue has been corrected). The key material’s presence in the crash dump was not detected by our systems (this issue has been corrected).
We found that this crash dump, believed at the time not to contain key material, was subsequently moved from the isolated production network into our debugging environment on the internet connected corporate network. This is consistent with our standard debugging processes. Our credential scanning methods did not detect its presence (this issue has been corrected).
After April 2021, when the key was leaked to the corporate environment in the crash dump, the Storm-0558 actor was able to successfully compromise a Microsoft engineer’s corporate account. This account had access to the debugging environment containing the crash dump which incorrectly contained the key. Due to log retention policies, we don’t have logs with specific evidence of this exfiltration by this actor, but this was the most probable mechanism by which the actor acquired the key.
Addressing the second mystery, the post explained how an expired signing key for a consumer account was used to forge tokens for sensitive enterprise offerings. In 2018, Microsoft introduced a new framework that worked with consumer and enterprise cloud apps. Human errors prevented a programming interface designed to cryptographically validate which environment a key from working properly.
The post continued:
To meet growing customer demand to support applications which work with both consumer and enterprise applications, Microsoft introduced a common key metadata publishing endpoint in September 2018. As part of this converged offering, Microsoft updated documentation to clarify the requirements for key scope validation—which key to use for enterprise accounts, and which to use for consumer accounts.
As part of a pre-existing library of documentation and helper APIs, Microsoft provided an API to help validate the signatures cryptographically but did not update these libraries to perform this scope validation automatically (this issue has been corrected). The mail systems were updated to use the common metadata endpoint in 2022. Developers in the mail system incorrectly assumed libraries performed complete validation and did not add the required issuer/scope validation. Thus, the mail system would accept a request for enterprise email using a security token signed with the consumer key (this issue has been corrected using the updated libraries).