Coupons as Cognitive Malware: Attacking Interconnected Systems

Dr. Matthew Canham

Imagine that you receive an email from your electricity provider offering a 50% discount on electricity for using power during certain times of the day. The message explains that rising demand for electricity costs caused by increased data center usage is stressing the power grid and the electricity provider is exploring ways to encourage users to increase usage during prescribed times to reduce straining the power grid. The message appears legitimate and there are no indications that it contains malware.

Would you alter your electricity usage times to take advantage of the discount?

This was the question posed to study volunteers by a group of security researchers. The researchers wanted to establish a baseline for how many people would take advantage of this discount. They then fed the response rate into a power grid simulation and discovered this was sufficient to cause a total or partial shutdown of the power grid.

This study is fascinating because it demonstrates how employing cyber deception could weaponize user behavior to cause a severe disruption of a physical system. While this was only a proof-of-concept demonstration, the technical feasibility, and ease of implementation, suggest that attacks like this one are likely, if they have not already occurred.

The Physical, Cognitive, and Cyber Domains: An Overview

When considering security, we may conceptualize three domains: physical, cyber, and cognitive.

Physical Security. This domain focuses on preserving the security and safety of physical assets, including both people and objects. An example an attack within this domain was observed during a string of attempts to damage electrical power infrastructure using firearms and other manual tools such as axes.

Cyber Security. The cyber domain focuses on securing information assets and by extension the systems which facilitate the storage and transmission of those information assets. Examples of attacks against cybersecurity include the stories of ransomware infections which seem to occur on a weekly basis. The capabilities offered by connecting physically disconnected individuals opened vast new attack surfaces never possible in an exclusively physical domain of operations. Electronic communications gave rise to the emergence of near instantaneous and widespread effects in the cognitive domain which previously was limited in reach.

Cognitive Security. The key differentiator of cognitive security is that this domain requires securing an individual’s decision-making actions and behaviors which act in response to information which may be transmitted through the cyber (or physical) domains. It is important to note that a human is not required to be that individual making a decision. One of the key differentiators between cognitive and psychological operations is that cognitive attacks may apply to humans and/or machines using the same, or similar, techniques. The principles underlying cognitive attacks have existed for millions of years, as demonstrated by the deceptive forms of animals in nature, such as a moth that looks like an owl. What’s new is the emergence of the interconnected digital world and semi-autonomous artificial systems (both digital and physical).

Vulnerabilities Arising from Interconnected Systems

Modern systems are becoming increasingly interconnected and the delineation between cyberspace, physical space, and cognitive space is rapidly disappearing. Our world has become an interconnected system-of-systems and as such, vulnerable in ways which are difficult to predict. Attacks which occur in one domain very often induce impacts to the operation and services of other domains.

This interconnectedness enables an attacker to deliberately take an action in one domain to cause an effect in an (apparently) unconnected third domain. What’s more is that the disconnected nature of security operations means that physical security operators rarely become involved with cybersecurity operations and vice versa; and cognitive security is not even considered by most organizations at all. This provides a unique opportunity for threat actors not only covertly induce effects through inter-domain attacks, but they gain the additional bonus of exploiting the gaps in bureaucratic security silos which rarely communicate with each other.

Let’s consider this concept through the lens of the power grid example in which hypothetical threat actors influenced the cognitive domain (electricity consumers), using messaging delivered via the cyber domain (discount offers delivered through email), to produce an effect in the physical domain (crashing the electrical grid). Consider what this might look like from the defender’s perspective. Power grid operators notice a sudden increase in usage by legitimate electricity customers with no apparent explanation. The cybersecurity operators observe no indicators of an attack. The most optimistic scenario is that some of the customers become suspicious and report the fraudulent discounts, and who do they report this to? The cybersecurity operations center? Customer service? The fraud department? And what do each of these departments do with that information? Cybersecurity operators find no indications of malware in the messages; the customer service professionals can confirm that the message did not originate from the power company but then what? And the fraud department cannot find any indicators that the threat actors are trying to profit from the hoax. A potentially well-coordinated attack by a nation-state, deliberately targeting national critical infrastructure, might appear to be nothing more than a prank.

The following examples argue that this represents a new style of attack and an evolution in cyber-physical-cognitive security which deliberately seeks to exploit gaps in the interconnection between cognitive, cyber, and physical systems.

Interconnected Systems Attacks

Attacks which deliberately seek to induce covert effects are differentiated from more traditional attacks in that attackers deliberately attempt to covertly manipulate one domain by inducing an effect in an (apparently) separate third (intermediary) domain. Moreover, attackers deliberately use the properties of one domain to circumvent the defenses of the target domain, while also using the intermediary domain to conceal the nature and origin of the attack. While these attacks involve multiple domains, a key factor is that the effected domain is unable to have direct awareness of the attack. Thus, manipulating the characteristics of multiple domains in concert, the attackers can induce effects on the targeted domain covertly.

Weaponizing Tinder

A solider receives a "like" through social media (cyber domain), which they decide to respond to (a cognitive domain), leading to the exposure of their physical location (physical domain).

Shortly after Russian soldiers began invading Ukraine, they began looking for dates on Tinder. In response, several attractive Ukrainian women began “swiping right”, meaning that they accepted the Russian soldiers as a match. Accepting invites from these soldiers meant that these account users were now aware of their approximate distance from these soldiers. The problem for the soldiers in this case was that they were not actually matching with potential dates but were in fact engaging with fake accounts. When these fake accounts were advantageously positioned at geographically known locations, intelligence collectors were able to triangulate the locations of these soldiers and relay this information to Ukrainian intelligence for kinetic action. A few years earlier, members of the U.S. military reportedly executed a similar ploy against Norwegian Defense Forces during a military exercise. In these examples, troops using the Tinder app received a “like” from a prospective match (a signal received through the cyber domain), to which the soldier responds (a decision action in the cognitive domain), which then revealed the position of that soldier (an operational security breach of the physical domain).

Recruiting Insider Threats

In May 2021, Egor Igorevich Kriuchkov pleaded guilty to attempting to recruit a Tesla employee to install ransomware from inside the company’s internal network in exchange for $1 million. This attempt was unsuccessful but let’s break it down. The threat actor attempted to recruit an employee with privileged access by enticing them with a monetary reward (a cognitive inducement) to physically access an organization’s internal information infrastructure (an action within the physical domain) to install ransomware from within the on-premises network (cyber domain), allowing the attackers bypass traditional perimeter defenses.

An insider threat is incentivized with the promise of monetary reward (cognitive domain) to plug a USB device into an internal network (physically introducing malware), which then deploys malware on the internal network (a cyber effect).

While the recruiting of insiders to launch attacks is not new, using social media (cyber domain) to assess and recruit them is a relatively recent evolution which is on the rise. A slight variation of this scenario may have occurred in the introduction of Stuxnet onto centrifuge. This Iranian system is alleged to have been “air gapped” with no direction connection with outside networks, and for that and other reasons, a connected USB was the likely source of infection. If these reports are accurate, then the most likely attack vector was a privileged insider with direct access to these systems. If this insider was recruited by a threat actor targeting those systems, this recruitment effort would have been an inducement through the cognitive domain, which lead to an action in the physical domain (connecting the USB), which then affected the cyber domain by introducing the malware.

Placing People Where You Want Them to Be

The previous examples illustrate how inducing the actions within the cognitive domain (human users) can negatively impact physical infrastructure. Other malign actors may wish to influence people with the intention of maneuvering them in physical space to cause them harm, impede their progress, or influence them to avoid a particular location or take a certain route. While not deliberately malicious, the Waze traffic app routed drivers toward wildfires which were burning in the West Los Angeles area, where the streets leading through typically congested traffic routes indicated little or no traffic in the area (because the wildfire had enveloped the roads).

A phone with navigator app in the physical domain, produces virtual traffic congestion in the cyber domain, causing the driver to alter course (a cognitive effect).

While this example was unintended, the effect could easily be accomplished by manipulating virtual traffic patterns using a technique like that used by a Berlin artist when he placed 99 smartphones, all running Google Maps navigator, in a toy wagon, giving the appearance of heavy traffic congestion on streets which were in fact empty. In this (proof-of-concept) example, the inducement occurred in the physical domain by using the devices themselves (the smartphones) to create an artifact in the cyber domain (the virtual traffic congestion), which could have produced an effect in the cognitive domain (human drivers or algorithmic navigational systems). This inducement mechanism was covert because commuters would have needed to disregard their navigation systems to become aware of the deception. It does not require a significant leap of imagination to speculate how such a ploy could be utilized to cause or enhance malicious acts. Such virtual traffic congestion could be used to reroute first responders to take less direct routes to respond to emergencies, or to funnel potential victims into a targeted area. As the real-world example of drivers following Waze directions into a wildfire, demonstrates user willingness to follow the directions of an algorithmic navigational assistant even when contradictory environmental information is readily observable. Alternatively, autonomously operated vehicles might similarly be deceived without a human driver even being present in the vehicle, demonstrating how the cognitive domain could be influenced in a similar attack without the need for a human-in-the-loop.

Attacks such as these have already been carried out at a smaller scale by criminals creating Pokemon Go Beacons as a means of luring victims to predetermined locations for the purpose of robbing them. Reports have also surfaced of child predators attempting to entice potential child victims in a similar manner by employing virtual Beacons as lures.

Attacking the Human Sensory System

Security researcher Matt Wixey has published the results of a proof-of-concept attack in which smart speakers were repurposed to generate ultrasonic (high-frequency) and infrasonic (low-frequency) noise through the introduction of malware to those speakers. His research team found that certain brands of speakers could generate noise that is beyond the range of human hearing, at levels which would produce irritation or harm to a human within range of the sound. Using this proof-of-concept study to speculate on a hypothetical attack targeting the human cognitive domain could be induced through the physical conduit of sound after the introduction of malware through the cyber domain. If a human target were within range of such sonic attacks during sleep, repeated exposure to harmful levels of infrasonic or ultrasonic noise could severely disrupt sleep patterns, severely degrading cognitive abilities, or could causing severe harm to the victim’s hearing. Moreover, because this exposure would be outside of the range of human hearing perceptions, there would be no readily apparent cause for the resulting malaise.

Malware delivered through the cyber domain, produces ultrasonic noise in the physical domain, which degrades the cognitive abilities of the target in the cognitive domain.

A slight variation in this type of attack was observed when the Twitter account of the Epilepsy Foundation was compromised in 2019, and when their website was compromised in 2008. In both cases the attackers caused strobe-like flashing animations to appear to the user viewing the screen. The flashing screens were intended to induce a seizure in those people who were most vulnerable to this type of visual stimulus. Again, this demonstrates how the covert insertion of malware (cyber domain) produced stimuli (electromagnetic energy transmitted in the physical domain) to induce a cognitive effect.

Conclusion: Implications for Defenders

Security professionals need to be aware of the importance of the cognitive attack surface (both human and AI) which is rapidly emerging. While the implications of these threats are still evolving, defending against such attacks will unquestionably require a systems approach to security, meaning that intended effects may emerge only as second or third order effects in an apparently separate domain. The evolution brought about because of our increasingly connected world will demand that security professionals be trained in basic systems engineering and should consider secondary and tertiary effects in their threat models.