LCD displays emit inaudible sound frequencies which, when processed by an artificial intelligence system, can be used to determine (with surprising accuracy) the content being displayed.
Data leaks often have the most improbable of origins. A group of researchers has pointed out that LCD displays emit ultrasonic frequencies when they are in operation. By taking a closer look at this phenomenon, they realized that these frequencies, which are inaudible to the human ear, make it possible to determine the on-screen content of an LCD display. In particular, these researchers demonstrated that it is possible to discern the characters being inputted via a virtual keyboard and to identify the words being displayed on a website being viewed.
What’s more, this extravagant form of espionage does not require any particularly sophisticated equipment. The audio recording capabilities of a webcam or smartphone are more than sufficient to capture these ultrasonic frequencies – which could open up all sorts of new avenues of attack to hackers. For instance, a hacked smartphone, if placed in proximity to an LCD display, could be used to record these inaudible frequencies. The researchers dubbed this form of attack “Synesthesia” and presented their findings on this website.
Automatic learning can be a good thing…
According to the researchers, these ultrasonic frequencies originate in an LCD display’s power circuitry. In the process of generating an image, the display consumes energy and, for what is still a rather obscure reason, induces a slight vibration in its power circuitry’s components. These ultrasonic vibrations vary in function of the image being displayed. For this reason, these researchers asked whether it would be possible to construct an artificial intelligence system capable of deducing a display’s content based on its sonic emissions. The system they devised relies on a neural network that is trained for this type of recognition by exposing it to a number of different possible scenarios.
In this way, the system was trained, for example, to identify a word typed via Ubuntu’s virtual keyboard. The result was that the correct word always appeared among the suggested possibilities generated by the artificial intelligence software. When it came to extracting the words appearing on a website (the address of which was known in advance) such as the authentication form of a bank’s website, the artificial intelligence system was able to guess successively inputted characters with a success rate of between 88 to 98%. This algorithm was also able to identify the address of a website being visited with an accuracy of 97% (on condition that this website was among the ones provided to it during its training).
… it can also be a bad thing
These impressive results were obtained by training the artificial intelligence system on the same LCD display it would later target. Under real-life conditions, an attacker would be hard-pressed to employ the same method, diminishing the potential danger of such an attack. Nevertheless, the researchers determined that training their system on an identical or similar model of LCD display to the targeted device still produced interesting results, albeit less impressive ones. Therefore, in the context of a targeted attack, this method could potentially yield precious information.
Protecting yourself against this type of espionage is not easy. It is not possible to blanket a display with a noise-insulating material since this would interfere with its cooling system. According to this group of researchers, the most promising solution would consist of incorporating an ultrasonic noise generator into an LCD display since this noise generator would be capable of drowning out the display’s ultrasonic emissions.