This clever AI hid data from its creators to cheat at its appointed task
Depending on how paranoid you are, this research from Stanford and Google will be either terrifying or fascinating. A machine learning agent intended to transform aerial images into street maps and back was found to be cheating by hiding information it would need later in “a nearly imperceptible, high-frequency signal.” Clever girl!
This occurrence reveals a problem with computers that has existed since they were invented: they do exactly what you tell them to do.
The intention of the researchers was, as you might guess, to accelerate and improve the process of turning satellite imagery into Google’s famously accurate maps. To that end the team was working with what’s called a CycleGAN — a neural network that learns to transform images of type X and Y into one another, as efficiently yet accurately as possible, through a great deal of experimentation.
In some early results, the agent was doing well — suspiciously well. What tipped the team off was that, when the agent reconstructed aerial photographs from its street maps, there were lots of details that didn’t seem to be on the latter at all. For instance, skylights on a roof that were eliminated in the process of creating the street map would magically reappear when they asked the agent to do the reverse process:
Although it is very difficult to peer into the inner workings of a neural network’s processes, the team could easily audit the data it was generating. And with a little experimentation, they found that the CycleGAN had indeed pulled a fast one.
The intention was for the agent to be able to interpret the features of either type of map and match them to the correct features of the other. But what the agent was actually being graded on (among other things) was how close an aerial map was to the original, and the clarity of the street map.
So it didn’t learn how to make one from the other. It learned how to subtly encode the features of one into the noise patterns of the other. The details of the aerial map are secretly written into the actual visual data of the street map: thousands of tiny changes in color that the human eye wouldn’t notice, but that the computer can easily detect.
In fact, the computer is so good at slipping these details into the street maps that it had learned to encode any aerial map into any street map! It doesn’t even have to pay attention to the “real” street map — all the data needed for reconstructing the aerial photo can be superimposed harmlessly on a completely different street map, as the researchers confirmed:
The colorful maps in (c) are a visualization of the slight differences the computer systematically introduced. You can see that they form the general shape of the aerial map, but you’d never notice it unless it was carefully highlighted and exaggerated like this.
More Top Stories
U.S. home sales tumbled to their lowest level in three years in December and house price increases slowed sharply, suggesting a further loss of momentum …read more
British opposition Labour Party leader Jeremy Corbyn moved a step closer to paving the way for another referendum on European Union membership by trying to …read more
Winter winds and ice sheets will bring extreme cold and ice-slick roads to the Midwestern and Eastern United States on Monday.read more
The U.S. Supreme Court on Tuesday let President Donald Trump enforce his policy barring certain transgender people from joining or staying in the military as …read more
Covington Catholic High School, the Kentucky school at the center of a national controversy over a viral video of an encounter between some of its …read more
Senator Rand Paul and Representative Thomas Massie, both from Kentucky, expressed support for the Covington Catholic High School students who were attacked by the media …read more