Google's self-driving cars are smart, but can they beat Murphy's Law?

Paint this scene in your mind. A quiet street in Mountain View, California. Bungalows, hybrid cars, sunshine. Now, add a wild turkey sprinting onto the road. Then a woman chasing it. In a wheelchair. Wielding a broom. And finally: because this town is the home of Google, a self-driving car pulls onto the street and, seeing the chaos unfolding in front of it, slows down to capture the entire thing with a camera and some laser and radar sensors.

This is not a George Saunders short story. It’s a scenario that one of Google’s self-driving cars actually encountered while logging miles on the streets of Silicon Valley over the last few years.

I’m watching video of the event in a new complex a few miles from Google’s main campus. This is the home of Google X, the company’s far-out research laboratory. It looks and feels like a building given the Ace Hotel makeover: reclaimed wood, neutral tones, nice typography, a place you can get a massage. The holding area for the journalists that Google gathered looked out onto a covered, open-air gym where buff young dudes were engaged in complex jumping exercises. This driverless car show-and-tell is a Google tradition for the past couple of years. They invite a select group of tech journalists to come down to Mountain View and get an in-depth look at the program’s latest milestones. And, of course, you get to ride in one of the cars. Afterwards, they give you a sticker that says you’ve been in a self-driving vehicle. I swear if you scratch it, it gives off a whiff of inevitability.

So why the wild turkey chase? These fringe scenarios were the theme of Google’s presentation in subtle and not-so-subtle ways. Google wanted to show that its self-driving cars are learning to be resilient to all kinds of one-in-a-million, can-you-believe-that events. If Google’s cars can detect a turkey and a swerving wheelchair in the road, recognize them as anomalies, and slow down to avoid them, maybe these cars are almost ready to transform the world’s transportation systems.

The truth is that right now, Google’s self-driving cars can deal with the vast majority of common driving situations. The basics of path-finding, obstacle avoidance, and smooth driving are pretty much in the bag. Having ridden in their vehicles several times, I can attest that the cars can drive. Like, you get in, and you get from point A to point B. At least in the mostly nice driving conditions of Mountain View.

The cars are safe, too. Google’s autonomous vehicles have driven over a million miles and only gotten into about a dozen accidents. A car was side-swiped once, but in every other accident, the cars have been rear-ended by humans, mostly while sitting at stop lights. Sure, their human handlers have had to take over a few times, which has kept the cars from getting in more accidents. The safety record is still pretty remarkable given how many miles they’ve logged.

Any time the topic of self-driving cars comes up,  though, almost everyone imagines some crazy situation that a computer surely couldn’t handle. What if the car encounters weird road construction? A guy swerving on a unicycle? An escaped zoo animal? A swarm of bees spilling out of a flaming Camaro crashing into a 18-wheeler carrying an aquarium filled with sharks? How would Google deal with that, eh?!

Google knows that until its cars can handle these freak occurrences, doubts about self-driving cars will persist. So the company has spent the past year working on how to deal with any number of unfortunate events. Basically, Google is up against Murphy’s Law—and they need to create software that can deal with the anything that could go wrong going wrong.

Even five years ago, the idea of a self-driving car that could safely handle 99% of the driving process would have seemed ludicrous. But Google brought together many of the world’s best autonomous vehicle researchers, mashing up the best teams from DARPA-sponsored self-driving vehicle competitions to create a superteam with supercharged resources courtesy of true believers within Google like co-founder Sergey Brin. “Rather than setting researchers up to compete for grants, space, funding, and all the other quotidian trials of university research,” wrote computer scientist Ian Bogost in The Atlantic, “Google just hired many of the best from Stanford, Carnegie Mellon, and elsewhere, and gave them access to the company’s massive array of computational power and collected data.”

The key insight—or computational shortcut—in Google’s program is that they could give the cars a massive headstart by creating incredibly detailed maps of the road and the logic of that road. The maps that Google’s cars work from tell the cars precisely where to expect curbs and lights and different kinds of lanes. They know the nominal rules of the road more precisely than any human. That lets the cars focus on the real-time aspects of driving.

In any Google self-driving car video, like the one we saw of the turkey chase, they show both a simple camera recording of the road and what we might say the Google car “sees” and processes from its array of radar, laser, and optical sensors.

The car takes in all the sensor data and synthesizes it into a simplified model of the world. Each object in or near the roadway is turned into a blocky wireframe and overlaid onto the detailed maps. Then, the car makes predictions about what each object might do. A parked car is likely to stay parked. A bicycle will move faster than a pedestrian but slower than a car. The car plots a future for every single object on the road as it chooses its path.

All of the sensing and number-crunching comes down to computing just a couple numbers: what angle to turn the steering wheel and how much to push the gas (or the brakes). Because, as a mechanical phenomenon, that’s what driving is. This is what your brain does, too, actually, just more efficiently and with fewer sensors.

Google’s cars are getting pretty good at navigating perfectly through tough driving events. The centerpiece of Google’s presentation was a series of videos showing their company’s cars successfully avoiding tiny cataclysms: a cyclist cutting across an intersection and going the wrong way down the street at night, a child sprinting across the road right as the car made a right onto the street, a kid in a Power Wheels truck, guy hopping out of a huge vehicle’s cab into the road, a human driver cutting off a Google car by making a right from the left lane.

The American road! What a crazy place. And Google was there to declare: we have seen it all. Almost.

“No matter how much experience you gain, there are always going to be things out in the world that happen to you that you have never encountered before,” said Dolgov.

And in these scenarios, Google has devised what Dolgov called an anomaly detection system. When the car’s sensors are telling it that something is going on beyond the normal predictions that the car would make, it deals with these edge cases (as they might be called) by taking it slow, much as a human driver probably would.

The real question is whether pushing these cars to become safer and more reliable—going from 99% safety to 99.9% or 99.99%—will get easier or harder. On the easier side of the ledger, one could say that computation is always getting cheaper, the cars are gaining more experience, and their sensors are now custom-made by Google for this purpose. On the harder side: the value of each mile driven is probably decreasing now, because the cars have learned most of what they’re going learn from each mile they drive.

One way Google is trying to accelerate the learning process is to create a team dedicated to messing with the cars on test tracks. Dolgov compared this team to the kind of security researchers who try to penetrate a company’s network, sometimes known as red teaming. On the self-driving car team, they call it structured testing.

The program is run by Jaime Waydo, whose last job was building autonomous rovers for planetary exploration at NASA. “We come up with all the diabolical cases that we can,” Waydo explained. “An example would be we put a port-a-potty next to the edge of the road and we had a person step out next to the car.”

Google does this testing in the old Castle Air Base near Merced, California, two hours southeast of San Francisco, in the heart of the San Joaquin Valley. In late 2013, the company leased 60 acres and built an urban test grid, complete with realistic road infrastructure. While the area is fenced off, it is in view of an RV park, where residents peer over the fence to watch the testing as entertainment.

“They had people actually walk out in front of the cars like they were crossing the street and the cars would stop automatically,” one local told a TV reporter. Because, again, life is a George Saunders story.

Watching all this footage, thinking about the 10 to 15 thousand miles per week that these autonomous vehicles spend on our roads, it’s hard not to wonder how close to reality the self-driving car future is. Chris Urmson, who leads Google’s program, said they’d have the cars on the road within four years, or by the time his 12-year-old is 16.

But UC Berkeley research engineer, Steve Shladover, who was not present at the briefing but has been working on automating transportation for decades, cautioned against reading too much into what Google presents. “They never share details about what they are doing, but only provide high level summary information, carefully packaged,” he said.

Other experts are more optimistic about Google’s progress. MIT professor John Leonard works on autonomous vehicles in the university’s Computer Science and Artificial Intelligence Laboratory (CSAIL). He was given a separate but similar presentation to the one I saw earlier this week.

“I was particularly impressed by videos that Dimitri showed of a difficult left-turn across traffic (at a junction with no traffic lights), a video showing a cyclist running a red-light (the Google car patiently waited whereas a human driver did not, creating a near-miss for the cyclist) and a video showing a child running out in the road in front of their car (their car slowed down appropriately),” Leonard told me in an email.

And certainly, Brin and other Google executives seem committed to seeing driverless vehicles through to some kind of commercialization—most likely in the form of transportation as service. Brin made a surprise visit to the media briefing, arriving in a long-sleeve t-shirt, basketball shorts, and Crocs. He took the microphone and lauded the team’s progress.

“When we embarked on this journey, we all felt it would be a long path… but over the past year or so, I’ve been really happy about the progress we’ve made. And I think the potential to change the way communities work and the way we can give access to people who are underserved by potential today, that day is coming closer and I’m super excited by it.”

In other words, all Google has to do in order to transform transportation forever is to solve the lady-in-a-wheelchair-chasing-a-turkey-with-a-broomstick problem. And it appears they’re getting closer.