Chris Urmson's SXSW 2016 talk on Google's self-driving car project, covering 1.3 million self-driven miles and the gap between hype and reality. It shows sensor visualizations and playbacks of hard situations the cars met, and discusses failures such as the low-speed bus collision.
Why Hacker News points people here: HN returns to it repeatedly as one of the best candid talks on how a real self-driving system sees the road and where it struggles.
“If you're going to comment on automatic driving, you need to see this.”
Detailed look at sensor data, bus collision and hard driving cases
Full comment
On a related note, here's Chris Urmson's long talk at SXSW on how Google does automatic driving.[1] This is worth watching. He shows what Google cars sense and gives an overview of how they analyze the data. This is the first time Google has released this level of detail. If you're going to comment on automatic driving, you need to see this. Urmson goes into great detail on Google's 2mph collision with a bus. The sensor data from the Google car is shown. The video from the bus (it had cameras, too) is shown. Exactly what the software was trying to do is discussed. The assumptions the software made about what the bus driver would do are discussed. What they did to prevent this from happening again is mentioned. Most of the talk is about the hard cases. In the beginning, Google developed a highway driving system, but that was years ago. Now they're working hard on dealing with everything that can happen on a road, including someone in an electric wheelchair chasing ducks with a broom. [1] this video
Strongly recommendsAnimatsJul 2018
“It's one of the best talks on the subject.”
Widely viewed talk with Google cars handling challenging conditions
Full comment
That's Chris Urmson's talk at SXSW in 2016 [1], which has been widely viewed. It's one of the best talks on the subject. He said "How quickly can we get this into people's hands? If you read the papers, you see maybe it's three years, maybe it's thirty years. And I am here to tell you that honestly, it's a bit of both." Then he showed video of Google cars dealing with various challenging conditions. This included the famous scene where a Google self driving car encountered a woman in a powered wheelchair using a broom to chase a turkey. The software recognized this as something to avoid, even though it didn't classify it further. [1] this video
RecommendskyrraMay 2016
“This was a good presentation by the Google team”
Google team presentation noting early highway driving success
Full comment
This was a good presentation by the Google team: this video I believe they say in there that they got highway driving down really early on. They have been focusing on urban driving for the last few years.
RecommendsAnimatsDec 2016
“That's worth a watch. It gives a good overview of how Google does it.”
Good overview of how Google's self-driving system works
Full comment
This article seems to be a summary of Chris Urmson's talk at SXSW.[1] That's worth a watch. It gives a good overview of how Google does it. [1] this video
RecommendsAnimatsJan 2017
“Watch Urmson's video from SXSW.[1]”
Google handling construction sites and unusual road situations
Full comment
Except that Google was able to handle construction sites, traffic cones, and people directing traffic last year. Watch Urmson's video from SXSW.[1] Google has put a lot of effort into handling unusual situations. Those cars driving on residential streets in Mountain View and Austin have accumulated data on the hard cases. The California DMV continues to publish autonomous vehicle crash reports. The last one was in October.[2] A Google vehicle was rear-ended at 3mph when it was entering an intersection with poor visibility, detected cross-traffic, and stopped. I'm looking forward to the 2016 self-driving disengagement reports appearing on on the California DMV site.[3] Those were due January 1, covering the year ending 1 December 2016. Then we'll have comparative info on how all the players are doing. [1] this video [2] link [3] link
RecommendsAnimatsJun 2017
“Compare Chris Urmson's SXSW talk from 2016 about Google's system.[1]”
Sensor visualizations and candid discussion of failures at Google
Full comment
Compare Chris Urmson's SXSW talk from 2016 about Google's system.[1] Look carefully at the graphics which show what the sensors are detecting, and compare with Tesla's. Watch Urmson discuss hard cases, problems, and failures. Not seeing that with Tesla. Or Uber. Or Cruise. Or Otto. [1] this video
RecommendsAnimatsJul 2017
“Watch Chris Urmson's self-driving car talk at SXSW 2016.[1]”
Google car handling a wheelchair user chasing a duck
Full comment
Watch Chris Urmson's self-driving car talk at SXSW 2016.[1] Especially the part where the Google car encounters someone in a powered wheelchair chasing a duck with a broom. Google can handle that. [1] this video
Cites itagildehausMar 2016
“The car simply cannot figure out where it is when there's a significant layer of snow on the ground”
Urmson on snow breaking LIDAR-based localization
Full comment
They do well in rain but snow remains a problem, not due to icy roads (at SXSW Chris Urmson said they do well on slick roads) but that the "better than GPS" LIDAR-based location tracking stops working when snow is on the ground. The car simply cannot figure out where it is when there's a significant layer of snow on the ground because the reference maps stop looking like what the car is seeing. this video
Cites itEmployedRussianApr 2016
“Urmson talks about it here”
Urmson on why handing drivers controls is dangerous
Full comment
When you are barreling down a highway at 65 miles per hour and are not paying attention (and you wouldn't, because the car drives itself just fine), giving you controls is much more dangerous (for you and others around you) then not. Urmson talks about it here: this video
Cites itstudentrobJul 2016
“Here's the part where he starts talking about the 2mph bus accident [1].”
Segment on Google's 2mph bus collision
Full comment
Cool! Thank you for sharing this. Here's the part where he starts talking about the 2mph bus accident [1]. He says the self-driving car should not have moved. He does not say why. He says they ran a lot of tests to make sure it will not happen again. In plain English, I'd say, when the SDC is stopped on the side of the road, give sufficient time for traffic coming from behind to clear out before the SDC moves forward. Also, if an accident is imminent, being in a non-moving position is probably safer and less likely to cause liability for the SDC than moving. Plus, rolling over some sandbags is probably preferable to being hit by a bus. [1] this video
Cites itAnimatsJul 2016
“See Chris Urmson's talk at SXSW, which has lots of playbacks of situations Google cars have encountered.[1]”
Playbacks of situations Google cars encountered
Full comment
Google does a lot of driving in simulation, using data captured by real vehicles. They log all the raw sensor data on the real vehicles, so they can do a full playback. They do far more miles in simulation than they do in the real world. That's how they debug, and how they regression-test. For near-misses, disconnects, poor decisions, and accidents, they replay the event in the simulator using the live data, and work on the software until that won't happen again. Analyzing problems which didn't rise to the level of an accident provides far more situations to analyze. They're not limited to just accidents. See Chris Urmson's talk at SXSW, which has lots of playbacks of situations Google cars have encountered.[1] [1] this video
Cites itAnimatsNov 2016
“Only Urmson from Google talks about that much.”
Google's obstacle handling beyond standard cars and pedestrians
Full comment
Yeah, me too; I was in the 2005 Grand Challenge. Vision processing has made enormous progress since then. Here's Mobileye's guy explaining what they do and what needs to be done.[1] Here's the NVidia guy.[2] Here's Chris Urmson from Google.[3] All three use their vision system to draw boxes around things that look like obstacles. Then the planner uses those boxes to construct a path. The vision system just gives you a rectangular box in image space; the LIDAR based systems turn point clouds into simple 3D solids. These vision systems do OK on things that look like typical road obstacles - cars and people. Other stuff, not so much. Only Urmson from Google talks about that much. The videos from the vision systems don't seem to be doing much with road edge clutter. They're not marking every post, fireplug, and low-hanging branch. Train a deep learning system on cars, trucks, pedestrians, and bicycles, and your system sees only cars, trucks, pedestrians, and bicycles. These vision systems do a lot less than many people think they do. Watch the videos. [1] link [2] link [3] this video
Cites itAnimatsFeb 2017
“It's in his SXSW talk.[1]”
Google/Waymo position on autonomy stated in the talk
Full comment
It's in his SXSW talk.[1] It's the Google/Waymo and Volvo position, those being the two companies closest to a product. [1] this video
Cites itAnimatsJun 2017
“Deep learning is useful, but it's not magic, and it's not strong AI.”
Lidar-based approach versus Tesla's camera-only strategy
Full comment
What this really reflects is that Tesla has painted itself into a corner. They've shipped vehicles with a weak sensor suite that's claimed to be sufficient to support self-driving, leaving the software for later. Tesla, unlike everybody else who's serious, doesn't have a LIDAR. Now, it's "later", their software demos are about where Google was in 2010, and Tesla has a big problem. This is a really hard problem to do with cameras alone. Deep learning is useful, but it's not magic, and it's not strong AI. No wonder their head of automatic driving quit. Karpathy may bail in a few months, once he realizes he's joined a death march. If anything, Tesla should have learned by now that you don't want to need to recognize objects to avoid them. The Mobileye system works that way, being very focused on identifying moving cars, pedestrians, and bicycles. It's led to at least four high speed crashes with stationary objects it didn't identify as obstacles. This is pathetic. We had avoidance of big stationary objects working in the DARPA Grand Challenge back in 2005. With a good LIDAR, you get a point cloud. This tells you where there's something. Maybe you can identify some of the "somethings", but if there's an unidentified object out there, you know it's there. The planner can plot a course that stays on the road surface and doesn't hit anything. Object recognition is mostly for identifying other road users and trying to predict their behavior. Compare Chris Urmson's talk and videos at
Cites itRetricAug 2017
“I am sure the car had no idea what this was”
Example of an unidentifiable object the car should avoid
Full comment
Can you give an actual example? Because, not being able to identify something is not necessarily an issue as long as the car notices something is there and it should not hit it. EX: I am sure the car had no idea what this was: this video but as long as it can tell it's bigger than a bread box and so it should not to hit it that's enough.
Cites itAnimatsOct 2017
“Chris Urmson's SXSW talk from 2016 [2] covered most of this material in more detail.”
Covers most of Waymo's report material in more detail
Full comment
Link to actual report.[1] (When you're posting a link to something important, post the link to the real document, not some pundit blithering about it. Yes, you may have to do some digging to find it when the pundit outlet doesn't provide a link.) There's not much new here if you've been following what Waymo is doing. Chris Urmson's SXSW talk from 2016 [2] covered most of this material in more detail. This is more of a PR-level version of that content. The new material covers mainly the limitations Waymo is imposing on their system. It works only where they have very detailed maps of the road system, with more information than even StreetView. This is conservative, but not a bad decision for initial release. They have a button to call customer service. (That's so unlike Google.) [1] link [2] h this video
Cites itunityByFreedomApr 2018
“found that users misbehave [1] when behind the wheel of semi-autonomous vehicles”
Google found test users misbehaved in semi-autonomous cars
Full comment
> Why not the other way around? You do the driving, and the computer is the backup in case you mess up? Unfortunately having the computer takeover has the same issues. If it thinks you're following the wrong lane it could choose to steer you into a barrier, for example. Chris Urmson's team at Google ran user tests at the start of their autonomous vehicle program and found that users misbehave [1] when behind the wheel of semi-autonomous vehicles, despite having been given instruction to pay attention. We're seeing that scenario play out in slow motion in Teslas and perhaps other driver-assistance programs that get less press. [1] this video
