Big Data

Hacking the Autonomous Vehicle

Bill Schmarzo By Bill Schmarzo CTO, Dell EMC Services (aka “Dean of Big Data”) November 1, 2017

I love it when I get feedback from a blog that I’ve written. I appreciate the different perspectives and insights that others bring to a topic of interest. And no blog that I’ve written has drawn more comments than my blog, “Isaac Asimov: The 4th Law of Robotics.”

The section of the blog that fueled the most comments stem from a scene in the movie I, Robot where Detective Spooner (played by Will Smith) is explaining to Doctor Calvin (who is responsible for giving robots human-like behaviors) why he distrusts and hates robots. He is describing an incident where his police car crashed into another car and both cars were thrown into a cold and deep river – certain death for all occupants. However, a robot jumps into the water and decides to save Detective Spooner over a 10-year old girl (Sarah) who was in the other car. Here is the dialogue between Detective Spooner and Doctor Calvin about the robot’s “decision” to save Detective Spooner instead of the girl:

Doctor Calvin: “The robot’s brain is a difference engine. It’s reading vital signs and it must have calculated that…”

Spooner: “It did…I was the logical choice to save. It calculated that I had 45% chance of survival. Sarah had only an 11% chance. She was somebody’s baby. 11% is more than enough. A human being would have known that.”

One of the readers, Warren, shared an MIT site (http://moralmachine.mit.edu/) that allows one to compare their answers to others around various autonomous vehicle life-and-death situations. Some of the scenarios are fairly straightforward…unless you’re a cat lover (see Figure 1):

Figure 1: Kills Cats or Little Old Ladies?

Figure 1: Kills Cats or Little Old Ladies?

 

However, the scenarios get increasingly more complex (see Figure 2).

Figure 2: Kill 5 Passengers in the Car or 5 Pedestrians?

Figure 2: Kill 5 Passengers in the Car or 5 Pedestrians?

 

Another reader, Swen, provided an interesting perspective about the potential insurance ramifications related to the “life-and-death” analytic models pre-programmed into the autonomous vehicle:

“But, there is another, very important party involved which has not been mentioned before. It is the very powerful insurance companies. Based on a general “zero law” they will have a very decisive impact on what will be and what not. They will only insure you and the damage you make if you have driving software version “XYZ” that complies with their regulations. Else you will not get insured.”

Hacking the Autonomous Vehicle

Maybe my favorite perspective came from Patrick Henz (@Patrick_Henz), who shared with me the article “Compliance Tasks Related to Self Driving Technology.” The article poses another challenge facing the autonomous vehicle industry hacking of the “life-and-death” analytic models:

“Today chip-tuning is already used to change the management of the engine and find additional horsepower. This is in most cases legal, but liberates the car manufacturer from its guarantee. When self-driving cars are a relevant market, it is a question of time, when programmers will offer software to ensure a higher safety for their owners, programmed preference for the passenger against the pedestrians.”

In the same way that there are after-markets for computer chips that override the engine performance settings that come with the automobile out of the factory, will there evolve an after-market for technicians who can “hack” the life-and-death settings that are pre-programmed into an autonomous vehicle?

We are already seeing situations where customers are resorting to “hacking” their vehicles. Farmers are hacking their John Deere tractor’s firmware (“Farmers Are Hacking Their Tractors So They Can Actually Fix Them”) in order to perform their own maintenance repairs on their John Deere tractors. Farmers are struggling with the John Deere software license that only allows Deere dealers and “authorized” shops to perform maintenance repairs on tractors.

According to some farmers, John Deere “charges out the wazoo” for repairs. Plus “authorized” mechanics might not arrive to fix a broken tractor in a timely manner, which can affect a farmer’s operations and eventually, their finances.

Summary

Will smart mechanics hack the life-and-death decisions pre-programmed into an autonomous vehicle? Or maybe there’ll be a “Death Selector” user setting in the autonomous vehicle preferences (see Figure 3).

Figure 3: Autonomous Vehicle User Settings

Figure 3: Autonomous Vehicle User Settings

 

On September 6th, the United States House of Representatives voted to speed the introduction of self-driving cars (“House Passes Bill to Speed Deployment of Self-Driving Cars”) by giving the federal government authority to exempt automakers from safety standards not applicable to the technology.

I’m not sure how this will end, but I’m certain that this is not an issue that should be decided by technology companies. And now I have concerns about the federal government’s ability to address this issue, given how quick they were to obfuscate the automakers from any safety liabilities associated with an autonomous vehicle.

However, I also know that I don’t want “machines” making these decisions themselves. Machines don’t fear death, and I’m not certain how to program an autonomous vehicle operating system that fully appreciates the moral consequences and ramifications of death.

 

Bill Schmarzo

About Bill Schmarzo


CTO, Dell EMC Services (aka “Dean of Big Data”)

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business” and “Big Data MBA: Driving Business Strategies with Data Science”, is responsible for setting strategy and defining the Big Data service offerings for Dell EMC’s Big Data Practice. As a CTO within Dell EMC’s 2,000+ person consulting organization, he works with organizations to identify where and how to start their big data journeys. He’s written white papers, is an avid blogger and is a frequent speaker on the use of Big Data and data science to power an organization’s key business initiatives. He is a University of San Francisco School of Management (SOM) Executive Fellow where he teaches the “Big Data MBA” course. Bill also just completed a research paper on “Determining The Economic Value of Data”. Onalytica recently ranked Bill as #4 Big Data Influencer worldwide.

Bill has over three decades of experience in data warehousing, BI and analytics. Bill authored the Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements. Bill serves on the City of San Jose’s Technology Innovation Board, and on the faculties of The Data Warehouse Institute and Strata.

Previously, Bill was vice president of Analytics at Yahoo where he was responsible for the development of Yahoo’s Advertiser and Website analytics products, including the delivery of “actionable insights” through a holistic user experience. Before that, Bill oversaw the Analytic Applications business unit at Business Objects, including the development, marketing and sales of their industry-defining analytic applications.

Bill holds a Masters Business Administration from University of Iowa and a Bachelor of Science degree in Mathematics, Computer Science and Business Administration from Coe College.

Read More

Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *

2 thoughts on “Hacking the Autonomous Vehicle

  1. Thanks for the great article. Reading it I was wondering what commands I would give if I was in a dire situation. If I was having a heart attack and I was in a panic wanting to get to the hospital as soon as possible or if I had a loved one (kid, wife, my father) that required immediate attention. Would I command the car to ignore traffic lights and maybe cause a number of accidents? Would the automaker offer such an emergency “mode” to get more shales? Would I want to install it “just in case”? In my panic state would I distinguish the ethical thing to do (not use it) or would I say to the car get me there no matter what. I think the self-driving car will put our values (as a society) to the test….

    • Vaggelis, great great question! Ideally, you’d hope that in an emergency situation, that your autonomous vehicle would communicate to the other autonomous vehicles the situation, and that they other AV’s would “clear a path” for you to the nearest emergency location. And that brings up a very key point – and maybe a requirement – for autonomous vehicles: how do they communicate to each other their intentions? If my AV is driving in the left-most lane on the highway and needs to get off at the next exit, how does it communicate its intentions so that the other AV’s can make room for my AV to start crossing lanes?

      I don’t think I’ve read anything about how these AV’s are suppose to communicate with each other. It would have to be peer-to-peer because one can not rely on transmitting data to and back from the cloud in these near real-time decisions. Maybe a role for Blockchain?

      Gain, great question!