Smartphones drive cars: DIY Assisted Driving by comma.ai and George Hotz

I recently learned about a fellow named George Hotz aka GeoHot a notable programmer known for his early ios jailbreaking, hacking, and general rabble rousing. His company comma.ai is now working to access existing car hardware to bring Tesla level driving assistance to as many cars as possible; on a smartphone combined with OSM II car interface. This is open hardware and software so it can be built with consumer hardware assembled by whoever can follow the plans. They have made the necessary hardware available on their website for less than 1k USD.

Essentially using a cooled oneplus smartphone sold at their webstore as a dashcam called EON along with bluetooth and wifi enabled car hardware interface modules Panda and Giraffe. These open hardware devices listen and talk to newer Honda and Toyota vehicles currently. Additional car sensory and actuation system protocols need exploring by the community so all the various data types can interface with the neural net models. At the moment each manufacture model has its own fingerprint which allows the variations in information from car to car to be dealt with.

Right now Comma AI sells the EON dashcam and car monitoring hardware that do not provide out of the box functionality in order to avoid regulatory problems. In order for the driving assistance system to function one must install the openpilot software and connect to the Panda and Giraffe hardware via bluetooth of wifi. For any unsupported vehicle we would need to reverse engineer the sensor outputs to feed to network as well as gain control of the Gas, Steering, and Brakes. If a car has steering assistance hardware as most cars after 2012 do assuming we(the community) are capable of designing hardware and software capable of interfacing any particular car. We would then have to create and implement a fork with Comma AI's openpilot software. If we do a good job they will add it to the main repository.

This is totally possible as there is a market of persons such as myself who would like to teach androids to drive our cars. We will surpass the safety of normal driver's safety with statistical significance in no time, we may need to create our own insurance company if none of the existing ones will reduce rates. Though crowdsourcing the porting of new vehicles to OpenPilot could be time consuming.

I have not heard much positive feedback from lay people I asked about potentially adding this capability to their vehicles. Only from people who like to tinker and explore. People seem to be extremely skeptical, which makes sense since they likely have no understanding of how it works. True for most everything really. Since diving in and learning about what is being done, I am confident I will be able to implement this to vehicles around me.

What I'm saying is I intend to purchase and install these for myself so that I will be able to create a platform of services to complement the capabilities that will be provided. Truck drivers are likely to be interested since they spend the most time on the road already. Though this market will be difficult to serve

Hotz says they are not coding in specific rules of the road but writing 'ground truths' solving for how to drive most like the people behind the wheel. George wants to own a slice of the self driving stack, leaving many more for other entrepreneurs. His is machine, crowd, platform. I would like to serve the lyft and uber market, ridesharing, we will offer drivers the system to increase their safety and get assistance, move toward statistically safer driving, which allows insurance to be reduced.

All this exploration of technology leaves me salivating at how to make openpilot software and hardware work for as many vehicles as possible. Since it is open source we have a great framework to explore the code and community to help, though I haven't compiled anything in a while. (I’m trying to fix that) The interface devices named panda and giraffe connect to an app called chffr available on ios and android, Hotz specifically references how if Tesla is the iOS/Apple of driverless cars then openpilot is the android of driverless software.

While thinking about adding hardware and software to vehicles on the road I have to mention a particular administration in the US. Hotz first attempt to bring the self driving capability to market in 2015 as the Comma One resulted in a letter from the National Highway Traffic Safety Administration. First is the Comma AI statement, and below that are the questions asked by the NHTSA.

Comma AI Statement

The comma one will not turn your car into an autonomous vehicle. It is an advanced driver assistance system. To put it in traditional auto manufacturer terms, it is “lane keep assist” and “adaptive cruise control”

For all these Tesla autopilot like systems, it is very important that you pay attention. This system does not remove any of the driver’s responsibilities from the task of driving. We provide two safety guarantees:

1. Enforced disengagements. Step on either pedal or press the cancel button to retake full manual control of the car immediately.

2. Actuation limits. While the system is engaged, the actuators are constrained to operate within reasonable limits; the same limits used by the stock system on the Honda.

The NHTSA's questions are:

How the Comma One is installed in a vehicle and to provide the installation instructions.
Describe the advanced driver assistance features of the Comma One, including how they’re different from the existing features in the cars in which the Comma One will be installed.
How a driver would use the Comma One and to provide a user manual.
Describe the conditions under which Comma.ai believes that a car equipped with the Comma One would be able to safely operate. Examples include types of roads, geographic area, speed, traffic conditions, environmental conditions and amount and type of necessary driver input.
Provide a detailed description of any testing or analysis that supports the answers found in Question Four.
Describe the steps that Comma.ai has taken or will take to ensure the safe operation of a car with the Comma One, including how the automated shutoff works and how owners will be educated.
Provide a list of the cars that Comma.ai supports or will support with the Comma One. Include make, model and year.
Detail the steps that Comma.ai has taken to make sure that installing the Comma One didn’t have unintended consequences on the car’s operation.
What would happen if the Comma One was installed in an unsupported car?
Has Comma.ai done any analysis or testing of the effects that the Comma One would have with the car’s compliance with the Federal Motor Vehicle Safety Standards? If so, provide the analysis. If not, describe why not.
How does the Comma One impact a car’s rearview mirror?
What is Comma.ai’s position on how the Comma One does or does not affect a car’s FMVSS rearview mirror requirements?
State the date that Comma.ai plans to start selling the Comma One. Provide a list of all retailers and websites that Comma.ai expects to sell the Comma One.
State the date that the Comma One will start shipping.
Provide any other information that Comma.ai thinks supports the safety of the Comma One.

Hotz and Comma AI decided to cancel the launch of the Comma One which promised Self Driving. In order to continue their work and prevent liability lawsuits they released the EON which is a dashcam contributing to their learning model for assisted driving. They also have an app chffr which accomplishes the same thing as the EON. We also need hardware called panda and giraffe to interface the cars computers and with the right protocol recognition software we can collect the available sensory data to interface and teach comma AI’s OpenPilot model.

I will attempt to answer as many questions posed by the Regulation Agency as I have found answers to:

Install the Panda in supported cars OSM II data port, connect to it via the smartphone APP or a computer, download and analyse your car profile with cabana cam interface software.
Current systems provide at minimum, a cruise control speed setting system. Auto Pilot and Super Cruise provide hands free highway driving. Newer model vehicles utilizing radar, actuation, and vision sensors can maintain a safe distance and make minor lane keeping corrections. (The latter are the target market) Primary feature added is driving assistance on the highway or in stop and go traffic.
User Manual and community support are found online. To Operate an installed system: Turn system on then drive like normal, driver is still responsible for their vehicle. (comma.ai no longer offers the Comma One self driving system due to this letter of questions from the Vehicle regulatory body)
Dash Cam Support is as simple as downloading the app and mounting inside windshield, car system monitoring and actuation are achieved via the Panda and Giraffe hardware. The system has been demonstrated to operate on highways and in traffic reference youtube videos. When openpilot is engaged by the driver it attempts to mimic the human driver and its goal is to reduce disengagements(a self driving effectiveness measurement unit) to as few as possible, better than humans can do.
In the open source community information is collected near the community members which support the development. Youtube videos and github documentation can validate the capability as well as trying the system out for yourself.
Safe operation of the vehicle is the responsibility of the driver, comma.ai while having written the openpilot software is not responsible for its download and execution. Since the additional comma hardware does not affect the existing hardware existing safety measures will maintain safety regulations. Along with the smooth driving principle that rapid actuation changes or high torque force are not a possibility. Refer to car manufacturer specifications.
Check the Open Pilot Supported Cars on GitHub
A Connected Panda to the OBD II port has no effect on car hardware, only reads inputs. Giraffe can talk to the car and tell the computers to actuate the brake, steering, and gas pedal. Though it cannot exceed car hardware safety implementations. Smooth driving is safe driving.
I imagine the software would recognize code failure and it would be reduced to a dash cam again.
They have not shared their testing information as to alterations from normal runtime. It is also difficult to determine what unintended consequences would result since we are unable to provide a baseline without installing our hardware or directly working with honda hardware and software engineers. I do not know what the safety standards are and while i'm sure they have good intentions but no one follows all the rules of the road and if they did they would greatly increase their commute travel times.
It should be mounted in front of the rear view as to prevent obfuscation however any location facing out the windshield should work so long as sensors can still be connected. No longer replaces rear view mirror.
The original design appeared to obstruct, replace, or commingle with the mirror. Since then the EON can be placed as to avoid violating the regulations for rear view mirrors.
They currently produce the EON, Panda, and Giraffe hardware, I bought a panda today
Comma One was canceled due to the regulatory issues it would have faced
Though the Comma One was cancelled the replacement system can be implemented to the same degree of capability and better since it is mostly a software problem at this point. Check the videos of OpenPilot users and their feedback about the inner workings of the system.

Here is what George Holtz at Comma AI has and is capable of selling: Manufacturer level access to data from OSM II ports via your phone or Computer(PANDA). Additional access to advanced sensor data and actuation systems can be achieved with the Giraffe hardware kit. The ability to read car data is rare and expensive at the moment, it is still a relatively small community of tinkerers and those enthusiasts looking for driver assistance as well as anyone who wants to record and monitor their vehicle data.

I found the video below to be helpful in understanding what GeoHot is doing and how he's going to get there. I intend to convert as many vehicles as I possibly can. I was thinking the points logged could be spent like tokens at some point incentivizing new drivers. Offering reference reward points for converted drivers, mapping new areas, providing safe edge case scenario data or porting more cars to the platform. Basically increase the bounty's available to those most capable of gathering the necessary data and interpreting it.

A Great Final Statement: "Comma AI's real purpose is to end all jobs" George Hotz. I like this because I never want to work again, though I have many interests.

I don't go to work all day, I go play all day

I noticed he has a blank keyboard

The whole thought about this is that there will now be commuters who will be interested in reducing their risk during their commute by getting assistance from the crowd and AI.

SteemOn!