Crowd-Source Vehicle ID

Description

Have owners identify vehicle types/id to improve AP’s object identification.

Competitive/Pricing/Notes

Let people identify vehicles and other objects from the car’s video. It would need to be done in a way that doesn’t distract the driver from their primary task of driving, though this will become less of a concern as AutoPilot improves. The option to contribute could be activated by a toggle on the vehicle controls page, or a voice command. For now, contributions could be allowed only when the vehicle is parked; as AP improves, it could be allowed when at a full stop, and with TACC activated (so the driver behind doesn’t have to honk to get us moving).

When safety criteria are met, the screen would show a snapshot from the front and rear wide-angle cameras. If AP already recognizes some objects, these have bounding boxes and hierarchical labels like Vehicle, Pickup truck, Ford, F-150; if the label can be improved, or needs correction, the user points to it, and edits the label or adds a level to the hierarchy (like model year). To identify an unrecognized object, the user starts by defining a bounding box with a diagonal swipe.

The hierarchical labels could be predefined and evolve and expand over time, so that users have a multiple choice at each level. At any level where none of the existing choices apply, the user could choose ‘Other’ and suggest an appropriate label. A ‘Yes’ voice command could confirm each currently-assigned level sequentially, and a ‘No’, followed by the number of the correct choice could correct or add to the label. I assume that Tesla knows which car any contribution comes from, so users whose edits need a lot of corrections could be respectfully (temporarily?) dropped, and users whose edits prevail would be taken more seriously.

The same idea could be applied to any kind of object. For example Person, adult, walking, age 50-60; Dog, Large, Dalmatian; Traffic sign, No parking, Weekdays, 8 am – 6 pm; Hazard, speed bump, 0 – 5 mph; Tree, deciduous, elm , 40 – 60 foot tall. Each observation could be associated with the geolocation, time, date, recent and current weather conditions (from the internet), so that fixed objects are located and changes in appearance are taken into account. Dimensions of recognized and precisely identified vehicles could be obtained from a database and used to calculate distance.

A lot of us would like to be able help make our cars smarter, and this would provide an easy and satisfying way to contribute our knowledge.

Status

Unknown.