Merge Detection in TACC



While in TACC, use successive video frames to detect vehicles shifting into the lane in front and slow down to allow for the merge.


One of the things that seems not to work yet with TACC is detecting cars that get in the way, either cutting in to our driving lane, or stopped in front of us, or to the side but partially intruding on our lane. The radar does well detecting moving vehicles in front, but performs poorly with stationary obstacles, and for some reason doesn’t slow down for cars that cut in. I’d like to suggest a simple solution:

Here’s an article from 2006 about detecting obstacles and estimating distance from monocular video: link.springer(dot)com/chapter/10.1007/11861898_48

The description of the calculations is a bit obscure, but they’re really very simple. In successive frames of video, an object grows as we come up on it. The growth (G = pixelsNow/ pixelsBefore) can tell us the distance to the object, if we know the distance travelled between frames. Distance travelled between frames (d) is easily calculated from velocity, acceleration, and time. Then the distance to the object is D = d * (1 – G) / G; this holds for any size object. We can skip frames if necessary to get a more accurate growth number. With 4K video the accuracy is easily within a few percent of ground truth up to about 75 meters (250’). Once we know the distance to the object, and its offset from the center of the image, we can determine if it intrudes on our path. From the lens’ angle of view we can calculate the dimensions of the object.

This calculation could be done on any stationary object in the video (e.g. potholes), completing the world view of moving objects provided by the radar. If we know an object’s dimensions (the side window height of a vehicle, for example, is fairly standard), we can estimate the distance and relative velocity of moving objects that are outside the field of view of the radar.

Cross traffic: Use a vertical dimension so we can get a distance reading as soon as it comes into view. The expected pixel distance from center for a stationary object can be calculated as we approach, so deviation from that can tells us if that car, cyclist or pedestrian is moving perpendicular to us. Check again and we can determine the perpendicular ac/deceleration, and from that we can tell if a collision is likely. These simple calculations could save lives.

Moderator: Objective is excellent, but the complexity is far from simple to produce a safe, error free design. Some of this technology described is likely already in use today in EAP.


Some form of this is expected in FSD and possibly EAP, but no timeline is known. Unknown if it will be added to TACC.

(lightly edited by moderator)

Category: Tags: entered 3-Oct-2017