The system relies on a simple model of a road, based upon how people perceive what a road is. Simply put, a road is a smooth path largely consistently colored with a contrasting color from its surroundings or delineations. This means the important aspects of a road are its flatness and its color in that order.
Here is an example road:
Flatness means there is no change in slope across a surface. One way a discretized surface can be modeled is by using unit vectors perpendicular to the surface known unit normal vectors. Where the surface is flat, difference between these vectors is zero
Using a 3D sensor, the depth is gathered then converted to a point cloud. The point cloud is then converted to a map of unit surface normal vectors. By taking the difference of the unit normal vectors to their neighbors, The rate of change of the surface is determined. If this rate exceeds the rate at which the vehicle at speed could travel over, it is excluded as eligible road surface.
This is the largest area directly in front of the vehicle that is sufficiently flat:
Roadway surface is typically well defined and fairly uniform in color, however large amounts of variance exists due to inconsistent wear, repairs, shade, and dryness. This variance requires a solution that is adaptive to the local conditions of the surface.
Adaptation is accomplished by sampling the surface directly in front of the vehicle and the sampled region's variance is determined. As the model is based upon human perception and the variance in the road surface color is primarily in lightness, conversion to a color space more consistent with human perception is required.
This is the sampled area:
This is the areas of the image that closely match the color of the sampled road within allowed variance:
Since a road must be flat enough to travel on and must contrast with the surrounding areas if it is unmarked, the two filters can be combined in a logical and.
This is the largest flat area that closely matches the color of the sampled road:
After some filtering, this area remains:
Extracting the outside of the blob, the left and right edges of the road are determined shown in blue and red, respectively:
Note the large outcrop on the right side line that extends beyond the edge of the road. This is due to the area being of similar color and flatness to the rest of the road. Also note the areas of outcrop on the left side line that extend into to the snowbank. This is due to the coarseness of the filtering pattern. These artifacts demonstrate the need for improved filtering in the prototypes.
The "no obstacles to vision" constraint is for simplification in the early systems that is expected to be removed down the road.
There are few, if any, unmarked roads in America that have a speed limit over 40 mph (at least in my area). If this project is expanded to consider other countries where high speed unmarked roads are present (like the UK), this constraint will have to be adjusted. 60 mph would likely be a reasonable upper bound for the system.
The perception time of an attentive human driver sits around 500 ms and includes making a decision based on the stimulus. As this system was only concerned with the perception of the lines, there needed to be some time to compute potential commanded actions.