Results & Conclusions

While we made several attempts at a more complex algorithm, the brute-force processing and correlation procedure remained our only measurably accurate one. We focused on stress-testing this algorithm and obtained some interesting results.

Accuracy: Useful but not Near-Perfect

Above are the results of running 1000 randomly-selected satellite images through the algorithm in satDemo.m. Accuracy was calculated separately for each 100-trial group, then averaged over all groups. The mean accuracy is 72.9%. This is not nearly enough to run a standalone matching program. However, since there are so many possible combinations of matches (about 1.2 million for our tests) the algorithm represents a large step in the right direction.

Failure Cases

Both the correlation matrix and the processed satellite images give insight as to where our algorithm failed. We classify its failures into two main error types: near-match error and algorithmic error.

Near-Match Error

In this case we consider satellite images that are placed near the desired location or in areas very similar to it. From the correlation matrix, we conclude that this is due to the abundance of similar-magnitude correlation values as a result of convolution across the map. The sum of similar features is so close at these points that small systematic error, not key features, determines the placement. An example is below, with corresponding correlation matrix. Note the abundance of similar values around a certain point in the map.

Algorithmic Error

This case is where the satellite processing steps do not create an image similar to the color-filtered map. Here the error occurs before we attempt correlation. This was an issue in trying to match city-center images, where there is an abundance of shadows. The difference between the desired road extraction and what actually happens in this error case is demonstrated below. The cause of error is fairly clear upon inspecting the processed satellite image (left).

Plans to Improve Accuracy

We believe that most of our failures result in over processing our images. In order to use simple methods like median filtering and correlation, we were in-discriminatory when tossing out information. If we are to generate a more reliably processed satellite image, we must consider less general ways to discover roads (for example, HoG filtering instead of Blob Removal). Our progress in these areas is detailed in the Methods section.