Many sites have their street address location in an unstructured html page which has a wide range of uses. From making searches for your local computer store easier to plotting hiking trails. The currently problem is getting this info into a structured format which can be geocoded.
MetaCarta has an api to retrieve some of the information, though not street level accuracy.
http://labs.metacarta.com/
True local provides data services
http://www.truelocal.ca
And digital reasoning have a GeoLocator product for retrieving data from unstructured documents.
http://www.digitalreasoning.com/index.php?option=com_content&task=view&id=3
These are fine for large commercial systems, though they are not of much for every day consumer use in tasks like adding a contact to your address book. This task would be seam less if there was an open source project developing this software.
The project could be based on gpl or some other open source license making it free for commercial use. After this key piece of technology is available to every one things like address books, map creating and searching can become seamless.