I don't have experience with this, but you might consider looking at OpenStreetMap again. They allow you to export any section of a map as SVG: http://wiki.openstreetmap.org/wiki/SVG. If that map doesn't have all the detail and buildings you need, consider adding it yourself! Either by memory, estimation or by looking at arial photographs of the same location. At some point, you'll need to add custom detail to any map, but this should do much of the work for you. Good luck!