If your state is anything like mine, there's loads of old platmaps and ownership data. Also, search google for old historical photgraphs. My states historical society has a website that is easy to navigate and has search functions. I've had great luck in past narrowing down where even outhouses have been built, all the way back to the mid 1800s.
Fastest, easiest way is going to be searching and viewing a late 1870s or 1880s plat map of your area. If it shows a big building on that spot of the land, it narrows it down. Some maps mark and index key spots like hotels and banks, churches etc.
What county, town are you in? I'd be happy to help try and find some maps in my spare time.