Sounds on simple terms like you have a good market but the population is way to small to support a B&M unless you have massive tourist traffic. That being said, if you have a good market of smokers in your area you may survive if you offer almost everything the B&M's 40 miles away do, at a competitive price point.
Believe it or not, I and many others will happily drive the 40Miles to walk into a well stocked humidor if they are friendly & offer me service. You are also competing with the online market.
Another way of supplementing your business is to offer high grade accompaniments like Coffee beans & Chocolate. Like I said, best of luck but these are purely my own thoughts as I have been weighing up a similar idea with a much larger population base.