For the home builder, there is no $per square foot formula. I’m sure that commercial builder’s probably have a rough estimate, but the variability in what the private hobbyist will do makes such an estimate impractical.
As far as sets go, there actually are some decent ones in N scale. Kato has a set that has their superb Unitrack, a fine F3 diesel locomotive, a caboose, and a few cars. You’ll need to get a separate power pack, and you can be off and running with very good quality stuff.
Expect that you’ll pay $60 (if very lucky) to $100 for a quality (Kato, Atlas) diesel locomotive. Figure on getting two locomotives. I’d recommend that you avoid steam for now, or do more research, as the price of steam locos in N is higher, and the quality is variable. You’ll pay from $7-15 average for quality rolling stock, plan on half a dozen or so to start, plus a caboose. Stick with shorter cars for smaller spaces, that way you get more train. $30 for a power pack, whatever you need for track, and scenery can cost as little as $10 total. A can of expanding insulation foam, a quart of brown/tannish “oops” paint from the hardware store, a 8oz bottle of white glue, and a single pack of gray/green foam groundcover, plus some sifted sand from the yard and voila, instant rolling desert. Conversely, you can make the entire thing an urban scene and spend almost nothing on scratchbuilt buildings (a few boxes of cereal, some paints and glue, and a lot of creativity) to spending a LOT on craftsman laser kits.
All of the above prices are based on new street retail prices. Better deals can frequently be had from various sources, especially when buying used, but as a newcomer, unless you’re willing to risk being burned, I’d say stick with buying new. You can’t go wrong with NEW Kato or Atlas engines. Not counting the cost of the coffee table itself, you’re looking at a floor of about $200 to get started with a decent train, power pack, simple track plan, and very simple scenery. Obviou