Anthony, nice setup! To your question about the rental mortgages, you haven’t said what interest rate you are paying. As a start, if you are paying more than the risk free rate (Treasury bills) which you probably are, then a true apples to apples comparison would be yes, pay off the mortgage. But, if you are comfortable taking more risk, you have other options to invest in which you *hope* will yield you more over the coming years. You also didn’t say whether the rentals generate net income and if so, how much? What is the implied rate of return on the equity you have invested in them? If you pay the mortgages off, you’ll have even more equity tied up, will the extra net income make that worthwhile? Maybe you should use the money to buy more rentals instead, if purchase opportunities still exist in your town. … this is less of an answer than a framework to analyze the decision, hope it is helpful.
I first discovered the power of passive income when I was a senior in high school. I started a mobile billboard business where I would rent a small piece of land from someone who had land along a busy highway. Then I would place one of my billboard trailers on the land and rent out the ad space on the billboard. I would usually charge about $300 per month for the ad space, meanwhile I was only paying $50 per month to the landowner for the ground rent. I got to the point to where I had 9 billboard faces and was making quite a substantial income for someone in high school. I really learned how passive income could free up my life… this business is what lead me into investing in real estate.
The U.S. Internal Revenue Service categorizes income into three broad types, active income, passive income, and portfolio income.[1] It defines passive income as only coming from two sources: rental activity or "trade or business activities in which you do not materially participate."[2][3] Other financial and government institutions also recognize it as an income obtained as a result of capital growth or in relation to negative gearing. Passive income is usually taxable.
Portfolio income can come from multiple sources – interest/bond coupons, stock dividends, financial strategies including derivatives and capital growth.  Each offers some cash flow and some also offer potential capital gain with some risk from liquidity and volatile prices.  In this low-yield environment, many investors rely too heavily on cash flows and to pursue this are venturing further into riskier areas than they’d normally consider.