No its not a massive concern, most of the clubs with higher capacities are making it through being bigger, more successful clubs so able to charge more and get better sponsorship deals, are in London, or as in the the old White Hart Lane made a fortune off all the corporate hospitality. Spurs had a smaller ground capacity but thanks to London prices, decent sponsorship deals and lots of corporate boxes were comfortably turning over more money than us. Basic bums on seats don't make the big difference to clubs ability to make money and the club will have worked all this out before deciding on the capacity.
When Spurs finally decided on rebuilding WHL 4/5 years ago we were their equals pretty much on and off the pitch there’s now a massive gap between us and them. Unless we look to close that gap ourselves with forward ambitions planning we’ll never succeed, in fact it’s only likely to increase. None of the clubs above us are likely to regress we need to improve ourselves and having a stadium that matches the top teams is part of that.
Ambition shouldnt be something we fear we shouldn’t simply accept our position below these clubs because we won’t just stagnant we’ll fall back and be over taken eventually.