It will always be a small segment of the gaming market.
Once they realize the fact that there is nothing else to do to sway people from stores where you pay to own the game (eg. Steam), you will see a decline in new games added and a rise in subscription prices.
Since their current business hedge is to eat the loses today in order to grow larger tomorrow.
Which is not sustainable without user growth.
The idea Microsoft will stop putting all new first-party games on Xbox Game Pass is the new "Microsoft is going to leave console gaming and discontinue Xbox" we use to hear on repeat. I would think people would look at the kind of money Microsoft is spending because of Game Pass and put two and two together.
It's the Office 365 model. Microsoft didn't spend $8.1 billion for Zenimax/Bethesda (the final price according to their official documents), and nor are they planning to spend $68.7 billion for Activision Blizzard - their largest-ever acquisition in history - citing Game Pass as the primary driving force behind both deals, to then suddenly turn around and not put all games in Game Pass as planned.
"Play it Day One with Game Pass" is what they advertise on every game for a reason. Starfield and Elder Scrolls 6 alone going to game pass day one should destroy any further doubt about all other games. Those two games will be two of the absolute biggest titles Xbox is going to have even for the next 20 years in terms of the sheer expected magnitude and anticipation of their releases. Few games that are not a Call of Duty release will rise to their level, potentially not even a new Halo, and I fucking love Halo. Will still be big of course, just not as big as Bethesda's RPGs.
Bethesda's single-player RPGs are right now, based on reputation and past financial success, the biggest heavy hitters Microsoft have in their entire arsenal (leaving out Call of Duty) until something else that's highly anticipated, such as Avowed, Redfall, Fable, Outer Worlds 2, Hellblade, Perfect Dark and all the rest of those other big AAA games coming, can prove whether they can be as big.