It was a worry for me some years back. But most of it is FUD and an attempt of self-fulfilling prophesy (possibly as a sleazy campaign for Red Hat's Wayland to reach maximum market penetration). Make no mistake, we will all have died of old age before something appears that will replace X11 fully.
Xorg might be lacking in maintenance (to be fair, the freedesktop.org is unable to maintain any decent large project anyway). However the future will likely be Xenocara. This is still a nicely maintained pseudo fork that IMO we should already be using. Xorg splitting up into tiny fragments, only to go into maintenance was dumb.
Xorg is also not a massive project, it has been simplified over the years with KMS modesetting and moving away from the user-mode vendor drivers. It really can be maintained by a few people (and is).
Do you recall first generation X11 Window Managers? These are no longer really around. From this I can project forward and guess that the first generation Wayland compositors will also go unmaintained and die in a similar timespan. So from this I hypothesize that Xorg / Xenocara will outlive Weston, Sway, etc.