I did not argue that ssh does not have bugs, but that the bugs are applicable to both the regular user and the root user. Thus, I am not understanding how the bugs can distinguish between the users.
OK, I need to learn to write more clearly. If nobody understands my point, that probably means that I didn't explain it correctly.
To begin with: Key based authentication should in theory be safe enough; with the key lengths we're using today (I think the default is 2048 bits for RSA), the possibility that it can be brute-force cracked is nonexistent.
But bugs exist. Those bugs don't typically distinguish between regular users, and root. This is where the risk/reward/cost thinking comes in.
First, is ssh access to regular user accounts necessary? Yes, for many people it is. For example, I store many important files on my server at home, and sometimes it is super convenient to be able to check one of those files while I'm at work or on the road. The alternative is about 2-3 hours in a car to drive home and back (if I were to completely close off outside network access to my home). I understand that there is some risk, random hackers could exploit a bug in ssh to get into my user account, and find out many secrets about me. That would be bad, and could lead to some embarrassment (they might for example find my medical records or tax returns), and to a lot of hassle (I would have to clean up after the hackers, make sure they didn't tamper with files, improve security). To reduce that risk somewhat (but not completely!), I hide my server and its ssh port (see above, IP address is not constant and not advertised in DNS, and ssh port is not where you expect it). This costs me a little convenience (I have to type "ssh -p 12345 ralphbsz@123.45.67.89" instead of "ssh house.ralphbsz.org"), but the cost is small, the improvement in risk is big.
Now, does the same logic apply to remote access to root? No. It is quite rare that I have to log in to root right away. And if I do want to become root, I can always log in to my normal user account, and then use su/sudo/doas. If I'm going to be doing stuff as root, then I'll have to spend some time being careful anyway, and typing one of those commands and a password is not a big inconvenience. On the other hand, random hackers are less likely to try to crack a system by first attacking user accounts and then trampolining to root from there; and if they did try that, the second stage of password checking would slow the down so much and leave so much audit trail, I might catch them. So for remote root access, the reward is not big, the risk reduction is very high, and the extra cost is minimal. Which is why I have turned all remote access to root off (key and password).
Could it be that ssh has a bug and turning off access to root doesn't actually work? Possible, but very unlikely. Even if that bug did exist, I would still have to have a hacker trying to crack my system, and the other defenses (audit, auto blacklist, unpublished IP address, unusual ssh port, ...) would hopefully be strong enough.
Now, the original question was: remote access is actually very necessary (for convenience) if one has to use rsync. I understand that, and I used to have the same problem: For certain copying operations, it would be more convenient to come in directly as root. But in my opinion, opening up key-based access to root for just this use case is not a good tradeoff, since this situation can be handled by other techniques, which just rely on file access control on the machine. Depending on your situation, your tradeoff might be different: how valuable is the data on your machine, how secure is your network access, how paranoid do you want to be?
Does this help explain the reasoning?
Adding a side remark: I just had an idea. It might be a nice idea to say that root login with key authentication is ONLY possible from a local network. This might be a nice compromise: say you have a local network 192.168.0/24, and you think you have firewalled it pretty well from the dangerous public internet. You think you can mostly trust people in the internal network. In that case, you might be able to have your cake and eat it to: allow ssh to root (just for the use case of rsync) only from the internal network, not from the outside. According to some chatter on stack exchange, one can do that by putting the "PermitRootLogin" line in sshd.conf into a match block, or by prefixing the key in ~root/.ssh/authorized_keys with "from 192.168.0/24". I haven't tried it yet, but just added this to my to-do list to test.