Problems with loader when using a serial console

I have an account on VPS and I don't have a KVM access to it, but serial console. While the console works (a bit slow, only 9600 baud), it works fine, except when loader is called (the one showing freebsd logo and that allows for OK prompt).

When I have console opened it periodically sends "n" character, it's makes it extremely hard to issue any commands because the frequency of this happening is relatively high.

It kind of looks like this:

Here it actually happened at earlier stage so pressing enter 2x solves the problem, but if there is a BSD logo, it often breaks the booting process and I had to quickly type <enter>boot<enter>, and doing anything else in that prompt is almost impossible.

Any idea what that could be? The serial console is expecting 9600,8N1 I'm not sure where to configure parity and stop bits. Or perhaps the cause is completely different?