Problems with the new Vodafone Router

blazingice

blazingice

I have recently changed broadband provider to Vodafone and received a new router (Pro II Ultrahub). Since then, I have been having some wired problems accessing my FreeBSD home server, installed on an old Dell laptop. Sometimes ssh to home server is timed out and I get the following output.

Code: 
ssh -vvv USERNAME@192.168.0.92
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/USERNAME/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug2: resolve_canonicalize: hostname 192.168.0.92 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/USERNAME/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/USERNAME/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: ssh_connect_direct: entering
debug1: Connecting to 192.168.0.92 [192.168.0.92] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: connect to address 192.168.0.92 port 22: Operation timed out
ssh: connect to host 192.168.0.92 port 22: Operation timed out

Sometimes it works without a problem. Whenever the ssh is timeout, I can't access any of the server services (e.g. nfs, adguard, etc). Sometimes the router keeps rebooting. If I disconnect FreeBSD server rebooting stops.

Having looked at the auth.logs in FreeBSD I have noticed this repetative strange log. 192.168.0.1 is the address of the router, which seems to be wanting to ssh FreeBSD server!
Code: 
Jul  4 00:18:07 server sshd-session[44314]: Received disconnect from 192.168.0.1 port 59174:11: SSH client disconnected [preauth]
Jul  4 00:18:07 server sshd-session[44314]: Disconnected from authenticating user root 192.168.0.1 port 59174 [preauth]

I'm not sure how to interpret these logs, and haven't been able to find the source of the problem. The only solution at the moment is not to have the FreeBSD laptop connected to the router, which will be very disappointing. Whilst the issue may not be FreeBSD specific, can someone point to what else I could be checking. I have also looked at the logs of the router, but apart from restarting logs, I can't see anything useful.
 
blazingice said:
Code: 
ssh -vvv USERNAME@192.168.0.92
Click to expand...
This is an internal connection, I'm assuming your LAN is 192.168.0.0/24? So this would be a directly connected network and your router has nothing to do with it. Unless your client and/or server is on Wifi that's bridged to the wired network? Or are client and server both connected to a switch? Is it a separate switch, or one builtin on the router?
 
Is your router compromised, hacked?
You should look what is running on your network with a sniffing tool.
 
Try setting 'UseDNS' to 'no' in /etc/ssh/sshd_config.
I bet there's some broken DNS config on that plastic thingy and/or they are intercepting/hijacking DNS requests for advertising purposes and have some broken configuration there. (seen this multiple times with german telekom back before they were sued because they assumed this was legal...)
 
blazingice said:
I'm not sure how to interpret these logs, and haven't been able to find the source of the problem. The only solution at the moment is not to have the FreeBSD laptop connected to the router, which will be very disappointing. Whilst the issue may not be FreeBSD specific, can someone point to what else I could be checking. I have also looked at the logs of the router, but apart from restarting logs, I can't see anything useful.
Click to expand...
The login procedure is:

1. Establish a TCP connection
2. Handshake encryption settings
3. Perform the User Validation
4. Launch the User's Shell

Only phases 1 and 3 can face timeouts, with 1 being the system's tcp timeout that defaults to 75 seconds and is configurable by the net.inet.tcp.keepinit sysctl(8), while 3 defaults to 2 minutes and can be configured using the host system's sshd_config(5).

Your log records you attempted to create a TCP connection in
Code: 
debug1: Connecting to 192.168.0.92 [192.168.0.92] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48

The next line then is
Code: 
debug1: connect to address 192.168.0.92 port 22: Operation timed out
which indicates the TCP connection attempt has failed. Had it succeeded, the message would have been
Code: 
debug1: Connection established.
followed by the initiation of the handshake, where your client will send it's client identifier and the server would transmit it's ssh server identifier to have the peers apply potential workarounds for known broken implementations prior to echanging certificates, i.e. you'd get the same line you have at the top of the output, just with a debug prefix as it's now being communicated, followed by a line indicating the server's SSH version, and then a myriad of cryptography settings.

If the host was not available or known to the router - it caches this knowledge for some time - it would reject connection establishment with a "Host Unreachable" ICMP package immediately, so the Router believes the host is reachable and transmits your connection attempt to the port it thinks connects to the host. The most feasible things to check is whether your box is reachable, e.g. by pinging it. If ping works, but TCP doesn't, it's most likely a firewall issue.
 
sko said:
Try setting 'UseDNS' to 'no' in /etc/ssh/sshd_config.
I bet there's some broken DNS config on that plastic thingy and/or they are intercepting/hijacking DNS requests for advertising purposes and have some broken configuration there. (seen this multiple times with german telekom back before they were sued because they assumed this was legal...)
Click to expand...
The login is performed to a host refered to by its IP address rather then a hostname, so DNS is only implied for recording the login attempt to syslog. If the DNS server was to provide a malicious response, it would only affect the syslog entry for the login attempt on the server, while a failure of the DNS server to provide an NXDOMAIN response will cause the handshake process to delay by around 5 seconds and then log the IP address instead. Even if the DNS timeout was set to be longer, as this is done during handshake, the server will send keepalive packages in order to keep the connection alive for whatever time it takes to finish the DNS lookup.

Whatever trojans the german intelligence agencies have installed on the router - eerr, I mean, whatever law enforcement mechanisms the regulatory bodies have installed on the router, of cause - they cannot compromize your SSH connection unless you either don't verify the autheticity of the host fingerprint you are shown when you login to the box for the first time, you purposefully manipulate your ~/.known_hosts file to allow the router to man in the middle your connection to a host you previously verified to be authentic, or you setup a DNS based authentication scheme and use a DNS server which doesn't use DNSSEC, allowing the router to MITM by spoofing DNS Responses.
 
Back then (maybe ~5+ years?) they hijacked DNS requests that returned NXDOMAIN and served a CNAME to some website garbage aggregation page where they basically told you "we couldn't find what you were looking for, but here are some articles from our news portal nobody wants to use and we found some stuff for you to buy because we sent that local hostname to a bunch of search engines..."

I found on multiple occasions, that local hosts behind such a router often failed to connect via ssh until the 'UseDNS' option was disabled. Mostly they would just take quite long (IIRC the timeout was reduced at some point), but around 50% of all tries the connection completely failed.

Recently I saw the exact same DNS hijacking and redirecting to some portal of the provider on some 5G mobile router someone was using; so I suspect this practice hasn't died yet and is still causing problems (because NXDOMAIN replies exist and are important for various reasons).


Given there are plenty of search results regarding this "vodafone ultra hub" (in various incarnations) and frequent reboots and/or connectivity problems with various hardware, I'd guess the best option is to get rid of that toy - or at least configure it to pass through the actual connection (if possible), so a real router can be used instead.
To verify, OP could connect that laptop and his client to a switch (without any uplink to the vodafone router) and check if ssh logins are working.
 
Thanks everyone for your comments. Sorry for delayed feedback due to summer holidays. Here is a bit more detail in response to some of your comments:

This is my home network 192.168.0.0/24. Laptop (serving as FreeBSD home server) is connect to Vodafone router via a seperate netgear switch. I also have a FreeBSD desktop (14/stable) connected via ethernet cable to the same netgear switch. Even when I'm trying to ssh from FreeBSD desktop to FreeBSD laptop (both connected to the same netgear switch) I get the same problem:

ssh USERNAME@192.168.0.92
kex_exchange_identification: read: Operation timed out
banner exchange: Connection to 192.168.0.92 port 22: Operation timed out

I have the same issue when trying to ssh via another Laptop (MacOS) which is connected to the same network via wifi.

The strange thing is that the above issue is not consistent. Sometimes (less often) the connection works, so now whenever I get the above issue I restart netif of the desktop:

service netif restart

Or switch off and on the wifi in MacOS. Then immediately ssh to the server. This works almost 95% of the times. If I was to exit the ssh connection and re connect or even try to establish a parallel ssh connection to the laptop (server), strangely I would get the same timeout issue.

To test the issue a bit further I installed linux on a similar dell laptop and connect it to the same netgear switch. Then tried to ssh to it via the same desktop and laptop. No issue at all with Linux! So I'm finding difficult to figure out what's causing this very inconsistent bevahour.

As for the router ssh-ing my server, this seems a separate issue, which I found out it is observed by other vodafone customers, as reported here.

Any suggestion how to troubleshoot the incosistant ssh-ing issue. I tried disabling UseDNS in sshd_config, but no difference.
 
Are you using DHCP or static IP?
Do you have ping between the laptop and desktop PC?
Show the current settings on both laptop and desktop:
ifconfig
netstat -rn
arp -a
ping -c 3 192.168.0.92 or whatever is your laptop IP address.
arp -a after the ping

Do you have firewall?
 
try to add the hosts you ssh from on /etc/hosts on the server
even with UseDNS turned off it may be that reverse lookup is done by libaccess
or monitor dns queries with tcpdump / strace the sshd process while trying to login
 
You must log in or register to reply here.
Back
Top