Solved Strange routing problem

P

Peter2121

Hi,
I have a FreeBSD server with VNET jails, connected to another network using IPsec (strongswan is working on this server).
The IPsec connection works fine, my jails can communicate with hosts in another network.
Now I want that my host itself communicates with remote hosts using the same tunnel.
So, I added a second loopback interface lo1 with the IP address from the jails network and added a route to the remote network by this lo1 interface:
# route add 10.192.20.0/24 -iface lo1
Now I can ping remote hosts.
The problem is that I only can ping them, neither TCP nor UDP connections work. For example, if I try to telnet a remote server - I get immediately the error on my side:
Code: 
# telnet 10.192.20.16 22
Trying 10.192.20.16...
telnet: connect to address 10.192.20.16: Network is unreachable
telnet: Unable to connect to remote host
Once again, the ping works correctly:
Code: 
# ping 10.192.20.16
PING 10.192.20.16 (10.192.20.16): 56 data bytes
64 bytes from 10.192.20.16: icmp_seq=0 ttl=63 time=10.061 ms
64 bytes from 10.192.20.16: icmp_seq=1 ttl=63 time=10.118 ms
...
What is the difference in routing of ICMP and TCP/UDP? How can I get this working?
 
You need to provide the source address of a network that's allowed to pass. Normal connections will just use the host's primary IP address as a source, which I assume isn't allowed to pass-through the IPSec tunnel (the left and right networks).

With ping(8) you can supply a source address: ping -S <source IP> <destination>

Peter2121 said:
What is the difference in routing of ICMP and TCP/UDP?
Click to expand...
There is no difference, they all use the same IP headers. Routing is based on those headers.

Code: 
root@armitage:~ # swanctl --list-conn
gw-gw: IKEv2, reauthentication every 10800s, no rekeying
  local:  X.X.X.X
  remote: Y.Y.Y.Y
  local pre-shared key authentication:
    id: armitage.example.com
  remote pre-shared key authentication:
    id: home.example.com
  net-net: TUNNEL, rekeying every 3600s
    local:  192.168.21.0/24
    remote: 192.168.10.0/24 192.168.11.0/24
root@armitage:~ # ping -c 2 192.168.10.190
PING 192.168.10.190 (192.168.10.190): 56 data bytes

--- 192.168.10.190 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
root@armitage:~ # ping -c 2 -S 192.168.21.1 192.168.10.190
PING 192.168.10.190 (192.168.10.190) from 192.168.21.1: 56 data bytes
64 bytes from 192.168.10.190: icmp_seq=0 ttl=63 time=14.576 ms
64 bytes from 192.168.10.190: icmp_seq=1 ttl=63 time=13.638 ms

--- 192.168.10.190 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 13.638/14.107/14.576/0.469 ms
 
SirDice said:
You need to provide the source address of a network that's allowed to pass. Normal connections will just use the host's primary IP address as a source, which I assume isn't allowed to pass-through the IPSec tunnel (the left and right networks).

With ping(8) you can supply a source address: ping -S <source IP> <destination>

There is no difference, they all use the same IP headers. Routing is based on those headers.
Click to expand...
When I use pings I don't provide -S option, it 'just works'.
Then, if it is a problem of source IP - I should get a timeout somehow, this is not the case, the 'Network is unreachable' error is immediate.
I think that the source address is ok when I put the route using lo1, this is another problem.
 
i suspect udp and tcp wont bound to lo1 local address
and will be rejected with icmp unreach by you or your upstream

make lo1 a gif or something not /32
add the route thru one of the IPS not thru the iface/device and see if it works
 
like ifconfig gif0 IP_OF_LO1 bogusip
route add 10.192.20.0/24 bogusip
ipsec policy should pick the packet and do what is needed
 
I'd have a look with tcpdump(1), something is returning an ICMP unreachable. Firewall perhaps?
Click to expand...
No traffic is seen by tcpdump, even in case of pings passing.

make lo1 a gif or something not /32
add the route thru one of the IPS not thru the iface/device and see if it works
Click to expand...
The mask of lo1 is already /24. If I use the address of lo1 in my route rule:
# route add 10.192.20.0/24 10.192.6.250
the pings still work, but TCP/UDP still don't work.
 
any ipv4 that wont interfere with you setup
192.168.22.33
so even if the kernel thinks that that ip is reachable you dont have any traffic to it
 
Code: 
# ifconfig gif0 create
# ifconfig gif0 10.192.6.251 192.168.255.254
# route add 10.192.20.0/24 192.168.255.254
add net 10.192.20.0: gateway 192.168.255.254
# ping 10.192.20.16
PING 10.192.20.16 (10.192.20.16): 56 data bytes
ping: sendto: No route to host
:(
 
Code: 
# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.17.14.254      UGS        vmx0
10.192.6.0/24      link#5             U       epair1a
10.192.6.2         link#5             UHS         lo0
10.192.6.251       link#11            UHS         lo0
10.192.20.0/24     192.168.255.254    UGS        gif0
127.0.0.1          link#2             UH          lo0
172.17.14.0/24     link#1             U          vmx0
172.17.14.10       link#1             UHS         lo0
192.168.255.254    link#11            UH         gif0
 
If I use a public IP in gif0 configuration - I see the packets leaving my external interface with the real IP as source. So, here I have a problem of source IP - that's why IPsec does not catch the packets.
It was the route missing. If it is correctly put - the same situation as with bogusip
 
if gif encapsulation is done before ipsec policy treats the packet then wont work (it used to work in the past)


can you post
setkey -D
 
Code: 
172.17.14.10[4500] XX.XXX.188.161[4500]
        esp-udp mode=tunnel spi=3278335944(0xc3676fc8) reqid=2(0x00000002)
        seq=0x00004086 replay=0 flags=0x00000000 state=mature
        created: Jun  3 16:18:29 2021   current: Jun  3 16:22:43 2021
        diff: 254(s)    hard: 800(s)    soft: 540(s)
        last: Jun  3 16:18:29 2021      hard: 0(s)      soft: 0(s)
        current: 3273352(bytes) hard: 0(bytes)  soft: 0(bytes)
        allocated: 16518        hard: 0 soft: 0
        sadb_seq=1 pid=56136 refcnt=1
XX.XXX.188.161[4500] 172.17.14.10[4500]
        esp-udp mode=tunnel spi=3268296528(0xc2ce3f50) reqid=2(0x00000002)
        seq=0x00004583 replay=4 flags=0x00000000 state=mature
        created: Jun  3 16:18:29 2021   current: Jun  3 16:22:43 2021
        diff: 254(s)    hard: 800(s)    soft: 447(s)
        last: Jun  3 16:18:29 2021      hard: 0(s)      soft: 0(s)
        current: 2370943(bytes) hard: 0(bytes)  soft: 0(bytes)
        allocated: 17795        hard: 0 soft: 0
        sadb_seq=0 pid=56136 refcnt=1
(encryption info removed)
 
Code: 
# setkey -DP
10.192.20.0/24[any] 10.192.6.0/24[any] any
        in ipsec
        esp/tunnel/XX.XXX.188.161-172.17.14.10/unique:2
        created: Jun  1 19:38:06 2021  lastused: Jun  3 16:35:04 2021
        lifetime: 9223372036854775807(s) validtime: 0(s)
        spid=5 seq=3 pid=56900 scope=global
        refcnt=1
10.192.6.0/24[any] 10.192.20.0/24[any] any
        out ipsec
        esp/tunnel/172.17.14.10-XX.XXX.188.161/unique:2
        created: Jun  1 19:38:06 2021  lastused: Jun  3 16:35:04 2021
        lifetime: 9223372036854775807(s) validtime: 0(s)
        spid=6 seq=1 pid=56900 scope=global
        refcnt=1
 
looks ok; try to make the gif with real tunnel endpoints
ifconfig gif0 localsame remotesame tunnel 172.17.14.10 XX.XXX.188.161
and see if it is any better
 
You must log in or register to reply here.
Back
Top