After IPSec is set up, the L2TP client cannot connect to the server

Dear all,

I have set up an L2TP server using mpd5, and it works fine. However, when I switch IPSec (ipsec-tools) on, the client doesn't seem to be able to connect to the server: it looks like the server doesn't get any packets from the client.
The IPSec configs follow:
Client setkey.conf:
Code:
flush;
spdflush;
spdadd 0.0.0.0/0 <server IP>[1701] udp -P out  ipsec esp/transport//require;
spdadd <server IP>[1701] 0.0.0.0/0 udp -P in  ipsec esp/transport//require;
Server setkey.conf:
Code:
flush;
spdflush;
spdadd 0.0.0.0/0[any] <server IP>[1701] udp -P in  ipsec esp/transport//require;
spdadd <server IP>[1701] 0.0.0.0/0[any] udp -P out ipsec esp/transport//require;
Client racoon.conf:
Code:
path pre_shared_key "/usr/local/etc/racoon/psk.txt";

log info;

padding {
  maximum_length 20;
  randomize off; 
  strict_check off;
  exclusive_tail off;
}

listen {
  isakmp <client local IP>;
  strict_address;
  adminsock "/var/db/racoon/racoon.sock";
}

timer {
  counter 5;
  interval 20 sec;
  persend 1;
  phase1 30 sec;
  phase2 15 sec;
}

remote <server IP> {
  exchange_mode aggressive,main;
  lifetime time 24 hour;
  my_identifier address;
  peers_identifier address;
  passive off;
  generate_policy off;
#     nat_traversal on;
  proposal {
  encryption_algorithm 3des;
  hash_algorithm sha1;
  authentication_method pre_shared_key;
  dh_group 2;
  }
}

sainfo anonymous {
  encryption_algorithm 3des;
  authentication_algorithm hmac_md5, hmac_sha1;
  lifetime time 1 hour ;
  compression_algorithm deflate;
}
Server racoon.conf:
Code:
path pre_shared_key "/usr/local/etc/racoon/psk.txt";

log info;

padding { 
  maximum_length 20; 
  randomize off; 
  strict_check off; 
  exclusive_tail off; 
} 



listen { 
  isakmp <server IP>; 

} 

timer { 
  counter 5; 
  interval 20 sec; 
  persend 1; 
  phase1 30 sec; 
  phase2 15 sec; 
} 

remote anonymous { 
  exchange_mode aggressive,main; 
  lifetime time 24 hour;   
  passive off; 
  generate_policy on;

  proposal_check obey; 

  ike_frag on; 

  proposal { 
  hash_algorithm sha1; 
  authentication_method pre_shared_key;
  dh_group 2;
  }
}

sainfo anonymous {
  encryption_algorithm 3des;
  authentication_algorithm hmac_md5, hmac_sha1;
  lifetime time 1 hour ;
  compression_algorithm deflate;
}

I tried switching NAT Traversal on and off without any effect. What am I missing here? I guess I've misconfigured IPSec. However, here's what setkey -D shows at the client while the IPSec connection is on:
Code:
% sudo setkey -D
<local client IP> <server IP>
  esp mode=transport spi=50815713(0x030762e1) reqid=0(0x00000000)
  E: 3des-cbc  96862cbd fc81562a 1558e323 450ab7b4 e1e0c588 8fa7dc3d
  A: hmac-md5  3ca7b767 b29bba44 f4044ccf ab795005
  seq=0x00000042 replay=4 flags=0x00000000 state=mature
  created: Oct 25 01:11:57 2015  current: Oct 25 01:27:44 2015
  diff: 947(s)  hard: 3600(s)  soft: 2880(s)
  last: Oct 25 01:18:48 2015  hard: 0(s)  soft: 0(s)
  current: 10560(bytes)  hard: 0(bytes)  soft: 0(bytes)
  allocated: 66  hard: 0 soft: 0
  sadb_seq=1 pid=67500 refcnt=2
<server IP> <local client IP>
  esp mode=transport spi=227125489(0x0d89a8f1) reqid=0(0x00000000)
  E: 3des-cbc  1f7dde64 8338c675 956bf667 68e850d8 1a953563 cd7ac898
  A: hmac-md5  16574733 c6e7ce84 47d195b4 2c8bd45f
  seq=0x00000000 replay=4 flags=0x00000000 state=mature
  created: Oct 25 01:11:57 2015  current: Oct 25 01:27:44 2015
  diff: 947(s)  hard: 3600(s)  soft: 2880(s)
  last:  hard: 0(s)  soft: 0(s)
  current: 0(bytes)  hard: 0(bytes)  soft: 0(bytes)
  allocated: 0  hard: 0 soft: 0
  sadb_seq=0 pid=67500 refcnt=1
At the server, setkey -D shows something like this.
Oh, and the client is behind a NAT, but as I've told before, switching NAT Traversal on and off doesn't help.
 
... when I switch IPSec (ipsec-tools) on, the client doesn't seem to be able to connect to the server: it looks like the server doesn't get any packets from the client. ...
...
Oh, and the client is behind a NAT, but as I've told before, switching NAT Traversal on and off doesn't help.

Perhaps this is related to the same inability that prevents Windows clients from connecting to a L2TP/IPsec server from behind NAT, by exactly the same symptom, namely, the IPsec connection has been established but nothing arrives on the L2TP port.

When decapsulating the NAT-T-udp packets, FreeBSD strips off the respective checksum flags, assuming that these are no longer valid. In file /usr/sys/netinet/udp_usrreq.c look for the function udp4_espdecap(), and there you will find:
Code:
...
      /*
      * We cannot yet update the cksums so clear any
      * h/w cksum flags as they are no longer valid.
      */
     if (m->m_pkthdr.csum_flags & CSUM_DATA_VALID)
         m->m_pkthdr.csum_flags &= ~(CSUM_DATA_VALID|CSUM_PSEUDO_HDR);
...
Once I removed that five lines from said file and rebuilt the kernel, Windows clients are able to connect from behind NAT. Chances are that this would resolve also your problem of connecting a non-Windows client from behind NAT. Of course the kernel had to be build with options IPSEC_NAT_T.
 
Perhaps this is related to the same inability that prevents Windows clients from connecting to a L2TP/IPsec server from behind NAT, by exactly the same symptom, namely, the IPsec connection has been established but nothing arrives on the L2TP port.

When decapsulating the NAT-T-udp packets, FreeBSD strips off the respective checksum flags, assuming that these are no longer valid. In file /usr/sys/netinet/udp_usrreq.c look for the function udp4_espdecap(), and there you will find:
Code:
...
      /*
      * We cannot yet update the cksums so clear any
      * h/w cksum flags as they are no longer valid.
      */
     if (m->m_pkthdr.csum_flags & CSUM_DATA_VALID)
         m->m_pkthdr.csum_flags &= ~(CSUM_DATA_VALID|CSUM_PSEUDO_HDR);
...
Once I removed that five lines from said file and rebuilt the kernel, Windows clients are able to connect from behind NAT. Chances are that this would resolve also your problem of connecting a non-Windows client from behind NAT. Of course the kernel had to be build with options IPSEC_NAT_T.
Just to clarify: you're talking about the server, right? So I have to comment out the mentioned lines and rebuild the server's kernel. Did I understand you correctly?
 
Yes, I was talking about the server, however, now, after you rose the question, I think that this issue could affect the client side as well, since NAT-T-UDP responses from the server would be decapsulated by the very same function on the client. I am sorry, however, I cannot be more specific because my L2TP/IPsec server is utilized by Windows, Mac and iOS clients only.
 
Recompiling the client kernel didn't work. What else can I do with that? What can be wrong with the IPSec config I'm using?
 
Please show the respective racoon logs of the client and the server of the same connection attempt.

By chance do you have an iOS device at hand? The builtin L2TP/IPsec client of iOS should simply work if your server is setup correctly. Thus, if you can connect with an iPhone or iPad, then you won't need to troubleshoot your server configuration, but you could focus on the client configuration only, and vice versa, if iOS cannot connect, then you would focus first on fixing your server configuration.

In addition verify that you have the following directive in file /etc/rc.conf:
Code:
...
gateway_enable="YES"
...
My IPsec server sits behind an ipfw(8) firewall. Of course I opened udp ports 500 and 4500 (as you did, otherwise you would not see any IPsec connection), however, in addition I need to allow all traffic on the netgraph interfaces that MPD5 would create for its L2TP connections:
Code:
...
/sbin/ipfw -q add 40 allow ip from any to any via ng*
...
 
Tried to connect an iOS device (iPad mini with the latest iOS), and the connection succeeded pretty well. The device was also behind a NAT, btw.
 
And here's racoon log on racoonctl vpn-connect <server IP>:
Code:
Oct 25 23:10:16 thinkpad racoon: INFO: initiate new phase 1 negotiation: <local client IP>[500]<=><server IP>[500]
Oct 25 23:10:16 thinkpad racoon: INFO: begin Aggressive mode.
Oct 25 23:10:16 thinkpad racoon: INFO: received Vendor ID: DPD
Oct 25 23:10:16 thinkpad racoon: INFO: ISAKMP-SA established <local client IP>[500]-<server IP>[500] spi:a1d220613fea447b:9e69abdcf2ac52af
 
Tried to connect an iOS device (iPad mini with the latest iOS), and the connection succeeded pretty well. The device was also behind a NAT, btw.
I have no experience with FreeBSD L2TP/IPsec clients, and cannot be of much help here. I would start the troubleshooting by comparing the racoon logs on the server of a failed FreeBSD connection attempt with a successful iPad connection establishment. Perhaps this reveals some insights on the problem.

As a side note, racoon got a record on issues with tearing down connections, and sometimes it leaves stale SA's sitting forever in the database, preventing the establishment of new connections. Until you sorted everything out, I suggest that you manually flush the SA database on the server and perhaps client side with /usr/local/sbin/setkey -F before any new connection attempt. Ah, and you want to use the setkey utility that has been installed by the security/ipsec-tools.
 
And here's racoon log on `racoonctl vpn-connect <server IP`:
Code:
Oct 25 23:10:16 thinkpad racoon: INFO: initiate new phase 1 negotiation: <local client IP>[500]<=><server IP>[500]
Oct 25 23:10:16 thinkpad racoon: INFO: begin Aggressive mode.
Oct 25 23:10:16 thinkpad racoon: INFO: received Vendor ID: DPD
Oct 25 23:10:16 thinkpad racoon: INFO: ISAKMP-SA established <local client IP>[500]-<server IP>[500] spi:a1d220613fea447b:9e69abdcf2ac52af
This is a pretty short connection log. Try again with NAT-T on the client side enabled nat_traversal on;. By the way, the log file of racoon should be /var/log/racoon.log. And in order to see directly what is happening on the server, I issue tail -fn50 /var/log/racoon.log before trying to establish a connection.
 
Here's what racoon at the server says when I type racoonctl vpn-connect <server IP>:
Code:
Oct 25 21:40:55 bein racoon: INFO: respond new phase 1 negotiation: <server IP>[500]<=><client ext IP>[500]
Oct 25 21:40:55 bein racoon: INFO: begin Identity Protection mode.
Oct 25 21:40:55 bein racoon: INFO: received Vendor ID: RFC 3947
Oct 25 21:40:55 bein racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
Oct 25 21:40:55 bein racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
Oct 25 21:40:55 bein racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-00
Oct 25 21:40:55 bein racoon: INFO: received broken Microsoft ID: FRAGMENTATION
Oct 25 21:40:55 bein racoon: INFO: received Vendor ID: DPD
Oct 25 21:40:55 bein racoon: [<client ext IP>] INFO: Selected NAT-T version: RFC 3947
Oct 25 21:40:55 bein racoon: [<server IP>] INFO: Hashing <server IP>[500] with algo #2
Oct 25 21:40:55 bein racoon: INFO: NAT-D payload #0 verified
Oct 25 21:40:55 bein racoon: [<client ext IP>] INFO: Hashing <client ext IP>[500] with algo #2
Oct 25 21:40:55 bein racoon: INFO: NAT-D payload #1 doesn't match
Oct 25 21:40:55 bein racoon: INFO: NAT detected: PEER
Oct 25 21:40:55 bein racoon: [<client ext IP>] INFO: Hashing <client ext IP>[500] with algo #2
Oct 25 21:40:55 bein racoon: [<server IP>] INFO: Hashing <server IP>[500] with algo #2
Oct 25 21:40:55 bein racoon: INFO: Adding remote and local NAT-D payloads.
Oct 25 21:40:55 bein racoon: INFO: NAT-T: ports changed to: <client ext IP>[4500]<-><server IP>[4500]
Oct 25 21:40:55 bein racoon: INFO: KA list add: <server IP>[4500]-><client ext IP>[4500]
Oct 25 21:40:55 bein racoon: INFO: ISAKMP-SA established <server IP>[4500]-<client ext IP>[4500] spi:1745521a905871e6:8fab0d195b1e0530
And here's what it says when I start mpd5:
Code:
Oct 25 21:43:36 bein racoon: INFO: respond new phase 2 negotiation: <server IP>[4500]<=><client ext IP>[4500]
Oct 25 21:43:36 bein racoon: INFO: Update the generated policy : 192.168.88.226/32[0] <server IP>/32[1701] proto=udp dir=in
Oct 25 21:43:36 bein racoon: INFO: Adjusting my encmode UDP-Transport->Transport
Oct 25 21:43:36 bein racoon: INFO: Adjusting peer's encmode UDP-Transport(4)->Transport(2)
Oct 25 21:43:36 bein racoon: INFO: IPsec-SA established: ESP/Transport <server IP>[500]-><client ext IP>[500] spi=5054525(0x4d203d)
Oct 25 21:43:36 bein racoon: INFO: IPsec-SA established: ESP/Transport <server IP>[500]-><client ext IP>[500] spi=262758263(0xfa95f77)
 
And here's the client racoon log:
Code:
Oct 26 00:40:50 thinkpad racoon: INFO: Flushing all SAs for peer <server IP>
Oct 26 00:40:56 thinkpad racoon: INFO: accept a request to establish IKE-SA: <server IP>
Oct 26 00:40:56 thinkpad racoon: INFO: initiate new phase 1 negotiation: <client int IP>[500]<=><server IP>[500]
Oct 26 00:40:56 thinkpad racoon: INFO: begin Identity Protection mode.
Oct 26 00:40:56 thinkpad racoon: INFO: received Vendor ID: RFC 3947
Oct 26 00:40:56 thinkpad racoon: INFO: received Vendor ID: DPD
Oct 26 00:40:56 thinkpad racoon: INFO: received broken Microsoft ID: FRAGMENTATION
Oct 26 00:40:56 thinkpad racoon: [<server IP>] INFO: Selected NAT-T version: RFC 3947
Oct 26 00:40:56 thinkpad racoon: [<server IP>] INFO: Hashing <server IP>[500] with algo #2
Oct 26 00:40:56 thinkpad racoon: [<client int IP>] INFO: Hashing <client int IP>[500] with algo #2
Oct 26 00:40:56 thinkpad racoon: INFO: Adding remote and local NAT-D payloads.
Oct 26 00:40:56 thinkpad racoon: [<client int IP>] INFO: Hashing <client int IP>[500] with algo #2
Oct 26 00:40:56 thinkpad racoon: INFO: NAT-D payload #0 doesn't match
Oct 26 00:40:56 thinkpad racoon: [<server IP>] INFO: Hashing <server IP>[500] with algo #2
Oct 26 00:40:56 thinkpad racoon: INFO: NAT-D payload #1 verified
Oct 26 00:40:56 thinkpad racoon: INFO: NAT detected: ME
Oct 26 00:40:56 thinkpad racoon: INFO: KA list add: <client int IP>[4500]-><server IP>[4500]
Oct 26 00:40:56 thinkpad racoon: INFO: ISAKMP-SA established <client int IP>[4500]-<server IP>[4500] spi:1745521a905871e6:8fab0d195b1e0530
Oct 26 00:43:37 thinkpad racoon: INFO: initiate new phase 2 negotiation: <client int IP>[4500]<=><server IP>[1701]
Oct 26 00:43:37 thinkpad racoon: INFO: NAT detected -> UDP encapsulation (ENC_MODE 2->4).
Oct 26 00:43:37 thinkpad racoon: INFO: Adjusting my encmode UDP-Transport->Transport
Oct 26 00:43:37 thinkpad racoon: INFO: Adjusting peer's encmode UDP-Transport(4)->Transport(2)
Oct 26 00:43:37 thinkpad racoon: INFO: IPsec-SA established: ESP/Transport <client int IP>[500]-><server IP>[500] spi=262758263(0xfa95f77)
Oct 26 00:43:37 thinkpad racoon: INFO: IPsec-SA established: ESP/Transport <client int IP>[500]-><server IP>[500] spi=5054525(0x4d203d)
Oct 26 00:44:29 thinkpad racoon: WARNING: unrecognized route message with rtm_type: 18
The last WARNING in the client log looks strange to me, let me google this warning, in fact.
NAT_T is enabled on both the client and the server: in the kernels and in racoon configs.
 
According to route(4), code 18 = hex 0x12 is:
Code:
RTM_IEEE80211   0x12   /* IEEE80211 wireless event */
I tend to assume that this is unrelated to the inability of establishing a L2TP/IPsec connection from your FreeBSD client to your server.
 
Hm, in fact, I'm trying to connect via wifi. Let me try using an ethernet cable.
EDIT: yeah, that was obviously unrelated
 
Back
Top