Solved Jail unreachable by other jails / pf

Hello,
i am struggling with pf and i don't know what i miss...
Any help is super welcome :)

I've got a jail-A with openldap running, a jail-B which run ejabberd, both on the same host, and a Nextcloud instance on a separate machine.
The Nextcloud instance can communicate with the Jail-A without trouble, it works well, but the Jail-B can't join slapd on the Jail-A.
- Jail-A can ping Jail-B
Bash:
[root@ldap /]# ping -c 3 192.168.0.X
PING 192.168.0.X (192.168.0.X): 56 data bytes
64 bytes from 192.168.0.X: icmp_seq=0 ttl=64 time=0.065 ms
64 bytes from 192.168.0.X: icmp_seq=1 ttl=64 time=0.061 ms
64 bytes from 192.168.0.X: icmp_seq=2 ttl=64 time=0.218 ms

--- 192.168.0.X ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.061/0.115/0.218/0.073 ms
- Jail-B can ping Jail-A
Bash:
[root@jabber /]# ping -c 3 192.168.0.Y
PING 192.168.0.Y (192.168.0.Y): 56 data bytes
64 bytes from 192.168.0.Y: icmp_seq=0 ttl=64 time=0.047 ms
64 bytes from 192.168.0.Y: icmp_seq=1 ttl=64 time=0.216 ms
64 bytes from 192.168.0.Y: icmp_seq=2 ttl=64 time=0.150 ms

--- 192.168.0.Y ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.047/0.138/0.216/0.070 ms
Jail-A is listening on both: 389 and 636 ports
Bash:
[root@ldap /]# sockstat -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS     
...
ldap     slapd      29964 6  stream /var/run/openldap/ldapi
ldap     slapd      29964 7  tcp4   *:636                 *:*
ldap     slapd      29964 8  tcp4   *:389                 *:*

The ldap server is working and reachable from the outside:
Bash:
dervishe@moss ~ $ ldapsearch -H "ldaps://ldap.mydoma.in" -D "cn=admin,dc=mydoma,dc=in" "(uid=operator)" -W
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <ou=membres,dc=mydoma,dc=in> (default) with scope subtree
# filter: (uid=operator)
# requesting: ALL
#

# operator, membres, mydoma.in
dn: cn=operator,ou=membres,dc=mydoma,dc=in
cn: operator
objectClass: inetOrgPerson
objectClass: shadowAccount
sn: operator
uid: operator

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1
But when i try from the other jail (with fqdn or ip):
Bash:
[root@jabber ~]# ldapsearch -H "ldaps://ldap.mydoma.in" -D "cn=admin,dc=mydoma,dc=in" -W "(uid=operator)"
Enter LDAP Password:
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)

[root@jabber ~]# ldapsearch -H "ldaps://192.168.0.Y" -D "cn=admin,dc=mydoma,dc=in" -W "(uid=operator)"
Enter LDAP Password:
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
The pf.conf from the host:
Bash:
ext_if = igb0                                                                                                                        
int_if = bridgepub
int_priv = bridgepriv
jabber = "192.168.0.X"
oldap = "192.168.0.Y"
icmp_types = "{ echoreq unreach }"
ports_tcp = "{ domain }"
jabber_ports = "{ 5222 5223 5269 5280 5349 }"
ldap_ports = "{ 389 636 }"
martians="{ 127.0.0.0/8 192.168.0.0/16 172.16.0.0/12
    10.0.0.0/8 169.254.0.0/16 192.0.2.0/24
    0.0.0.0/8 240.0.0.0/4 }"

set skip on lo0
set block-policy return
set loginterface $ext_if

scrub in on $ext_if all fragment reassemble

rdr on $ext_if inet proto tcp from any to ($ext_if) port $jabber_ports -> $jabber
rdr on $ext_if inet proto tcp from any to ($ext_if) port $ldap_ports -> $oldap
nat on $ext_if inet from $int_if:network to any -> ($ext_if)
nat on $ext_if inet from $int_priv:network to any -> ($ext_if)

antispoof quick for $ext_if

block return in log all
block out all
anchor "blacklistd/*" in on $ext_if
block drop in quick on $ext_if from $martians to any
block drop out quick on $ext_if from any to $martians
pass inet proto icmp icmp-type $icmp_types
pass proto udp to port domain
pass proto udp to port 6277
pass proto tcp to port ssh
pass proto tcp to port $ports_tcp
pass proto tcp to port $jabber_ports
pass proto tcp to port $ldap_ports
I don't know what i'm missing...
 
Please post the output of ifconfig on the host, I need to see where things are connected to.
 
Thanks for your reply :)
I put it in the attach file

Code:
bridgepub: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 58:9c:fc:10:ff:8e
	inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: e0a_rproxy flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 10 priority 128 path cost 2000
	member: e0a_ldap flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 9 priority 128 path cost 2000
	member: e0a_jabber flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 8 priority 128 path cost 2000
	member: e0a_mail flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 7 priority 128 path cost 2000
	groups: bridge
	nd6 options=9<PERFORMNUD,IFDISABLED>
bridgepriv: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 58:9c:fc:10:ff:bf
	inet 172.16.1.1 netmask 0xffffff00 broadcast 172.16.1.255
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: e0a_search flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 15 priority 128 path cost 2000
	member: e0a_redis flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 14 priority 128 path cost 2000
	member: e0a_db flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 13 priority 128 path cost 2000
	member: e0a_www flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 11 priority 128 path cost 2000
	member: e1a_rproxy flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 12 priority 128 path cost 2000
	groups: bridge
	nd6 options=9<PERFORMNUD,IFDISABLED>
e0a_mail: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:40:e9:fe:75:0a
	inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e0a_jabber: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:b0:91:7b:06:0a
	inet 192.168.0.7 netmask 0xffffff00 broadcast 192.168.0.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e0a_ldap: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:a7:d0:98:59:0a
	inet 192.168.0.9 netmask 0xffffff00 broadcast 192.168.0.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e0a_rproxy: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:49:59:33:2a:0a
	inet 192.168.0.5 netmask 0xffffff00 broadcast 192.168.0.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e1a_rproxy: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:9b:a8:9a:67:0a
	inet 172.16.1.3 netmask 0xffffff00 broadcast 172.16.1.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e0a_www: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:65:26:23:1b:0a
	inet 172.16.1.5 netmask 0xffffff00 broadcast 172.16.1.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e0a_db: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:e0:ca:45:b8:0a
	inet 172.16.1.7 netmask 0xffffff00 broadcast 172.16.1.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e0a_redis: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:08:be:29:5c:0a
	inet 172.16.1.9 netmask 0xffffff00 broadcast 172.16.1.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
e0a_search: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:7b:de:e0:f7:0a
	inet 172.16.1.11 netmask 0xffffff00 broadcast 172.16.1.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

[Mod: copy/pasted the info in a code block, much easier to read]
 
Ok, so these are VNET jails. And both jabber and ldap hosts are connected to bridgepub. Both are on the same subnet (192.168.0.0/24) and thus can directly connect to each other (no routing involved).

As these are VNET jails, is there a firewall running in the ldap jail? That might be blocking the access.
 
Try just testing the port; nc -zv ldap.mydoma.in 389 for example. Not sure if this is just some obfuscation, but also check if ldap.mydoma.in resolves to the correct IP address.
 
I made a mistake, the connection fail.
Bash:
[root@jabber /]# nc -zv ldap.mydoma.in 389
nc: connect to ldap.mydoma.in port 389 (tcp) failed: Connection refused
And when i ping the domain name, i got the external ip responding.
Edit: i changed the result of the test
 
And just for the record, this is the ldap jail rc.conf:
Bash:
[root@ldap /]# cat /etc/rc.conf
slapd_enable="YES"
slapd_sockets="/var/run/openldap/ldapi"
slapd_flags='-h "ldapi://%2fvar%2frun%2fopenldap%2fldapi/ ldaps://0.0.0.0/ ldap://0.0.0.0/"'
 
Code:
nc: connect to ldap.mydoma.in port 389 (tcp) failed: Connection refused
Ok, 'connection refused' means you're trying to connect to a closed port. You're either connecting to the wrong IP address or there's nothing listening on that port. If a firewall was blocking the connection you typically get a 'connection timed-out' error.

I assume ldap.mydoma.in actually resolves to the address of igb0. That won't work from the jabber host. The redirection only happens on incoming (i.e. from outside the host) traffic coming in on igb0. The traffic between jabber and the ldap host never passes this interface (it never leaves the host). And so it tries to connect to port 389 on the host itself. Which doesn't have the LDAP service running, and thus responds with 'connection refused'. The jabber host needs to connect to the IP address of the ldap host; 192.168.0.9, not the 'external' IP of the host.
 
yep but it is what i don't get:
the ldap jail is listening on ports 389 and 636 and when i ping the fqdn of the ldap jail, i got the host ip (as when i try to connect to it from another host). But after, if the query come from the outside, the ldap respond. If it comes from the other jails, it fail...
 
and when i ping the fqdn of the ldap jail
You're pinging the IP address, not the ldap.mydoma.in hostname. At least not in anything you posted. When you ping ldap.mydoma.in it's the host that responds, not the jail.
 
In fact, if i put the internal ip address for my ldap jail in the ejabberd config, it works...
I still don't get it.
 
You're pinging the IP address, not the ldap.mydoma.in hostname. At least not in anything you posted. When you ping ldap.mydoma.in it's the host that responds, not the jail.
Sorry, when i ping the fqdn (ldap.mydoma.in) from another host or from another jails, it responds ok from the public ip address without trouble.
.
 
I still don't get it.
This line:
Code:
rdr on $ext_if inet proto tcp from any to ($ext_if) port $ldap_ports -> $oldap
Only works for packets coming in on igb0, that's external traffic coming in on the host.

Sorry, when i ping the fqdn (ldap.mydoma.in) from another host or from another jails, it responds ok from the public ip address without trouble.
It's the host itself that responds to those pings.
 
Ok, i understand that, but from Jabber jail to LDAP jail it works with IP and not with FQDN:
Bash:
[root@jabber /]# nc -zv 192.168.0.Y 389
Connection to 192.168.0.8 389 port [tcp/ldap] succeeded!
In fact if i put the ldap jail IP address in the /etc/hosts file in the jabber jail, it work.
There is something i'm missing here...
 
here is something i'm missing here...
ldap.mydoma.in resolves to the IP address of the host, not the ldap jail's IP address. So you're connecting to the wrong IP address (from the jabber jail's point of view).
 
Ok then putting the address ldap jail address in the /etc/hosts of the others jails that have to communicate with it, is the good way to solve the problem ?
Btw, thanks for your time :)
 
Back
Top