Solved Problem with PPPoE (not sorted out by searching the net)

[brunoschwartz]

Dear all,

I am new to FreeBSD and trying to connect to my ISP by using the PPP according to the handbook.

It is disconnecting with the following in the log

Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: Received NGM_PPPOE_ACNAME (hook "acc-aln1.mr-alt")
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: Received NGM_PPPOE_SESSIONID
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: Received NGM_PPPOE_SUCCESS
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: deflink: carrier -> login
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: deflink: login -> lcp
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Warning: deflink: Reducing configured MRU from 1500 to 1492
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: bundle: Authenticate
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: deflink: his = CHAP 0x05, mine = none
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: Chap Input: CHALLENGE (41 bytes from acc-aln1.mr-alt)
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: Chap Output: RESPONSE (login name)
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: Chap Input: SUCCESS (CHAP authentication success)
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: deflink: lcp -> open
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Phase: bundle: Network
Jan 18 21:09:40 testingbungalow ppp[534]: tun0: Warning: 0.0.0.0: Change route failed: errno: No such process
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: open -> lcp
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Warning: 0.0.0.0: Change route failed: errno: No such process
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: bundle: Terminate
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: Received NGM_PPPOE_CLOSE
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Device disconnected
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Disconnected!
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: lcp -> logout
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Disconnected!

I used the standard ppp.conf from the handbook

default:
  set log Phase Chat tun Command Connect Filter Error Alert
  set ifaddr 10.0.0.1/0 10.0.0.2/0

plusnet:
  set device PPPoE:nfe1 # replace xl1 with your Ethernet device
  set authname xxxxxxx
  set authkey yyyyyyyy
  set dial
  set login
  add default HISADDR
  enable dns

I also tried another configuration mentioned in an old post on this forum with similar problem

default:
  set log Phase Chat tun Command Connect Filter Error Alert

plusnet:
  set device PPPoE:nfe1 # replace xl1 with your Ethernet device
  set redial 1 0
  set reconnect 3 23
  set mtu max 1492
  set mru max 1492
  set speed sync
  set server /var/run/internet "" 0177
  set dial
  set login
  set authname xxxxx
  set authkey xxxxxx
  disable acfcomp protocomp
  disable ipv6cp
  enable mssfixup
  enable dns
  enable lqr
  enable echo
  accept lqr
  add! default HISADDR
  set timeout 0
  open

Can someone tell me how this can be fixed?

BW,

B

 

[Chris_H]

OK. It's been a l-o-n-g time since I've had the need to setup DialUP. But here's trying...
Are you sure your ISP uses PPPoE? Could it be PPPoA? I see you get authenticated, but the problem seems to be in receiving an IP from their DHCP. Have you asked them for the details you need to get connected? They might request that you use a different private IP block -- 192.168.0.1, for example. It might be a trip to their support page, or maybe better, a phone call.

All the best.

--Chris

 

[brunoschwartz]

Hello Chris,

Thanks for your reply. It is PPPoE I am after, indeed. When I talked to them they could not give me any suggestions. I have a static IP which is however assigned through DHCP, but it is always the same IP. I don't understand in details how ppp works, I am guessing that tun interface will get address for ppp and my network adapter should get the IP address by DHCP.

One of the things then can have something to do with my problem is a log during boot

Jan 23 23:06:36 testing kernel: Entropy harvesting: interrupts ethernet point_to_point swi.

I don't know what it does, but can it be interfering with the process of getting IP?

I am also attaching several outputs just in case it could be helpful
---------------------

nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=c219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
  ether 00:17:31:8f:35:5a
  inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
  inet 192.168.1.100 netmask 0xffffff00 broadcast 192.168.1.255
  inet 192.168.1.102 netmask 0xffffffff broadcast 192.168.1.102
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (100baseTX <full-duplex>)
  status: active
nfe1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=c219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
  ether 00:17:31:8f:45:60
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (100baseTX <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1492
  options=80000<LINKSTATE>
  inet 172.20.95.87 --> 172.16.13.11 netmask 0xffffffff
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  Opened by PID 464
urtw0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 2290
  ether 00:15:af:04:ad:4c
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
  status: no carrier

netstat -nr

Internet:  
Destination  Gateway  Flags  Netif Expire  
default  172.16.13.11  UGS  tun0  
127.0.0.1  link#3  UH  lo0  
172.16.13.11  link#4  UHS  tun0  
172.20.95.87  link#4  UHS  lo0  
192.168.1.0/24  link#1  U  nfe0  
192.168.1.2  link#1  UHS  lo0  
192.168.1.100  link#1  UHS  lo0  
192.168.1.102  link#1  UHS  lo0  
192.168.1.102/32  link#1  U  nfe0  
  
Internet6:  
Destination  Gateway  Flags  Netif Expire  
::/96  ::1  UGRS  lo0  
::1  link#3  UH  lo0
::ffff:0.0.0.0/96  ::1  UGRS  lo0
fe80::/10  ::1  UGRS  lo0
fe80::%lo0/64  link#3  U  lo0
fe80::1%lo0  link#3  UHS  lo0
ff01::%lo0/32  ::1  U  lo0
ff02::/16  ::1  UGRS  lo0
ff02::%lo0/32  ::1  U  lo0

And finally my rc.conf

# Auto-Enabled NICs from pc-sysinstall
ifconfig_nfe0="inet 192.168.1.2 netmask 255.255.255.0"
# defaultrouter="192.168.1.254"
ifconfig_nfe1="up"
hostname="testing"
pcsysconfig_enable="YES"
zfs_enable="YES"

# Disable sendmail
sendmail_enable="NO"

dbus_enable="YES"
hald_enable="YES"

# Start iocage jails
iocage_enable="YES"

# Start sshd server
sshd_enable="YES"

# Enable fail2ban
#fail2ban_enable="YES"

# Enable IPFW
firewall_enable="YES"
firewall_logging="YES"
#firewall_script="/etc/ipfw.rules"
firewall_type="workstation"
gateway_enable="YES"

#dhcpd_enable="YES"
#dhcpd_ifaces="nfe0"

ppp_enable="YES"
ppp_user="root"
ppp_nat="NO"
ppp_profile="plusnet"
ppp_plusnet_mode="ddial"
ppp_plusnet_nat="NO"

 

[wblock@]

inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
inet 192.168.1.100 netmask 0xffffff00 broadcast 192.168.1.255
inet 192.168.1.102 netmask 0xffffffff broadcast 192.168.1.102

That doesn't look right, although it has been a long time since I used PPP.

pcsysconfig_enable="YES"

Does that mean you are running PC-BSD?

 

[brunoschwartz]

Hello wblock@, yes it is TrueOS.

The lines you are questioning are regarding different interface, which is set up for local LAN (nfe0), one physical and two aliases for jails. The one I would like to use with ppp is nfe1. Why do you think it doesn't look right?

 

[wblock@]

The thing is that PC-BSD or TrueOS are not just FreeBSD. They are FreeBSD with additional customizations. Because of that, plain FreeBSD solutions might not work on them. For that reason, we always ask that people identify when they are using differently-packaged FreeBSD versions, and to first ask for help in the support forums for those systems. It's not that we don't want to help, it's just that most of the people here use unmodified FreeBSD.

The broadcast addresses for my jails are the same as the jail address. I don't know if that matters.

 

[Chris_H]

wblock@ makes some very good points.
The only thing I might like to add; is that it will likely be a great deal easier for you to diagnose your problem, if you reduce the variables involved. Meaning; you should turn off any services that are not absolutely necessary to get connected to the internet -- your jails, for eample. Then, only after figuring out how to get connected to the internet, will you start enabling those services. This, if nothing else will help eliminate any potential IP/routing issues that may be standing in the way of your getting connected via PPP. Especially given that I see that you pass the CHAP challenge, but fail to get an address to connect.

HTH, and all the best.

--Chris

 

[mickey]

Sorry that I have to ask this, but what exactly is the problem?
Looking at the output that you have posted, things seem to be working as intended:

tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1492
  options=80000<LINKSTATE>
  inet 172.20.95.87 --> 172.16.13.11 netmask 0xffffffff
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  Opened by PID 464

There is your point-to-point interface which has been setup by the ppp process 464.
It has been assigned the IP address 172.20.95.87 and connects to your ISP's gateway which has the IP address 172.16.13.11.

Internet:
Destination  Gateway  Flags  Netif Expire
default  172.16.13.11  UGS  tun0

And there is the default route that has been set up to route traffic to your ISP's gateway via interface tun0.

Regarding your ppp.conf I would suggest you add the following:

set mtu 1492
set mru 1492
set timeout 0
enable mssfixup

And change the ifaddr line, so that it reads:

set ifaddr 0 0 255.255.255.255

Does your firewall configuration allow traffic to pass on interface tun0 and does it perform network address translation?

Some things to keep in mind:

• The ethernet interface (nfe1) is used only to transmit encapsulated PPP frames and possibly to access your DSL hardware's management interface (TELNET/WEB). It does not connect to the internet, that is what the tun0 interface does. The nfe1 interface should be assigned a reserved IP address from the same network that your DSL hardware is configured to use.
  
• You will need to perform network address translation somewhere, unless you have a routed IP network. PPP can perform NAT on it's own (deprecated) or your firewall needs to take care of that (recommended).
• Your firewall rules need to be properly setup to allow communication with the internet over the tun0 interface, and to block all unwanted traffic (especially inbound).
• If you want to be able to access your DSL hardware's management interface(s) and/or have your DSL hardware use services from your FreeBSD machine (like time, daytime, ntp, syslog, etc), then your firewall also needs to allow for this traffic on the nfe1 interface.

 

[brunoschwartz]

Thanks Chris, I have now turned off all the services leaving the bare minimum.

 

[brunoschwartz]

Hello Mickey,

Thank you for your suggestions. I am out and about for these two days so won't have a chance to try them straight away. However, I have some questions regarding what you said.

There is your point-to-point interface which has been setup by the ppp process 464. It has been assigned the IP address 172.20.95.87 and connects to your ISP's gateway which has the IP address 172.16.13.11.

It is the tun0 which was given 172.20.95.87 address!
nfe1 (the external interface) has not been assigned any address. I am expecting the nfe1 to be given a public IP address through DHCP. I used rp-ppoe under debian in the past and that was the case. Is it different? Does that mean that the freebsd machine will never be given a public IP at all?

And there is the default route that has been set up to route traffic to your ISP's gateway via interface tun0.

How do I set it up?

Does your firewall configuration allow traffic to pass on interface tun0 and does it perform >network address translation?

No I haven't done that. I read all the available guidance on ppp and it wasn't mentioned anywhere. Eventually I will have a small LAN the freebsd machine will perform NATing for, but I was only testing it without it. Does that mean if you only connect a single machine through pppoe, you still have to perform firewall NAT for it to work?
Please also see my point below.

• The ethernet interface (nfe1) is used only to transmit encapsulated PPP frames and possibly to access your DSL hardware's management interface (TELNET/WEB). It does not connect to the internet, that is what the tun0 interface does. The nfe1 interface should be assigned a reserved IP address from the same network that your DSL hardware is configured to use.

Thanks for pointing this out. I wasn't aware of that.
The machine is connected directly to a FTTC fiber modem, there is nothing that needs managing there.
As I wrote above, the nfe1 hasn't been assigned any address, is your "set ifaddr....." command you suggested meant to achieve that? Why did it not happen?

• You will need to perform network address translation somewhere, unless you have a routed IP network. PPP can perform NAT on it's own (deprecated) or your firewall needs to take care of that (recommended).

• Your firewall rules need to be properly setup to allow communication with the internet over the tun0 interface, and to block all unwanted traffic (especially inbound).

Yes, I will do it through ipfw. I am planning to use this setup as an inspiration

https://forums.freebsd.org/threads/using-trueos-as-a-ipfw-based-home-router.49949/

I presume if I just change the external interface to tun0, things should work. Am I right? Is there any other command neede to NAT from nfe1 to tun0 like you wrote above:

• If you want to be able to access your DSL hardware's management interface(s) and/or have your DSL hardware use services from your FreeBSD machine (like time, daytime, ntp, syslog, etc), then your firewall also needs to allow for this traffic on the nfe1 interface.

I don't understand, why does it have to be on nfe1 rather than tun0?

Thanks for your help again, I spend a lot of time on it without making any progress. My next step was to install rp-pppoe to see if it works the same like in Debian.

B

 

[mickey]

It is the tun0 which was given 172.20.95.87 address!
nfe1 (the external interface) has not been assigned any address. I am expecting the nfe1 to be given a public IP address through DHCP. I used rp-ppoe under debian in the past and that was the case. Is it different? Does that mean that the freebsd machine will never be given a public IP at all?

Your external interface is tun0 not nfe1!
The nfe1 interface is solely used for transmission of PPP frames encapsulated in ethernet frames. There is not even IP involved here, so the IP address you assign to the nfe1 interface is pretty much irrelevant for the purpose of PPPoE. The IP address on nfe1 does however matter if/when you want to access the management interface of your modem and/or have the modem use services provided by your FreeBSD machine. Also there is no DHCP involved here at all, as the assignment of your external IP address is completely handled by ppp's LCP protocol. So when ppp successfully establishes a connection, it will have done a number of things already:

1. Create the tunX point-to-point interface with the IP address assigned by your ISP and the gateway address as the remote peer.
2. Negotiate DNS server address(es) and update your /etc/resolv.conf file (unless you configure ppp not to).
3. Create a default route that points to your ISP's peer address.

No I haven't done that. I read all the available guidance on ppp and it wasn't mentioned anywhere. Eventually I will have a small LAN the freebsd machine will perform NATing for, but I was only testing it without it. Does that mean if you only connect a single machine through pppoe, you still have to perform firewall NAT for it to work?

While it might not strictly be necessary for a single machine, it should not hurt either, as NAT should normally only translate traffic originating from reserved IP addresses and passing to the internet. And if you already plan for a small LAN, the requirement is foreseeable.

The machine is connected directly to a FTTC fiber modem, there is nothing that needs managing there.
As I wrote above, the nfe1 hasn't been assigned any address, is your "set ifaddr....." command you suggested meant to achieve that? Why did it not happen?

The set ifaddr line has nothing to do with the address on your nfe1 interface. It is used when the LCP protocol negotiates IP address(es) with your ISP. Basically it says: "I have no idea what my IP address or the peer's address will be and I will accept any IP address that my ISP gives me and I will accept any IP address for the connection's peer."

Yes, I will do it through ipfw. I am planning to use this setup as an inspiration

https://forums.freebsd.org/threads/using-trueos-as-a-ipfw-based-home-router.49949/

I presume if I just change the external interface to tun0, things should work. Am I right? Is there any other command neede to NAT from nfe1 to tun0 like you wrote above:

My knowledge of ipfw is a bit rusty as I have switched over to pf years ago, but i guess it should be able to get the job done. There is however nothing NATed from nfe1 to tun0. NAT happens only on the tun0 interface.

• If you want to be able to access your DSL hardware's management interface(s) and/or have your DSL hardware use services from your FreeBSD machine (like time, daytime, ntp, syslog, etc), then your firewall also needs to allow for this traffic on the nfe1 interface.

I don't understand, why does it have to be on nfe1 rather than tun0?

To clarify this, there are really only three scenarios where you will ever have to worry about the nfe1 interface:

1. In your ppp.conf file, to tell ppp which interface to transmit/receive PPPoE frames on.
2. If you want to access your modem's management interface (WEB/TELNET/SNMP/etc)
3. If you want your modem to use services provided by your FreeBSD machine (syslog, daytime, time, ntp, etc).

For all other intents and purposes the tun0 interface is your external interface which will get the external IP address and connects you to the internet.

 

[brunoschwartz]

Hello Mickey,

Thanks for your explanation.

I will try things out again tonight.

There are 3 more things I would like to ask:
1) my external interface nfe1 is not getting any address assigned. I though it should be automatic. How can I give it an IP address (as you said really any address is OK, e.g. 192.168.1.1?)
2) I still don't understand the need for NATing from nfe1 to tun0. The firewall example I mentioned above does not have anything in this respect. Do I need to add it? If so, what would the command be? Or will the ppp ensure those two interfaces crosscommunicate?
3) Finally the biggest sticking point, the ppp ends with error like so

Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Warning: 0.0.0.0: Change route failed: errno: No such process
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: bundle: Terminate
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: Received NGM_PPPOE_CLOSE
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Device disconnected
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Disconnected!
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: lcp -> logout
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Disconnected!

You are saying everything is working as intended, why is this happening then? Even if I sort out steps 1 and 2 above, it will still now work because of the error.

Thanks again,
B

 

[mickey]

There are 3 more things I would like to ask:
1) my external interface nfe1 is not getting any address assigned. I though it should be automatic. How can I give it an IP address (as you said really any address is OK, e.g. 192.168.1.1?)

Please stop calling nfe1 the external interace. It is a transfer interface, nothing more. It transfers PPP frames encapsulated in ethernet frames from your FreeBSD machine to your DSL modem and vice versa. The decoded traffic is then seen on the tun0 interface, so this your external interface.

To assign an address to the nfe1 interface change this line in your /etc/rc.conf:

ifconfig_nfe1="up"

like so:

ifconfig_nfe1="inet 192.168.2.1/24 up"

This will assign the address 192.168.2.1 and netmask 255.255.255.0 to the nfe1 interface upon boot time.

2) I still don't understand the need for NATing from nfe1 to tun0. The firewall example I mentioned above does not have anything in this respect. Do I need to add it? If so, what would the command be? Or will the ppp ensure those two interfaces crosscommunicate?

Again, there is no NAT from nfe1 to tun0.
NAT is done on traffic passing in/out your tun0 (external) interface.

The ipfw configuration you mentioned does NAT.
It contains the configuration file for natd (/etc/natd.conf) and diverts traffic to natd by means of the rules 00100 and 600.

The fact that ipfw uses an external program ( natd) to perform NAT does not exactly contribute to the simplicity or performance of such a setup. If you don't have any prior knowledge or experience with either ipfw or pf, I would strongly suggest you take a look at some working pf examples. You might find that pf is much easier to setup and use in some regards. Setting up a working NAT in pf usually requires only one line of configuration and does not use any external programs (which need to be configured seperately).

3) Finally the biggest sticking point, the ppp ends with error like so

Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Warning: 0.0.0.0: Change route failed: errno: No such process
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: bundle: Terminate
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: Received NGM_PPPOE_CLOSE
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Device disconnected
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Disconnected!
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: lcp -> logout
Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Phase: deflink: Disconnected!

You are saying everything is working as intended, why is this happening then? Even if I sort out steps 1 and 2 above, it will still now work because of the error.

You are referring to this?

Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Warning: 0.0.0.0: Change route failed: errno: No such process

First off, that is a warning, not an error. It has been in the code for ages now and I guess almost anyone dealing with ppp has come across this or similiar warning messages, including myself:

ppp[799]: tun0: Warning: 0.0.0.0: Change route failed: errno: Value too large to be stored in data type

I have spent quite some time researching this issue, but the only answer I could come up with so far is that it can be safely ignored.

The other messages are just the PPP link being disconnected, which could happen cause there was no traffic flowing for a period of time. This can be fixed by disabling the idle timeout for the connection by adding the following to your ppp.conf file:

set timeout 0

 

[brunoschwartz]

Hello Mickey,

Thank you for your response. I have just spend 5 hours playing around with this issue. Sadly, it remains unresolved.

I have incorporated all the things you suggested:

1) I added the lines you suggested to the ppd conf - the result is same as in my original post, the ppp link gets established and correct gateway is setup when viewing netstat -nr .

2) initialised nfe1 interface with 192.168.2.1 address

3) I set up natd for doing nat on tun0 interface. It seems to be working correctly as 8668 interace gets set up.

4) then I used firewall rules as in the example I quoted above.
I also tested a very simple set of firewall rules

ipfw -f flush
ipfw add divert natd all from any    to any via tun0
ipfw add pass all    from any to any

To conclude, I can ping the IP address of tun0 interface (i.e. my side of ppp link) but can't ping the gateway IP (ISP side of the ppp link).

I would strongly suggest you take a look at some working pf examples. You might find that pf is much easier to setup and use in some regards. Setting up a working NAT in pf usually requires only one line of configuration and does not use any external programs (which need to be configured seperately).

IPFW can do kernel NAT as well, I tested it as well with the same result.

I am reluctant to switch to pf now as I setup a server with several iocage jails, fail2ban etc....with ipfw. The last step was to get it online as I was doing it all on private LAN. It all seems to be working perfectly, besides the bloody pppoe that is . Being new to bsd, the learning curve was steep and my time is sparce.

You are referring to this?

Jan 18 21:17:14 testingbungalow ppp[534]: tun0: Warning: 0.0.0.0: Change route failed: errno: No such process

First off, that is a warning, not an error. It has been in the code for ages now and I guess almost anyone dealing with ppp has come across this or similiar warning messages, including myself:

Yes, I was referring to that. Thanks for clarifying.

What can I do from now to get it sorted? (Apart from going to back to debian preferably ;-).

BW,

B

PS: One matter that perplexes me. It should be possible to connect via pppoe from the client machine even without having any firewall or NATing. Is my ppp setup a problem rather that firewall?

PS2: here is the output of ipfw show

00020  0  0 allow ip from any to any via nfe0
00025  0  0 allow ip from any to any via lo0
00100  44 3102 divert 8668 ip from any to any in via tun0
00105  0  0 check-state
00110  0  0 skipto 600 tcp from any to any out via tun0 setup keep-state
00120  83 5784 skipto 600 udp from any to any out via tun0 keep-state
00300  0  0 deny ip from 192.168.0.0/16 to any in via tun0
00301  0  0 deny ip from 172.16.0.0/12 to any in via tun0  
00302  0  0 deny ip from 10.0.0.0/8 to any in via tun0  
00303  0  0 deny ip from 127.0.0.0/8 to any in via tun0  
00304  3  108 deny ip from 0.0.0.0/8 to any in via tun0  
00305  0  0 deny ip from 169.254.0.0/16 to any in via tun0  
00306  0  0 deny ip from 192.0.2.0/24 to any in via tun0  
00307  0  0 deny ip from 204.152.64.0/23 to any in via tun0  
00308  0  0 deny ip from 224.0.0.0/3 to any in via tun0  
00310  0  0 deny icmp from any to any in via tun0  
00550  0  0 deny tcp from any to any via tun0  
00551  0  0 deny udp from any to any via tun0  
00600  49 3378 divert 8668 ip from any to any out via tun0  
00610 110 7860 allow ip from any to any  
65535  0  0 allow ip from any to any

 

[mickey]

Thank you for your response. I have just spend 5 hours playing around with this issue. Sadly, it remains unresolved.

I have incorporated all the things you suggested:

1) I added the lines you suggested to the ppd conf - the result is same as in my original post, the ppp link gets established and correct gateway is setup when viewing netstat -nr .

2) initialised nfe1 interface with 192.168.2.1 address

3) I set up natd for doing nat on tun0 interface. It seems to be working correctly as 8668 interace gets set up.

4) then I used firewall rules as in the example I quoted above.
I also tested a very simple set of firewall rules

ipfw -f flush
ipfw add divert natd all from any    to any via tun0
ipfw add pass all    from any to any

To conclude, I can ping the IP address of tun0 interface (i.e. my side of ppp link) but can't ping the gateway IP (ISP side of the ppp link).

It would help if you could post the following again, now that you made modifications:
/etc/ppp/ppp.conf
/etc/rc.conf
And the output of these commands, while the ppp connection is up:
ifconfig -a
netstat -rn

Does the connection now stay up, or is it still being dropped after a couple of minutes?

IPFW can do kernel NAT as well, I tested it as well with the same result.

I am reluctant to switch to pf now as I setup a server with several iocage jails, fail2ban etc....with ipfw. The last step was to get it online as I was doing it all on private LAN. It all seems to be working perfectly, besides the bloody pppoe that is . Being new to bsd, the learning curve was steep and my time is sparce.

I can understand that. And firewall configuration also has much to do with personal preference I guess. If you feel comfortable with ipfw and have got some understanding of how it works, then stick with it, at least for the time being. I have used it for years, but now that I made the step to use pf I never looked back.

What can I do from now to get it sorted? (Apart from going to back to debian preferably ;-).

PS: One matter that perplexes me. It should be possible to connect via pppoe from the client machine even without having any firewall or NATing. Is my ppp setup a problem rather that firewall?

Yes, without NAT and without the firewall (or just one rule that passes all traffic) you should be able to access the internet from your FreeBSD machine. You could try running a traceroute -n command on an IP address you know exists out there.

PS2: here is the output of ipfw show

00020  0  0 allow ip from any to any via nfe0
00025  0  0 allow ip from any to any via lo0
00100  44 3102 divert 8668 ip from any to any in via tun0
00105  0  0 check-state
00110  0  0 skipto 600 tcp from any to any out via tun0 setup keep-state
00120  83 5784 skipto 600 udp from any to any out via tun0 keep-state
00300  0  0 deny ip from 192.168.0.0/16 to any in via tun0
00301  0  0 deny ip from 172.16.0.0/12 to any in via tun0
00302  0  0 deny ip from 10.0.0.0/8 to any in via tun0
00303  0  0 deny ip from 127.0.0.0/8 to any in via tun0
00304  3  108 deny ip from 0.0.0.0/8 to any in via tun0
00305  0  0 deny ip from 169.254.0.0/16 to any in via tun0
00306  0  0 deny ip from 192.0.2.0/24 to any in via tun0
00307  0  0 deny ip from 204.152.64.0/23 to any in via tun0
00308  0  0 deny ip from 224.0.0.0/3 to any in via tun0
00310  0  0 deny icmp from any to any in via tun0
00550  0  0 deny tcp from any to any via tun0
00551  0  0 deny udp from any to any via tun0
00600  49 3378 divert 8668 ip from any to any out via tun0
00610 110 7860 allow ip from any to any
65535  0  0 allow ip from any to any

I see a couple of problems in these rules:

• You are denying inbound traffic on tun0 destined for 192.168.0.0/16, but the firewall sees the traffic after it passes through NAT. This will effectively deny all access to your LAN IP range.
• I don't think that denying traffic to 0.0.0.0 is a good idea, as that would be INADDR_ANY.
• Rule 310 denies ICMP traffic to pass in which breaks ping and friends.

 

[brunoschwartz]

Hello Mickey,

I am coming with some good news. It is all working now.

You wouldn't believe what the problem was.

I could not understand why the tun0 gets assigned private ip and the gateway was private ip as well. Out of desperation I logged into my ISP account on the web with username:xxxxx and password:yyyyy (same I used for login into the pppoe). looking for gateway settings. Once logged in I noticed that my account name was shown as xxxxx@plusdsl.net rather then xxxxx only. I put that into the ppp.conf and whoaaa. It suddenly started to work. On ifconfig I got public proper ip as well.

For completion if you could have a one more look on my ppp.conf (version n.1254)

 default:
  set log Phase Chat tun Command Connect Filter Error Alert

plusnet:
  set device PPPoE:nfe1 # replace xl1 with your Ethernet device
  set ifaddr 0 0 255.255.255.255
  set speed sync
  set mru 1492
  set mtu 1492
  set timeout 0
  set ctsrts off
  enable echo
  set echoperiod 15
  enable lqr
  set lqrperiod 15
  enable ipcp
  enable dns
  disable ipv6cp
  enable mssfixup
  set dial  #new
  set login  #new
  set server /var/run/internet "" 0177
  set authname xxxx@plusdsl.net
  set authkey yyyyy
  set server open #new
  add! default HISADDR

You were right about the firewall. When I turn it on, nothing can go out. But the ppp is sorted and that's the main thing for now.
I tried to address the points you raised regarding the firewall to no avail. It is very late though and I have a very busy day tmrwtomorrow, will have to wait.

I am attaching also my ipfw rules here, if you could suggest some change. Don't want to take too much of your time, grateful for what you have spent till now.

#!/bin/sh
# Flush out the list before we begin.
ipfw -q -f flush

# Set rules command prefix
cmd=”ipfw add”
pif=”tun0″ # interface name of NIC attached to Internet
iif=”nfe0″ # interface name of NIC attached to LAN
ks=”keep-state"
skip=”skipto 600″

# Allow everything through the local NIC
######################################
$cmd 00020 allow all from any to any via $iif

######################################
# No restrictions on Loopback Interface
######################################
$cmd 00025 allow all from any to any via lo0

# NAT the inbound stuff
######################################
$cmd 00100 divert natd ip from any to any in via $pif # NAT any inbound packets

######################################
# Allow packet through if it matches existing entry in dynamic rules
######################################
$cmd 00105 check-state

######################################
# Allow all outgoing packets
######################################
$cmd 00110 $skip all from any to any out via $pif $ks

# Deny all inbound traffic from non-routable reserved address spaces
######################################
$cmd 00300 deny all from 192.168.0.0/16 to any in via $pif #RFC 1918 private IP
$cmd 00301 deny all from 172.16.0.0/12 to any in via $pif #RFC 1918 private IP
$cmd 00302 deny all from 10.0.0.0/8 to any in via $pif #RFC 1918 private IP
$cmd 00303 deny all from 127.0.0.0/8 to any in via $pif #loopback
$cmd 00304 deny all from 0.0.0.0/8 to any in via $pif #loopback
$cmd 00305 deny all from 169.254.0.0/16 to any in via $pif #DHCP auto-config
$cmd 00306 deny all from 192.0.2.0/24 to any in via $pif #reserved for doc’s
$cmd 00307 deny all from 204.152.64.0/23 to any in via $pif #Sun cluster interconnect
$cmd 00308 deny all from 224.0.0.0/3 to any in via $pif #Class D & E multicast
######################################

# Deny public pings
######################################
$cmd 00310 deny icmp from any to any in via $pif

# Log all the other troublemakers
$cmd 00550 deny log all from any to any

# Skip location for NATD
$cmd 600 divert natd ip from any to any out via $pif # skipto location for outbound stateful rules
$cmd 610 allow ip from any to any

Once again, thanks for your help.

BW,
B

 

[pkubaj]

I guess the config you found and use is the one I got in one of the topics here almost 6 years ago It's quite strange how rarely people use PPPoE that after 6 years it's still the only example apart from the one from handbook (which doesn't work for me).

Lately, I had to use PPPoE again and unfortunately, this config didn't work. However, I found that net/mpd5 works great. It actually seems to be only a set of configs, so one should be able to extract the relevant part and copy it to /etc/ppp/ppp.conf, although I haven't tried that myself.

 

[mickey]

I am coming with some good news. It is all working now.

You wouldn't believe what the problem was.

I could not understand why the tun0 gets assigned private ip and the gateway was private ip as well. Out of desperation I logged into my ISP account on the web with username:xxxxx and password:yyyyy (same I used for login into the pppoe). looking for gateway settings. Once logged in I noticed that my account name was shown as xxxxx@plusdsl.net rather then xxxxx only. I put that into the ppp.conf and whoaaa. It suddenly started to work. On ifconfig I got public proper ip as well.

Glad you got it to work. It's often the small things that make the big difference

I had a look at your ppp.conf. I removed some clutter, which I think shouldn't be necessary and made some small modifications:

• Removed 'set speed' as it should not be necessary for PPPoE links.
• Removed 'set ctsrts off' for the same reason.
• Removed 'enable ipcp' as it is enabled by default anyways.
• Removed 'set server open' as it should not be necessary.
• Added 'nat enable no' to the default section to disable PPP's internal NAT features by default.
  
• Added 'rename plusnet' so the link is renamed from 'deflink' to 'plusnet' (makes the log files a bit prettier)
• Changed 'set server' line to use the directory /var/run/ppp (I think it is there for this very reason), and renamed the socket to 'plusnet' also. So you may use pppctl /var/run/ppp/plusnet
• Collapsed all 'enable' lines into a single line.
• Collapsed all 'disable' lines into a single line.

So here's what I would suggest:

default:
  set log Phase Chat tun Command Connect Filter Error Alert
  nat enable no

plusnet:
  rename plusnet
  set device PPPoE:nfe1
  set ifaddr 0 0 255.255.255.255
  set server /var/run/ppp/plusnet "" 0177
  set authname xxxx@plusdsl.net
  set authkey yyyyy
  set mru 1492
  set mtu 1492
  set timeout 0
  set dial
  set login
  set echoperiod 15
  set lqrperiod 15
  enable dns echo lqr mssfixup
  disable ipv6cp
  add! default HISADDR

You were right about the firewall. When I turn it on, nothing can go out. But the ppp is sorted and that's the main thing for now.

That is indeed a big leap forward from the previous state. Now we do at least know, that the remaining problems are firewall/NAT related and can go about these.

I tried to address the points you raised regarding the firewall to no avail. It is very late though and I have a very busy day tmrw, will have to wait.

I am attaching also my ipfw rules here, if you could suggest some change. Don't want to take too much of your time, grateful for what you have spent till now.

#!/bin/sh
# Flush out the list before we begin.
ipfw -q -f flush

# Set rules command prefix
cmd=”ipfw add”
pif=”tun0″ # interface name of NIC attached to Internet
iif=”nfe0″ # interface name of NIC attached to LAN
ks=”keep-state"
skip=”skipto 600″

# Allow everything through the local NIC
######################################
$cmd 00020 allow all from any to any via $iif

######################################
# No restrictions on Loopback Interface
######################################
$cmd 00025 allow all from any to any via lo0

# NAT the inbound stuff
######################################
$cmd 00100 divert natd ip from any to any in via $pif # NAT any inbound packets

######################################
# Allow packet through if it matches existing entry in dynamic rules
######################################
$cmd 00105 check-state

######################################
# Allow all outgoing packets
######################################
$cmd 00110 $skip all from any to any out via $pif $ks

# Deny all inbound traffic from non-routable reserved address spaces
######################################
$cmd 00300 deny all from 192.168.0.0/16 to any in via $pif #RFC 1918 private IP
$cmd 00301 deny all from 172.16.0.0/12 to any in via $pif #RFC 1918 private IP
$cmd 00302 deny all from 10.0.0.0/8 to any in via $pif #RFC 1918 private IP
$cmd 00303 deny all from 127.0.0.0/8 to any in via $pif #loopback
$cmd 00304 deny all from 0.0.0.0/8 to any in via $pif #loopback
$cmd 00305 deny all from 169.254.0.0/16 to any in via $pif #DHCP auto-config
$cmd 00306 deny all from 192.0.2.0/24 to any in via $pif #reserved for doc’s
$cmd 00307 deny all from 204.152.64.0/23 to any in via $pif #Sun cluster interconnect
$cmd 00308 deny all from 224.0.0.0/3 to any in via $pif #Class D & E multicast
######################################

# Deny public pings
######################################
$cmd 00310 deny icmp from any to any in via $pif

# Log all the other troublemakers
$cmd 00550 deny log all from any to any

# Skip location for NATD
$cmd 600 divert natd ip from any to any out via $pif # skipto location for outbound stateful rules
$cmd 610 allow ip from any to any

There are some things here whose logic I don't seem to grasp:

• Why have separate rules to NAT inbound and outbound traffic? Normally you would just pass all traffic through NAT that passes through the external interface.
• Why have that skipto-magic for outbound traffic, when all that happens down there is to allow for the traffic to pass? I dont't see that this increases readability or performance in any way.
• There is no need to explicitly deny things (like inbound ICMP ECHO) when it would be denied anyways (unless you want to log specific traffic).

My ipfw skills are a bit rusty, but lets give it a try anyways:

#! /bin/sh

# Define some variables that might come in handy.
cmd=”ipfw”
int_if="nfe0"  # Our LAN interface
ext_if="tun0" # Our external interface facing the internet

# Create a lookup table for IP addresses we want to block externally.
$cmd table 0 flush
$cmd table 0 add 192.168.0.0/16 # RFC 1918 private IP
$cmd table 0 add 172.16.0.0/12 # RFC 1918 private IP
$cmd table 0 add 10.0.0.0/8 # RFC 1918 private IP
$cmd table 0 add 127.0.0.0/8 # Loopback
$cmd table 0 add 169.254.0.0/16 # APIPA auto-config
$cmd table 0 add 192.0.2.0/24 # Reserved for doc’s
$cmd table 0 add 204.152.64.0/23 # Sun cluster interconnect
$cmd table 0 add 224.0.0.0/3 # Class D & E multicast

# Flush the ruleset before we begin.
$cmd -q -f flush

######################
# Loopback Interface #
######################

# No restrictions on Loopback Interface
$cmd add 01000 allow all from any to any via lo0

#################
# LAN Interface #
#################

# No restrictions on LAN interface
$cmd add 02000 allow all from any to any via $int_if

######################
# External Interface #
######################

# Pass all traffic through natd first.
$cmd add 03000 divert natd ip from any to any via $ext_if

# Check state of already established connections.
$cmd add 03010 check-state

### Outbound traffic ###

# Block traffic destined for addresses in our table.
$cmd add 03200 drop ip from any to table(0) out via $ext_if

# Allow all other outbound traffic and record state.
$cmd add 03210 allow ip from any to any out via $ext_if keep-state

# Block and log all other outbound traffic that makes it this far.
$cmd add 3499 drop log all from any to any out via $ext_if

### Inbound traffic ###

# Block traffic coming from addresses in our table.
$cmd add 03500 drop ip from table(0) to any in via $ext_if

# If you want to allow external access to services on your LAN add these here.
# $cmd add 3510 allow ip from any to 192.168.1.100 dst-port 80 setup in via $ext_if keep-state

# Block and log any other inbound traffic that makes it here.
$cmd add 3999 drop log all from any to any in via $ext_if

 

[brunoschwartz]

Hello Mickey,

Thanks for your reply and suggestions again.

The most important thing first......what happened to your avatar? That guy looked exactly like the main hero from wolfenstein I used to play on my intel 80486dx 66mhz 20 or so years ago. I clearly remember that there was only one place where you could see your own face, I think there was a mirror in level 8 or so.

Now the rest.
I had a crazy day at work but my mind was constantly thinking about getting my little server online, haha......one of these days I should get my priorities straight.

I tried your script and it works very well for the main machine. The nating for local LAN is not working. I am sure I can sort it out by RTFM and I am going to do it. May I ask you about couple of clarifications regarding your rules though:

I)why does outbound traffic needs to be blocked for addresses in the table. Is it not only a matter of blocking incoming connection?

II)there is no rule to block incoming pings from outside, yet the box is not responding to pings from the outside. I can ping but can't be pinged, that is exactly how I want it I just want to know how it is achieved.

III) the rule 3510, why is there a private IP in the rule?
Is it OK to use

allow ip from any to any 443 setup in via £ext_if keep state

for example to open https port? Cause that's what I did and I was greated with my webserver running in a jail. Sweet feeling.

PS: As this is a massive offtopic to the original problem, I may take it to a new thread, should we continue.

BW,
B

 

[mickey]

The most important thing first......what happened to your avatar? That guy looked exactly like the main hero from wolfenstein I used to play on my intel 80486dx 66mhz 20 or so years ago. I clearly remember that there was only one place where you could see your own face, I think there was a mirror in level 8 or so.

I had this avatar for so long now, that I decided it was time for a new one. But you are right, that guy was also famous for his appearance in the HUD of the game Doom, where he made funny faces depending on your actions. Now that I think about it, must have been around the same time I first made contact with 386BSD 0.1

I had a crazy day at work but my mind was costantly thinking about getting my little server online, haha......one of these days I should get my priorities straight.

I tried your script and it works very well for the main machine. The nating for local LAN is not working. I am sure I can sort it out by RTFM and I am going to do it.

The ppp.conf or the ipfw script, or both? The NAT problem could also reside in the natd.conf file. To be honest, now that I learned that ipfw seems to support an internal NAT mechanism, I would replace natd with that. Years have passed since I last used natd and I think I remember that it was some strange kind of NAT problem that brought me to pf in the first place.

May I ask you about couple of clarifications regarding your rules though:

I)why does outbound traffic needs to be blocked for addresses in the table. Is it not only a matter of blocking incoming connection?

While not strictly being necessary, I consider it bad practice to let anything go out to the internet that is not supposed to go there. Be it due to misconfiguration or otherwise. Despite that it is also a good example of the usefulness of tables. They provide fast lookup, are reusable in several places and can be modified at run time. As the rule blocks traffic from your LAN to the outside world, it may however be more polite to change the 'drop' to a 'block return', so the sender of the packet gets the message.

II)there is no rule to block incoming pings from outside, yet the box is not responding to pings from the outside. I can ping but can't be pinged, that is exactly how I want it I just want to know how it is achieved.

When you ping a machine on the outside the outbound icmp traffic is allowed by rule 3210 and a state is recorded. This state is checked by rule 3010 and also allows for the return traffic of this 'connection' to pass back in. There is however no rule that would allow for unsolicited pings from the outside and there is also no state recorded, so the inbound icmp packet will reach rule 3999 which denies it. So there is no need to explicitly deny icmp as it is simply not allowed either.

III) the rule 3510, why is there a private IP in the rule?
Is it OK to use


allow ip from any to any 443 setup in via £ext_if keep state


for example to open https port? Cause that's what I did and I was greated with my webserver running in a jail. Sweet feeling.

Rule 3510 was made for the scenario, where another machine on your LAN located behind your router provides a service that shall be accessible to the outside world. This would of course also require a port mapping to be established in your natd.conf, to map a port from your public IP address to an internal LAN IP address. I am assuming the setup for jails would look quite similar, if each jail has it's own (private) IP address, but my knowledge of jails is rather limited.

Now that I see the rule again, it should of course specify the correct protocol (tcp/udp) instead of just 'ip'. The important part is that it matches the first packet of an inbound 'connection' and records a state, so that consecutive packets are then again matched by 'check-state'. For TCP connections this is accomplished by specifiying the 'setup' keyword. UDP which is connection-less by nature just records a state that times out when the 'connection' has not received any further traffic.

Whenever possible (and known) you should specify the destination IP address of your inbound services (remember, packets at this point already passed through NAT, so you will have to match on translated packets, if this makes a difference). For services bound to internal private IP addresses this is easy as the IP address is known and presumably fixed. It's a little more complicated if the service you want to allow access from the outside is bound to your public IP (either explicitly or implicitly by binding to INADDR_ANY). As your public IP address gets assigned by your ISP, it's not exactly fixed. In that case you can use the 'me' keyword in place of the destination address, which at least limits the addresses to those assigned to any interface on the local machine. This is still better than using 'any'.