Solved Help Getting PF to work with my Git Jail

Rand0m

Rand0m

Hi all,
I have an issue with my PF rules and I would like to understand why this is happening and how to solve it. I have very basic knowledge of PF and this is kind of learning curve for me.
I have gitea server https://www.freshports.org/www/gitea/ running inside a jail in a vm. It works perfectly. Yet I decided to close all unnecessary ports and keep only the ones that I run services on so I added block all to my /etc/pf.conf.

I realized since I added that I'm unable to push or fetch from the server and I am getting this error. (git commands work only when I comment it)
Code: 
Gitea: Internal error
UpdatePublicKey: Post https://localhost:2000/api/internal/ssh/5/update: dial tcp 127.0.0.1:2000: connect: permission denied
To ssh://myremoteserver:10000/Base/libdstruct.git
! [remote rejected] v1.1.0-development -> v1.1.0-development (pre-receive hook declined)
error: failed to push some refs to 'ssh://git@myremoteserver:10000/Base/libdstruct.git'
every time I do for example a git pull

A bit more details on my curent setup:
  • Gitea inside the jail runs under port 2000
  • The jail ssh port is 10000 (so I can forward based on ports)
when you access from web browser 443 pf will direct you to 2000 which works perfectly all the time and I can, as below, ssh the git jail even with the block all is on.

Code: 
/home/code/libdstruct >ssh git@myremoteserver -p 10000
Enter passphrase for key '/home/code/.ssh/id_rsa': 
PTY allocation request failed on channel 0
Hi there, You've successfully authenticated, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.
Connection to myremoteserver closed.

I would like some help to understand why block all is blocking the git functionality, although it comes (in oder) after the rdr in the PF. Please find my pf.conf below,

Code: 
#Define the interfaces
ext_if = "vtnet0"
int_if = "lo1"
jail_net = $int_if:network

table <badhosts> persist file "/etc/badips"
table <bruteforce> persist
table <sshguard> persist

#Define the IP address and the ports of jail
git_jail = "192.168.1.1"
git_jail_http_port = "2000"
git_ext_http_port = "{https}"
git_jail_ssh_port = "10000"

# don't filter on the loopback interface
set skip on lo0

# all incoming traffic on external interface is normalized and fragmented
# packets are reassembled.
scrub in on $ext_if all fragment reassemble

# Redirect traffic on port 3000 to the git jail
rdr pass on $ext_if inet proto tcp from { !<badhosts>, !<bruteforce>, !<sshguard> } to any port $git_ext_http_port  -> $git_jail port $git_jail_http_port
rdr pass on $ext_if inet proto tcp from { !<badhosts>, !<bruteforce>, !<sshguard> } to any port $git_jail_ssh_port  -> $git_jail port $git_jail_ssh_port

antispoof for $ext_if
antispoof for $int_if

block all
block quick from <badhosts> to any
block drop in quick on $ext_if inet from <sshguard> to any

block quick from <bruteforce>
pass inet proto tcp from any to $ext_if port 5566 flags  S/SA keep state (max-src-conn 10, max-src-conn-rate 10/5, overload <bruteforce> flush global)
 
abishai said:
You don't have rules for localhost.
Click to expand...
There's a set skip on lo0 in his rule set.

But looking at the error (permission denied) it's not an error you would get if a firewall was blocking things. If the firewall was an issue you typically get 'connection refused' or 'connection time-out' errors. "Permission denied" hints at filesystem access permissions.
 
SirDice said:
There's a set skip on lo0 in his rule set.

But looking at the error (permission denied) it's not an error you would get if a firewall was blocking things. If the firewall was an issue you typically get 'connection refused' or 'connection time-out' errors. "Permission denied" hints at filesystem access permissions.
Click to expand...
It works perfectly when I comment out block all and reload pf rules. And that’s what confuses me.
 
When you use rdr pass all other filter rules are ignored.
 
SirDice said:
When you use rdr pass all other filter rules are ignored.
Click to expand...
true and that is my understanding.
Is there anything I have in my rules that is causing this.
In my rules what is block all doing exactly? and can I write it differently? Like spell it out so I can see what port exactly is causing this!
Is there a way to capture or emulate for a certain call what is passed and what was blocked.
 
blueCub said:
Final question for today, is ipfw better on FreeBSD than PF?
Click to expand...
In some cases, yes. IPFW is the native firewall of FreeBSD, PF was originally imported from OpenBSD. There's been a massive amount of changes to FreeBSD's PF though to make it integrate better. But on certain loads IPFW would perform a little better. For an average server it's not going to matter much which one you pick.

I personally like PF more, the syntax is much easier to understand for me.
 
Dear blueCub,
I think git needs to communicate via port 9418., at least this has been necessary some time ago.... I checked my IPFW config and found
Code: 
 75 # Allow git
 76 # ---------
 77 ipfw -q add 00800 allow tcp from any to any 9418 out via $device setup keep-state
I hope it helps. I guess you can setup something similar using PF.
 
chrbr said:
Dear blueCub,
I think git needs to communicate via port 9418., at least this has been necessary some time ago.... I checked my IPFW config and found
Code: 
75 # Allow git
76 # ---------
77 ipfw -q add 00800 allow tcp from any to any 9418 out via $device setup keep-state
I hope it helps. I guess you can setup something similar using PF.
Click to expand...

Thanks chrbr I will open the port and try. But few things to mention here for you to know from where I’m coming from:
  • The problem occurs only when I have block all which occurs after the redirect
  • I don’t direct traffic to this port inside the jail. And this work when no block all
  • When you use ssh I thought you use the ssh port which is 10000 in my case. Jail Ssh port
Anyway best way is to try and see
 
I think you could also have a problem with the jail itself. In your first post you get an error message ("PTY allocation request failed on channel 0") right after you logged on with SSH. That isn't normal behavior:
Code: 
peter@zefiris:/home/peter $ ssh -p 23 psi
Password:
Last login: Thu Dec 13 21:46:43 2018 from psi.intranet.lan
FreeBSD 11.2-RELEASE-p4 (ZEFIRIS) #7 r340719: Sun Dec  2 11:37:32 CET 2018

Welcome to FreeBSD!
peter@psi:~ %
See? no error messages about PTY allocations, so if this issue also manifests itself with Git then I can well imagine that you're running into weird problems. Leading up to: how did you configure the jail?
 
ShelLuser said:
I think you could also have a problem with the jail itself. In your first post you get an error message ("PTY allocation request failed on channel 0") right after you logged on with SSH. That isn't normal behavior:
Code: 
peter@zefiris:/home/peter $ ssh -p 23 psi
Password:
Last login: Thu Dec 13 21:46:43 2018 from psi.intranet.lan
FreeBSD 11.2-RELEASE-p4 (ZEFIRIS) #7 r340719: Sun Dec  2 11:37:32 CET 2018

Welcome to FreeBSD!
peter@psi:~ %
See? no error messages about PTY allocations, so if this issue also manifests itself with Git then I can well imagine that you're running into weird problems. Leading up to: how did you configure the jail?
Click to expand...

I was only trying to see if the git ssh traffic will be allowed through the port so kind of emulating manually what I think git command will do.
No one in theory is supposed to ssh to that git jail, ssh is configured not to allow root and the only other user is git which is a special user. I am not really concerned about that much.
I use qjail to create and manage jails but that is a separate issue I guess. And I can ssh all my other jails normally (on different VMs and on my lan no problem).

What I want to do is, be able to block all other traffic and keep only git working as supposed.
 
blueCub said:
What I want to do is, be able to block all other traffic and keep only git working as supposed.
Click to expand...
I re-read the original post and your Git repository itself is also having problems:

blueCub said:
Code: 
! [remote rejected] v1.1.0-development -> v1.1.0-development (pre-receive hook declined)
error: failed to push some refs to 'ssh://git@myremoteserver:10000/Base/libdstruct.git'
Click to expand...
Notice 'pre-receive hook declined'? That error is triggered by Git itself and not caused by anything outside the jail, something SirDice also hinted at above.

Also: the 'remote-rejected' error is generally caused by trying to push updates into a working copy instead of a bare repository ( git clone --bare ...). A firewall cannot have any direct influence on that, because if it had then the hook would never have been triggered in the first place.

So if you then also take the PTY error into consideration again then my conclusion is that your jail is misconfigured somehow. Does it have a proper /dev structure by any chance? And what's configured in .git/hooks/pre-receive anyway considering that this hook isn't used by default (for all I know those hooks could also be related to these issues)?

I know you already said that things started working again after you removed the block all rule, but are you sure that you didn't also somehow fix (or change) the repository as well in the meantime? And what exactly do you consider "working"? If you don't plan on using SSH then this is obviously not a good example to demonstrate what went wrong.

What's the error message if you use pull or try to clone the repository? Also: what's the exact command you're using to do that? (edit): And: what error message does that give you?
 
ShelLuser said:
I re-read the original post and your Git repository itself is also having problems:


Notice 'pre-receive hook declined'? That error is triggered by Git itself and not caused by anything outside the jail, something SirDice also hinted at above.

Also: the 'remote-rejected' error is generally caused by trying to push updates into a working copy instead of a bare repository ( git clone --bare ...). A firewall cannot have any direct influence on that, because if it had then the hook would never have been triggered in the first place.

So if you then also take the PTY error into consideration again then my conclusion is that your jail is misconfigured somehow. Does it have a proper /dev structure by any chance? And what's configured in .git/hooks/pre-receive anyway considering that this hook isn't used by default (for all I know those hooks could also be related to these issues)?

I know you already said that things started working again after you removed the block all rule, but are you sure that you didn't also somehow fix (or change) the repository as well in the meantime? And what exactly do you consider "working"? If you don't plan on using SSH then this is obviously not a good example to demonstrate what went wrong.

What's the error message if you use pull or try to clone the repository? Also: what's the exact command you're using to do that? (edit): And: what error message does that give you?
Click to expand...
Thanks for that ShelLuser for coming back with all these details.
What I said is, it always worked without the Block All. I recently decided to add the Block All to close all unwanted ports and I discovered only then that I am unable to push or fetch.
So the current situation is when Block all exists; git doesn't work, when block all is commented out; git works normally from command line.
The gitea itself as web app works fine all the time.
What I should do, I will backup the data (archive the whole existing jail) and create new fresh one and see if I will have the same issue. I will do that tonight after work.

Edit:
Regarding to ShelLuser comments
So if you then also take the PTY error into consideration again then my conclusion is that your jail is misconfigured somehow. Does it have a proper /dev structure by any chance?
Click to expand...
I did the same test at my company git (github running on linux) which is not running on FreeBSD nor in jail, and got the same message exactly, So I don't think the PTY error is due to jail misconfiguration.
 
The mystery has been solved finally, I added pass on the loop back lo1 for jail inter traffic, and everything works fine.

pass on $int_if inet from $git_jail to $git_jail keep state

Thanks for everyone.....
 
You must log in or register to reply here.
Back
Top