Having trouble installing haproxy

[Snurg]

Thank you, I see we have somewhat different approaches.
I must admit that I probably never will learn programming using sh. Even simple conditionals for error handling are so difficult and need so much tedious typing, and the results are so hard to understand. I already have succumbed to my laziness and will stick with Perl. As a good Unixoid FreeBSD has Perl in the base, so I don't see any reason not to use it.

And, what I do not understand yet: Are there other reasons aside of deleting the temp and log files etc, for having extra scripts for starting and stopping jails? I mean, why not just service jail start myjail etc?

On the other hand, it would be probably much better to make sure that all important services get shut down cleanly ( service nginx stop, for example) instead of just killing them. So I should take more care of rc.shutdown than in the past. Thank you for pointing me at that!

And what's really sweet, too, I learned from you how to prevent dhclient from tampering with resolv.conf. That had often annoyed me in the past.

A suggestion though: Removing configuration lines for disabled things like sendmail and inetd makes the config shorter and easier to read.

 

[SirDice]

As a good Unixoid FreeBSD has Perl in the base

Perl has been removed from the base since FreeBSD 5.0. It's often installed though, typically as a dependency of some other port (I'm a heavy Perl user myself ).

 

[max21]

Thank you, I see we have somewhat different approaches.
I must admit that I probably never will learn programming using sh. Even simple conditionals for error handling are so difficult and need so much tedious typing, and the results are so hard to understand. I already have succumbed to my laziness and will stick with Perl. As a good Unixoid FreeBSD has Perl in the base, so I don't see any reason not to use it.

And, what I do not understand yet: Are there other reasons aside of deleting the temp and log files etc, for having extra scripts for starting and stopping jails? I mean, why not just service jail start myjail etc?

On the other hand, it would be probably much better to make sure that all important services get shut down cleanly ( service nginx stop, for example) instead of just killing them. So I should take more care of rc.shutdown than in the past. Thank you for pointing me at that!

And what's really sweet, too, I learned from you how to prevent dhclient from tampering with resolv.conf. That had often annoyed me in the past.

A suggestion though: Removing configuration lines for disabled things like sendmail and inetd makes the config shorter and easier to read.

No brag, but there is nothing I don’t know about what I done or what seems not so wise to do. I just keep stuff separated until I find the time to play with it again. I presented it to you to help provide some ideas for your real HOW-TO. Just dedicate to it for a week or two then all of your question will be answered. Pay attention to my tiny notes. Afterwards you know exactly what you can do with it - - you are not bound by the system anymore - - you got an opportunity to piece it all together to suit your needs, afterwards. For me, I got more testing to do such as delete-unwanted-file down to the bone, that will work with all future versions of FreeBSD. Sure src.conf can do most of that but who want to installworld--jail after each and every failed test. That is only one example. Once I’m done all I need to do is automate these scripts, and if another script can’t run it … C++ will. Right now it’s all about the web for me.

Making everything disappear when I shutdown is one of the many things of what it’s all about. It ensure me that there is nothing left to find and see. And above all it's amazing to have it all come back again like the jail was created for the first time. See my friend, there is a reason for everything here.

jail.conf and rc.conf will NOT complete job, all the time, or ever again after it get locked in by the system by those conf files.

Now you know why people been having jail issues for years … the darn thing got stuck (is my guess)!

For example, use your own [filejail.conf[/file]. You must use ONESTART/ONESTOP because this jail is standalone to the bone. This only happen most of the time, and it can happen at the wrong time as seen below. It happen after the second onestop, then onestart. Now how many people even thought to check? That is what you got to do until you know for sure. That’s why I say dedicate for at least 2 week and keep your eyes open.

Start the jail and shut it down a few times. Afterwards, you will find all of those devices still in there. So if you had 1000 jails turn-off after use, you might have 9000 links to the kernel floating around all over the place, including null-mounts that were once hidden, now wide open. And to make the matter even worse, you got to reboot to recover to test anything jail related again. That is not secure. Whatever it is, I don’t want it, so I start thinking out-side the box a long time ago. I don’t care that it don’t look pretty, lt runs like a plain jail, but I have near TOTAL control, per se.

For example:
This is what jail.conf leaves you:

(~) service jail onestart proxy
Starting jails: proxy.
(~) 
(~) service jail onestop proxy
Stopping jails: proxy.
(~) cd /mnt/z/proxy/dev
(/mnt/z/proxy/dev) ls
fd      null    pts     random  stderr  stdin   stdout  urandom zero
(/mnt/z/proxy/dev)
. . . 
. . . and this is all you get even when it is
. . . generating the ssh_host_keys for the first time.

.  . .NOTE the left-over devices

Now this is what you get with the scripts that I posted:

# ./.s
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
32-bit compatibility ldconfig path: /usr/lib32
Setting hostname: proxy.ka.cell.
Creating and/or trimming log files.
Starting syslogd.
Clearing /tmp (X related).
Generating RSA host key.
2048 SHA256:3RMWM05PiroGyPlFlw+oOrtDlPKEHi+FBg9MGakSamE root@proxy.ka.cell (RSA)
Generating ECDSA host key.
256 SHA256:6noOz+g3jBbWHBNFaWXS5nicu2Om2wEmIhdLUEAg/CE root@proxy.ka.cell (ECDSA)
Generating ED25519 host key.
256 SHA256:noe/Dmt8gK2GedKaMQtBrMWYNcUWNhfJLRdJcFjIzBE root@proxy.ka.cell (ED25519)
Performing sanity check on sshd configuration.
Starting sshd.
Starting cron.

Sat Dec  2 05:59:58 UTC 2017

#
. . .
. . .
I bet more than 99% of all jails users never
saw this before!  It’s running like a real system.

But you got to make one extra call kill it all.
The question is; Is it worth it for the unknowing user?

./.q = shutdown terminal
./.e_kill-environment

Also, don’t forget to use this. I forgot to post it. Now FreeBSD jail-system don’t need to do anything. It’s the kernel job to follow orders:

/root/.cache
setenv	EDITOR	vi
setenv	PAGER	more
setenv	BLOCKSIZE	K

setenv RESORT_0  < -- here
setenv RESORT_1  < -- Now it's buried so
setenv RESORT_2        deep even beastie can't find it

if ($?prompt) then

I don't know what FreeBSD really does, but it's sure fun trying to figure him out



... but, still, trying to explain all the details about this jail all night long done gotten me confused. I’ll post it in the HOW TO section once complete and properly worded (that's the hard part), someday. Thanks for the tip about rc.conf Snurg

Back to solution-1 for me

See ya latter

 

[max21]

....
....
. . . On the other hand, it would be probably much better to make sure that all important services get shut down cleanly ( service nginx stop, for example) instead of just killing them. So I should take more care of rc.shutdown than in the past. Thank you for pointing me at that!

I just realize I did not answer to this seemly kind of a question, As usual, I wondered off into everything else trying to figure out what was actually being ask. Anyway, the reason I kill off most or all applications is because they are only suppose to be jailed processes, virtual-like. However, I do cleanly shutdown any process that may affect the host. “iirc”, I did indicate all of this is work in progress ... and I learn that there is nothing wrong if you kill a process as long as you clean-up behind it then flush-it if you find any leftover and if you do, you'll be glad you found it. My goal was to know for a fact at how far one can go without affecting the host. The only thing I had to leave along was sshguard because of the special function for it is in the host rc.conf. That’s all I needed to know. Now that I have experienced all of this , I might find a reason why to use it in production.

service cron stop
service syslogd stop
service sshguard stop
/usr/bin/pkill sshd
/usr/bin/pkill nginx
/usr/bin/pkill ld-elf
/usr/bin/pkill syslogd
/usr/bin/pkill ld-elf32
/usr/bin/pkill cron
/usr/bin/pkill clean_var
rm -Pr /var/run/*

What surprise me even more was that you can totally wipe out /var/run and all that other stuff and I still never have a problem. And as far as cleaning, It don’t get better then rm Pr, then flushing out the entire environment to close the rest of the shell. The only thing I might save would be the logs. It’s worth it.


Good grief! It’s weird that I now come to realize what you meant by “I see we have somewhat different approaches.” I swear, I really didn’t completely understand. Here you were ready to write about the relationship between jail.conf and the rc.conf and I been not using either for a very long time without knowing it .. with the exception of sshguard, until now. And to speak of missing the mark. Dang! No wonder why I been tripping. For those who think I’m slower than molasses .. I agree, but you got to admit I been doing a little bit better lately.



more about HAProxy:

Can a Reverse Proxy use SNI with SSL pass through?

https://serverfault.com/questions/625362/can-a-reverse-proxy-use-sni-with-ssl-pass-through

 

[Snurg]

I still prefer to shutdown programs the normal way. I don't like, for example, if I cannot clearly determine from the logs whether a program has exited abnormally (problem) or was just killed (no problem) if the program shutdown messages are missing in the logs.

Regarding the idea of a howto, I realize it makes much more sense to do the thing another way around.
Just the sheer quantity of files needing to be touched on the host side only when one creates or deletes a jail: /etc/rc.conf, /etc/jail.conf, /etc/pf.conf, /etc/fstab, occasionally also /etc/hosts, /etc/haproxy.conf.
On the jail side there are also a number of files: /etc/rc.conf, /etc/fstab, optionally also /etc/hosts, /etc/jail.conf.

This is about a dozen files minimum. And then it gets even more complicated. If you do not only want to clone base system jails, but preconfigurable jails for, say, web servers or even web application servers like forum servers.
You then need to handle even more files, depending on the kind of jails you want to clone. If you want to clone, say, applications like BBs that use, say, SQL servers, you'll even need to provide a seamless way to configure the cloned jails from inside. For all that I use a sort of "plugin concept" so one can configure his clone jails using selectable modular sets of config scripts.

And I want all that usable from either fully console interactive-dialog-based to fully automated operation (via scripts), for example to administer the whole stuff via CGI apps some time later (essential if you want to host).

You see, this whole configuration chore is incredibly cumbersome and error-prone. I do not like the idea doing that manually. But to be able to see if my script work correctly, I only use it in simulation mode. So I get a complete list of actions, like a custom-generated shell script, combined with the list of file modifications, that I can check out and refine until I am satisfied, so I can do the first real tests, first in a jail, and finally on my real systems. And when this works, I can release the stuff to the public.

The output also can serve in seeing what all needs to be touched when administrating such stuff. The few insane people like me, you and some others here who want to learn more of the "inner workings" can study these outputs and get the idea. The more normal people, who just want to get it work, just can use the easy-to-use frontends (command line or even easier with CGI). I think this is a better approach.
And if you ask yourself why I do this. Having this done is just a prerequisite for my main project, so I must do that anyway. And why not share it, when it could profit the FreeBSD community)

 

[max21]

I still prefer to shutdown programs the normal way. . . .

How could daemons possibly agree with each other? I guess that is why beastie is in charge.

Your post is very, very informative, and for production, as planned, I will be changing the way I shutdown applications among many other things that you speak of even though there are many other ways to detect how a application or the jail itself was shutdown. I am not anti-jail.conf. I got a jail.conf to run the same jail and I build on that jail.conf for what it can accept so to match the untouchable jail; outside of rc.conf, pf.conf, jail-fstab and sysctl.conf, on the host, as I go. I thought I mention that much earlier. It’s like Double Mint Gum, it has two, two kicks in one.

As far a jails are concern, I’m in no rush. I have all winter to prefect it now that I know what it needs, at the very least to match your requirements. Thanks for all of that. If you do document your HOW TO could you please add some information about what sysctl calls can works inside the jail rc.conf. I already know that a jail would/could use the host sysctl, but my jail might not, however I do know that a jail rc.conf could be used to call some of those sysctl functions. I want to know what are some of the common ones needed, and how to test that it works, regardless. All I get are error warning with sysctl.conf inside a jail with anything in it. From what I gather, jails are required to have /etc/sysctl.conf there or else rc.conf will not work completely and syslogd will not show up inside /var/run. I learn these things only by making everything disappear at shutdown time (total-teardown), then a full-first-run-boot. This could be just because of the way I build jails, but I doubt that. jail.conf prove that they both are of the same family.

On the jail side there are also a number of files: /etc/rc.conf, /etc/fstab, optionally also /etc/hosts, /etc/jail.conf.

I did not know a jail could have a jail.conf. Did you mistakenly include it?

. . . on the host side only when one creates or deletes a jail: /etc/rc.conf, /etc/jail.conf, /etc/pf.conf, /etc/fstab, occasionally also /etc/hosts, /etc/haproxy.conf.

About HAProxy, it seems that you have it installed on the host. Are there any pros and cons vs HAProxy being installed in jail? I been planning to use it while jailed.

Here I found a very important point when it comes to learning HAProxy SNI.

HAProxy HTTPS setups can be a little tricky. So make sure you have a working one first before adding SNI to the mix.

https://stuff-things.net/2016/11/30/haproxy-sni/

 

[Snurg]

did not know a jail could have a jail.conf. Did you mistakenly include it?

I want to explore jail in jail Like the traditional Russian figurines.

About HAProxy, it seems that you have it installed on the host.

No. All what is needed on the host is the base system and Perl. All other stuff jailed.
Atm I have almost finished the basic functions of the script and expect to do the first dry runs today or tomorrow.
So I need only install FreeBSD and Perl, and then run the script in automatic mode to configure the jails system and PF, and set up haproxy and DNS jails for me. Giving me a more preconfigured base ready to start the whole web stuff instantly, without need to do the same configuration chores again.

 

[max21]

I want to explore jail in jail Like the traditional Russian figurines. ...
. ...
No. All what is needed on the host is the base system and Perl. All other stuff jailed.
Atm I have almost finished the basic functions of the script and expect to do the first dry runs today or tomorrow.
So I need only install FreeBSD and Perl, and then run the script in automatic mode to configure the jails system and PF, and set up haproxy and DNS jails for me. Giving me a more preconfigured base ready to start the whole web stuff instantly, without need to do the same configuration chores again.

That did it! I was about to send another gigantic post seeking verification about a complete jail install. … So,all I need to do is include only this part:

After more research, It seem that there are two types of jails.. The technical terms might be jails and jailed-shell. Also, someone at FreeBSD pipermail explained the complete differences, and pounded it in. I wish I could re-find that post but I can’t.

Well, its time to get started

Thanks Snurg

Thanks SirDice

I should be-able to walk right up on it. No time to monk around, I'm going to use my jail.conf and learn some Perl.

Hey, I just thought, it ain't over until the fat lady sing. Can't wait to read more about your project. Good Luck. Need anything done, let me know.

 

[max21]

. . . as long as you have routable public IP space you can easily route it through the firewall and keep the publics right on the proxies.

This is what this thread was supposed to be about before the installation problem bit-me, haproxy_behind_a_firewall. I’m glad that happened.

haproxy-devel-1.8.1.txz
JIT - 2017-Dec-05 19:30
.

 

[max21]

That’s the way it goes:

haproxy-1.7.9
/usr/local/etc/rc.d/haproxy: WARNING: failed precmd routine for haproxy
All previous versions produce the same error, including the port version.

haproxy-devel-1.8.1.txz
/usr/local/sbin/haproxy: Undefined symbol "stat@FBSD_1.5"
/usr/local/etc/rc.d/haproxy: WARNING: failed precmd routine for haproxy


Any ideas at what’s going on?

 

[Snurg]

No idea. I just did pkg -j haproxy <haproxypkg>.
I used the current production (not devel) version I found listed on freshports.
The jail's rc.conf contains these haproxy-related lines:

hostname="haproxy"
haproxy_enable="YES"
haproxy_flags="-f /etc/haproxy.conf -d"

 

[max21]

pkg -j haproxy <haproxypkg> don’t work for me. It’s probably a Perl thing. Even without the <haproxypkg> I thought it should had work but it didn’t. I don’t expect anyone to know, especially about the new version. It’s suppose to be their best version ever but it seem to me they worte haproxy for Linux and screwed it up for FreeBSD. Now we get this ---> … stat@FBSD_1.5 is NO where to be found, absolutely NO-WHERE, and NOTHING no where near about it, either. FreeBSD might have to reverse-engineer it just to make it work… evidently, they didn’t do something right or not document it so that we know what to do.

I’m going to go to my friend home tomorrow, he has DSL. I got Comcast cable and I don’t trust cable like I do DSL. For years when I clicked the Thank you button, I get a BIG popup WARNING: you’re about to … do you really want to do that …. Heck no, so I shut it down.

But when I just did that for you guys, I did not care anymore, and when I clicked the Thank you button, both times, for the very first time … there was not pop-up’s. So, this really make me thing that I was right that there is a man-in-the-middle and he knows this is the only place that I go (a member of) so it’s automatic I guess. My XP vbox can’t be hacked anymore, but whoever got in already, at least he know he can only go so far. He use to could turn-off my firewall until I got wise. I saw how Windows svchost.exe was being use to hack the heck out of me. It was so funny when I caught it (like peep-ka-boo). I’m not going to TRIP.

SVN could be messed up on my FreeBSD vbox. I’m thinking of doing a standard install. I forgot about this link **sniproxy**. I'm still glad.. if I did not learn all the above, I would be wasting months to years knowing nothing, once installed. Just like VPN client, over 3 months, off and on, down the drain, trying.

https://serverfault.com/questions/625362/can-a-reverse-proxy-use-sni-with-ssl-pass-through

# pkg install haproxy

Updating FreeBSD repository catalogue...
….
….
New packages to be INSTALLED:

          haproxy: 1.7.9
…
…
[haproxy.ka.cell] Extracting haproxy-1.7.9: 100%
 
# service haproxy start

/usr/local/etc/rc.d/haproxy: WARNING: failed precmd routine for haproxy

#

 

[max21]

sniproxy don't seem right. It might work but haproxy might too if I got to go through all of that. Looks like I'm going to have to take it to the top starting at the bottom of this page and learn something.

https://www.haproxy.org/download/1.6/doc/management.txt

 

[max21]

SVN forgot something, or buildworld skipped it, or my make.conf has too much clang.. or the system burped.

https://www.haproxy.org/download/1.6/doc/management.txt

HAProxy is designed to isolate itself into a chroot jail during startup, where it cannot perform any file-system access at all. This is also true for the libraries it depends on (eg: libc, libssl, etc).

No wonder nothing important work out the box since 10.2p7. If I stuck with ports I would had detected something by now. I wonder would installing something like devel/libcxxrt do the trick. Maybe it was the cause of the debug code not working. Now I need to find out all of haproxy eg: libc, libssl, etc - etc -etc. I show like to know..... I’ll never delete-old-lib ever again!


buildworld

===> lib/libc (install)

install  -C -o root -g wheel -m 444   libc.a /mnt/z/proxy/usr/lib/
install  -C -o root -g wheel -m 444   libc_p.a /mnt/z/proxy/usr/lib/
install: libc_p.a: No such file or directory

*** Error code 71

Stop.

make[5]: stopped in /usr/src/lib/libc

*** Error code 1

make[4]: stopped in /usr/src/lib

make[3]: stopped in /usr/src

make[2]: stopped in /usr/src

make[1]: stopped in /usr/src

make: stopped in /usr/src

*** Creating the temporary root environment in /var/tmp/temproot

/etc/make.conf

 # ...........................................      make.conf
MAKE_JOBS_NUMBER?=8
CPUTYPE?=native
OPTIONS_SET=OPTIMIZED_CFLAGS
WITH_CLANG=yes
WITH_CLANG_FULL=yes
WITH_CLANG_EXTRA=yes
WITH_CLANG_IS_CC=yes

cc=/usr/local/libexec/ccache.world/clang
cc=/usr/local/libexec/ccache.world/clang++
cc=/usr/local/libexec/ccache.world/clang-cpp

BUILD_STATIC=1

DEFAULT_VERSIONS+=ssl=libressl
DEFAULT_VERSIONS+=php=7.1

WRKDIRPREFIX=/ram

Time to find libc, rebuild the system, or downgrade. Better to find out now then never

 

[SirDice]

Remove everything except DEFAULT_VERSIONS and WRKDIRPREFIX. Then try again.

 

[max21]

I followed your instrustions to a tee. Same result. I even deleted everything using rm –Pr and chflags command - - /ram, /usr/obj, /usr/src and /usr/ports, including old files under /var/cache/pkg and /var/db/pkg and ports. I downloaded the src and ports for FreeBSD-11.1 and did a complete buildworld. I delete-old-files and I delete-old-lib. At least I know I got a perfect FreeBSD-11.1 install:

First try fail:

cd /usr/ports/net/haproxy
make –DBATCH install clean
Errors:
libiconv-1.14_11
/usr/local/sbin/pkg-static: not found
make[2]: stopped in /usr/ports/converters/libiconv
make[1]: stopped in /usr/ports/devel/gmake
make: stopped in /usr/ports/net/haproxy

So I installed the pkg versions of pkg, libiconv and gmake so to let ports knows that something is there. Interesting enough, it cleared up that situation. I experienced this before already.

pkg install pkg
pkg install libiconv
pkg install gmake

Then I tried to reinstall the port version of haproxy and this is what I got:

cd /usr/ports/net/haproxy
make –DBATCH install clean
…
                               ^~~~~~~~~~~~~~~~~~
59 warnings generated.
gmake[1]: Leaving directory '/ram/usr/ports/net/haproxy/work/haproxy-1.7.9/contrib/halog'
===>  Staging for haproxy-1.7.9
===>   haproxy-1.7.9 depends on file: /usr/local/lib/libcrypto.so.42 - found
===>   Generating temporary packing list

http://www.haproxy.org
===>  Cleaning for gmake-4.2.1_1
===>  Cleaning for libiconv-1.14_11
===>  Cleaning for libressl-2.6.3
===>  Cleaning for pkgconf-1.3.10,1
===>  Cleaning for haproxy-1.7.9
(/usr/ports/net/haproxy)

see how it took.

TEST IT:

 (~) service haproxy onestart
/usr/local/etc/rc.d/haproxy: WARNING: /usr/local/etc/haproxy.conf is not readable.
/usr/local/etc/rc.d/haproxy: WARNING: failed precmd routine for haproxy
(~)
(~)
(~) service haproxy onestart
/usr/local/etc/rc.d/haproxy: WARNING: failed precmd routine for haproxy
(~)

Lets see if haproxy is in there:

 (~) haproxy -vv
HA-Proxy version 1.7.9 2017/08/18
Copyright 2000-2017 Willy Tarreau <willy@haproxy.org>

Build options :
  TARGET  = freebsd
  CPU     = generic
  CC      = cc
  CFLAGS  = -O2 -pipe -fstack-protector -fno-strict-aliasing -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -DFREEBSD_PORTS
  OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_CPU_AFFINITY=1 USE_ACCEPT4=1 USE_REGPARM=1 USE_OPENSSL=1 USE_STATIC_PCRE=1 USE_PCRE_JIT=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : LibreSSL 2.6.3
Running on OpenSSL version : LibreSSL 2.6.3
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.40 2017-01-11
Running on PCRE version : 8.40 2017-01-11
PCRE library supports JIT : yes
Built without Lua support
Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY

Available polling systems :
     kqueue : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use kqueue.

Available filters :
   [SPOE] spoe
   [TRACE] trace
   [COMP] compression

 (~) haproxy -c
HA-Proxy version 1.7.9 2017/08/18
Copyright 2000-2017 Willy Tarreau <willy@haproxy.org>

Usage : haproxy [-f <cfgfile|cfgdir>]* [ -vdVD ] [ -n <maxconn> ] [ -N <maxpconn> ]
        [ -p <pidfile> ] [ -m <max megs> ] [ -C <dir> ] [-- <cfgfile>*]
        -v displays version ; -vv shows known build options.
        -d enters debug mode ; -db only disables background mode.
        -dM[<byte>] poisons memory with <byte> (defaults to 0x50)
        -V enters verbose mode (disables quiet mode)
        -D goes daemon ; -C changes to <dir> before loading files.
        -q quiet mode : don't display messages
        -c check mode : only check config files and exit
        -n sets the maximum total # of connections (2000)
        -m limits the usable amount of memory (in MB)
        -N sets the default, per-proxy maximum # of connections (2000)
        -L set local peer name (default to hostname)
        -p writes pids of all children to this file
        -dk disables kqueue() usage even when available
        -dp disables poll() usage even when available
        -dG disables getaddrinfo() usage
        -dR disables SO_REUSEPORT usage
        -dr ignores server address resolution failures
        -dV disables SSL verify on servers side
        -sf/-st [pid ]* finishes/terminates old pids.

(~)

I am sick of thinking negative. It makes me wonder if anti-Net Neutrality has anything to do with it. Even IP providers has the right to intervene according to this new law, if already in effect. Other then that I don’t know what to think. It’s suppose to just work. I even tried it on FreeBSD-10.2 and that produce the same warning. It has no business compiling with so many errors warning for such a tiny file.

It makes no since to keep asking more about it. I’m just going to have to read until I figure out how the entire thing works and what I’m missing -- and I will. I might find another way to start it. Thanks for all.

 

[Snurg]

As you experienced the same behaviour on 10.2, too, that hints that maybe there is some overlooked configuration step to be done.
I remember that I had to do some manual testing (starting haproxy manually in the jail using the console until I found out how to start it up correctly), but do not remember the details what I did.

 

[SirDice]

The error is fairly obvious why it's not starting, there is no /usr/local/etc/haproxy.conf. There is no default configuration file so you'll have to create one from scratch. The service simply refuses to start if it can't read its configuration file or if there's a configuration error (that's the function of the precmd part of the rc.d(8) script).

(~) service haproxy onestart
/usr/local/etc/rc.d/haproxy: WARNING: [b]/usr/local/etc/haproxy.conf is not readable[/b].
/usr/local/etc/rc.d/haproxy: WARNING: failed precmd routine for haproxy

 

[max21]

The error is fairly obvious why it's not starting, there is no /usr/local/etc/haproxy.conf. There is no default configuration file so you'll have to create one from scratch. . . . .

I spend the entire week cleaning house. It been weird being away for the old computer this long but it did me good. I understood you guys replies and I could not wait get back. Today, I really got into the haproxy configuration.txt so deep until your examples haproxy.conf became understandable, masterpieces, my haproxy bibles. Now I’m only inches away. I think I done got greedy already. Now I know what haproxy can do for nginx and what nginx can do for apache. Soon I’m going to find out if haproxy can do what nginx reverse-proxy can do for apache child processes (rendering them useless)… I suspect haproxy can do that too. Youtube? … never knew what I was missing until now. There is little to nothing about haproxy as reverse-proxy, but these been very helpful. There is nothing like hearing it live. I imagine all the tech you guy talk at the 9-5.

Nginx vs Apache part in the middle:

About child processes at beginning:

Let's Encrypt through HAProxy

Prove that it works first.

global

        daemon
        maxconn 256

    defaults
        mode http
        timeout connect 5000ms
        timeout client 50000ms
        timeout server 50000ms

    listen http-in
        bind *:80
        server site1 127.0.0.1:8000 maxconn 32

It WORKS!

# /usr/local/etc/rc.d/haproxy onestart

Starting haproxy.

Available polling systems :

     kqueue : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK

Total: 3 (3 usable), will use kqueue.

Available filters :

          [SPOE] spoe
          [TRACE] trace
          [COMP] compression

Using kqueue() as the polling mechanism.

Finally… but I bet it would have took a full semester to get this far anyway.

edit--

You got to know this too.

I forgot to mention that I had pieced together quite a few haproxy.conf files, then after Snurg evaluation and SirDice calculation I knew that my file(s) were not readable for-sure by haproxy. Without a proper setup to run those full examples or any other examples that I could find that should had worked, the haproxy configuration.txt had a tiny test file, so I tried that today and it worked. The best thing that came out of all of this also is that I have a full FreeBSD-11.0 stable with all of FreeBSD-10.2p7 /usr/lib and /usr/lib32 included or else Curl would never install, and I got less errors when compiling haproxy … It’s all listed above.

The new CURL proved that those old lib were missing from 11.0 stable and CURL and whatever else definitely needed them. I wrote them down and founded them all in my old database1 FreeBSD-10.2-p7 jail. If it was not for that, I’ll would have gave up or something. So for those of you who never delete-old-lib since 9x or 10x, that is why stuff still works for you. Not all new ports have caught-up. I see that at the very least FreeBSD-10.2-p7 lib proved safe to use on 11.0 or else I be reverse-proxying with nginx right now that I think about it. And I’m glad 10.2-p7 was where I stopped until todays 11.0. Don't replace lib, only paste those that are missing. Knocking on wood. Now we know HAProxy.

Thanks guys!

 

[Snurg]

Welcome back, max21
Yes, I know that... when there is a new project it's at first a big pile of work to be done. To keep in the flow, I also often reduce housekeeping chores to a minimum for a while.
Didn't do dishwashing for two weeks and today it's dishwashing day because I ran out of clean plates, knives and forks

Regarding your mixed system (10.2+11), I'd suggest to just tear it down and start from scratch.
That version mismatches can only lead to more frustrating difficulties, easily avoidable by clean reinstall. This is one of the reasons why I want my jail manager to be able not only to manage jail.conf, but also to actually do generic setup of the most-important things.
I want to be able to do a clean base system install from scratch/CD, then enter jailboy setup, and there having the possibility to install/preconfigure a variety of useful jailed applications (in addition to reverse proxy and unbound dnssec cache):
-letsencrypt: container for certificate generator
-webservers: apache, nginx-trap server for disclosing session recording companies' customers
-jailed browsers
-jailed thunderbird
-jailed WINE
-and maybe even more (suggestions welcome)
... and It will be even possible to dynamically launch separate jailed instances of jailed firefox, for example.
This way I hope to make the thing attractive not only for advanced users, but also for every desktop user.

This is a horrible chore I do not want to do manually again. The main steps of that in the handbook are about this (part of the script generated by the setup part of my 'jailboy'):

mkdir -p /home/jails/templates/default.ro
cd /usr/src
make installworld DESTDIR=/home/jails/templates/default.ro
cd /home/jails/templates/default.ro
mkdir usr/ports
portsnap -p /home/jails/templates/default.ro/usr/ports fetch extract
cpdup /usr/src /home/jails/templates/default.ro/usr/src
mkdir /home/jails/templates/default.rw /home/jails/templates/default.rw/home /home/jails/templates/default.rw/usr-X11R6 /home/jails/templates/default.rw/distfiles
mv etc /home/jails/templates/default.rw
mv usr/local /home/jails/templates/default.rw/usr-local
mv tmp /home/jails/templates/default.rw
mv var /home/jails/templates/default.rw
mv root /home/jails/templates/default.rw
mergemaster -t /home/jails/templates/default.rw/var/tmp/temproot -D /home/jails/templates/default.rw -i
cd /home/jails/templates/default.rw
rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev
cd /home/jails/templates/default.ro
mkdir rw
ln -s rw/etc etc
ln -s rw/home home
ln -s rw/root root
ln -s ../rw/usr-local usr/local
ln -s ../rw/usr-X11R6 usr/X11R6
ln -s ../../rw/distfiles usr/ports/distfiles
ln -s rw/tmp tmp
ln -s rw/var var

And this is only the part to create the basic R/O and R/W skeletons...
You could run this from sh to get that installed in a snap.
(You might wonder why the above does not contain error checking. It gets executed command by command by my script, so that errors are caught.)

 

[max21]

. . .
. . .
Regarding your mixed system (10.2+11), I'd suggest to just tear it down and start from scratch.
That version mismatches can only lead to more frustrating difficulties, easily avoidable by clean reinstall.

I remember when GNOME-3 said take or leave it. GNOME-3 gave up the ability to be a true desktop because they lost control of their code. Everybody know a Gnome2 type desktop runs like a bat-out-of-he** which will do me just fine until quantum comes about. It’s no big deal, I just found a way to keep on upgrading without losing my favorite desktop. I lost the best twice since upgrading from FreeBSD-8.2. There is little hope once it’s gone. Support ends 3-months and a day at next release. However, do understand, that was all about how I maintained the last kick-a** desktop of the century, and not about a machine in production or jails. I’m about to remove/edit some of the stuff I wrote here. If you kind-of misunderstood the propose, I can only imagine what might happen to a newbie who try some of the things I wrote. I don’t want to feel responsible for that. One mistake, it over, back to Linux or Windows.

Anyway, after reviewing Jailboy, I see now that running many manual jails can be a major problem. If I have 300 manual-jails, I would have 300 directories full of devices. I never really thought about that before. So that is where the stuff hits the fan. One is better than many, when many can use, just one. Your project is way above my head especially the application parts, traps, etc and it seems like a timely mission. You might have to start an early website forum, message board, dedicated thread or something for it where you can document your work and pickup on ideas. Are you actually just getting started or are you near completion? It’s kind of like the Nginx story, build with FreeBSD, but in this case it’s specifically for FreeBSD. It sounds like something that would put the MAC in a state of shock.

I don’t know if this will help you or not, but as you know may know, I always talk about stripping stuff down to the bone. Well, last night I nearly created the smallest jail in the world; I just like to say stuff

The first one is design for pkg’s only. Thru rapid trial and error I removed many file from /etc and /usr, including nearly everything else in there other than /usr/bin and /usr/sbin so that they only contain what the system need to classify it as a fully functionable jail (ssh, log),etc;. I was surprise that both, HAProxy and sshguard only required three files inside of /usr/lib to run. The thing was they both use the same files or else neither would run:

1) libgnuregex.a
2) libgnuregex.so
3) libgnuregex.so.5

So far I got this kind of jail (running tiny pkg’s) down to 98.6MB untared. Tar size: 28.3MB. That almost the size of a Windows 10 module. For what I see now I don’t think it is possible to get any jail smaller than 85MB. It tars up in exactly 5.5 seconds. This is the kind of stuff I come back to from time to time. My theory is be it right, or wrong, I don’t have another life time to studying networking surrounded by in-house hackers. So I think I’ll just do some web hosting to earn some jail-bird seeds or to have some fun. If some genius breaks into the public facing jail, he would have no way to check-out and no tools to work with when needed; because for that jails everything goes to the dumpster such as /bin/chflags, /bin/dd, /usr/sbin/mountd, including everything else that the host and jail applications don’t need. Then I’ll raise the security level so high that even the host can’t jump the fence. The only thing he is allowed to do is to detect, destroy and replace. Above all, I just like learning how far can I go. Unlike Alcatraz, my jails accept no prisoners.

Whatever the case, FreeBSD got my back!

 

[Snurg]

The core jailboy stuff is about 3/4 done (installing and configuring the jail system, either interactive or command-line based).
Atm I am working on the configuration of the reverse proxy and PF things.

While doing the whole stuff I have incorporated several ideas I forgot in the basic concept at the beginning.
For example, grouping jails (think of the unix group model) makes many things easier. Modifying, reconfiguring, starting, stopping, multiple jails at once.

The jailboy thing creates a log of created/changed config files, so I am still doing "simulation" tests only, trying out the newly implemented stuff and refining until I am satisfied with the generated output (config files and batch of shell commands).
When the groups thing is implemented, I can finally do some first actual tests.
I need to create, modify and delete a lot of test jails to make sure I find the most obvious bugs before I can think of releasing a first alpha preview.
The script has >2400 lines atm. An ugly big chunk. But this will be modularly split up later into much smaller core script and (optional) "addons" (think docker etc, but easier to use).

Regarding the "size of the jail", this depends on perspective. An example:
I want Firefox in dynamically creatable jails. However, installed over the base system Firefox adds about a gigabyte of files. Thus to have actually fun with that, it is necessary to move the read-only part of that stuff to a new (read-only) base jail, reducing the "virtual system" that has to be created for each instance to near zero. This way new instances of can be fired up without actually creating a virtual system by copying the whole OS stuff.
Since summer even Windows offers a sandboxed version of its Edge browser. (albeit with serious disadvantages, reserves 8gb of physical ram and needs to copy around 1 gb of system to the sandbox. So this takes at least 10 sec to start, and above all it's only available on Enterprise.)
I think such should be easily possible for every desktop FreeBSD user with an 1-click-install, and without the abovementioned disadvantages of Edge.