Having trouble installing haproxy

Anyone know why I’m getting this error? I'm trying to install haproxy into a jail on Virtualbox with FreeBSD as host. The first few time I tried with /usr/ports/net/haproxy. Afterwards I found all of it dependencies and installed them first. Then I installed /usr/ports/sysutils/hatop which is especially required by haproxy. I had no problem installing ports for other web applications so far until now.

Code:
            warning:
 taking address of packed member 'branches' of class or structure
      'eb_node' may result in an unaligned pointer value [-Waddress-of-packed-member]
        root->b[side] = eb_dotag(&new->node.branches, EB_NODE);
                                  ^~~~~~~~~~~~~~~~~~
59 warnings generated.
gmake[2]: Leaving directory '/ram/usr/ports/net/haproxy/work/haproxy-1.7.9/contrib/halog'
===>  Staging for haproxy-1.7.9
===>   haproxy-1.7.9 depends on file: /usr/local/lib/libcrypto.so.42 - found
===>   Generating temporary packing list
install  -s -m 555 /ram/usr/ports/net/haproxy/work/haproxy-1.7.9/haproxy /ram/usr/ports/net/haproxy/work/stage/usr/local/sbin/
install  -s -m 555 /ram/usr/ports/net/haproxy/work/haproxy-1.7.9/contrib/halog/halog /ram/usr/ports/net/haproxy/work/stage/usr/local/sbin/
install  -m 444 /ram/usr/ports/net/haproxy/work/haproxy-1.7.9/doc/haproxy.1 /ram/usr/ports/net/haproxy/work/stage/usr/local/man/man1
/bin/mkdir -p /ram/usr/ports/net/haproxy/work/stage/usr/local/share/doc/haproxy
(cd /ram/usr/ports/net/haproxy/work/haproxy-1.7.9/doc/ && /bin/sh -c '(/usr/bin/find -Ed $1 $3 | /usr/bin/cpio -dumpl $2 >/dev/null 2>&1) &&  /usr/bin/find -Ed $1 $3 \(   -type d -exec /bin/sh -c '\''cd '\''$2'\'' && chmod 755 "$@"'\'' . {} +  -o -type f -exec /bin/sh -c '\''cd '\''$2'\'' && chmod 0644 "$@"'\'' . {} + \)' COPYTREE_SHARE \* /ram/usr/ports/net/haproxy/work/stage/usr/local/share/doc/haproxy)
*** Error code 127

Stop.
make[1]: stopped in /usr/ports/net/haproxy
*** Error code 1

Stop.
make: stopped in /usr/ports/sysutils/hatop
#

There are 3654 warning.

With the crazy ones looking like this to near end of the install ...
Line: 11801 - -> Line: 21752

That's a lot.

Code:
:ebtree/ebmbtree.h 'branches'             :from resultresult300 of  :134 ininwarning14 class  : :to anan  or  -122 unalignedunalignedtaking
structure   warning       pointerpointeraddress: [-Wconstant-conversion]'eb_node'  
valuevalueof
         taking                        *msg_type = PEER_MSG_STKT_INCUPDATE_TIMED;may[-Waddress-of-packed-member][-Waddress-of-packed-member]packed
  addressresult

member                                   ~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~                         up_ptr = &old->node.node_p;                        up_ptr = &old->node.node_p; of
in

'branches'   packedan                                  ^~~~~~~~~~~~~~~~                                  ^~~~~~~~~~~~~~~~of src/peers.c

 member:unalignedclass 259  'node_p':pointerebtree/eb32tree.hebtree/ebmbtree.hor 16 :: of:value438300structure   :: class[-Waddress-of-packed-member]1114'eb_node' :: orwarning
  may :         new_left = eb_dotag(&new->node.branches, EB_LEFT);
structure
       warningwarningresult'eb_node'implicit                             ^~~~~~~~~~~~~~~~~~: :  
inmayconversion
 takingtakingan      fromebtree/eb32tree.h   result :addressaddressunaligned 'int'444   in :ofofpointer to23   an :packedpackedvalue 'char'    unaligned membermember[-Waddress-of-packed-member] changes  pointer warning'branches''node_p'
 value:                   head->branches.b[EB_RGHT] = eb_dotag(&new->branches, EB_NODE);value ofof
 from  [-Waddress-of-packed-member] takingclassclass                                                      ^~~~~~~~~~~~~128

Pointer errors?
 
Unless I actually missed something, for what I see Freshports had no reference to include python as a dependency (strange), and when I included it anyway it still did not work. I use FreeBSD 11 stable, and its version of ports.
  1. libpcre.a : devel/pcre
  2. gmake : devel/gmake
  1. sysutils/hatop
net/haproxy


The pkg way works!
Code:
New packages to be INSTALLED:
   hatop: 0.7.7
   haproxy: 1.7.9
   python27: 2.7.14
   readline: 7.0.3
   libffi: 3.2.1_1
   python2: 2_3

CORRECTION: that is if you use pkg install hatop

So haproxy is standalone. I guess something is wrong with the port version or my updated SVN compiler is missing something. However, it did compile everything else.
 
SirDice, I removed debugging using the make.conf. It did compile. I kind of forgot what I did but the port version worked. Anyway, may I ask you one quick question here? I know you recommend to place haproxy on the host but is it ok to place haproxy or nginx in the first jail to do reverse-proxy for a few other jails with public websites. I'll be using a single server at a datacenter. It will lead me to eliminating this question in my next thread. I’m beginning to understand jail networking, but it seems that jailing can be design for difference use. Even if all this additional security is not needed (reverse-proxy in jail), I want to try it anyway. BTW, no more size and color to threads for me. It been a habit for docs on my mate-desktop.

https://forums.freebsd.org/threads/54445/
 
Didn' t try ports. Just directly installed into jail, it was -j option iirc. It works fine together with pf for several jails serving different web domains. Didn't try out hatop, as haproxy has sweet logging.
 
Didn' t try ports. Just directly installed into jail, it was -j option iirc. It works fine together with pf for several jails serving different web domains. Didn't try out hatop, as haproxy has sweet logging.

What is IIRC? Is this for iocage, ezjail or jails in ZFS. I only know how to build jails manually. The clues I just found might be related to those and one of them mention something about mounting a jail inside a jail or something like that. What key words could you provide so I can do a search? I am excited that this is actually possible. Thanks Snurg. I believe I just read some of your replies to a few threads within the past few days. I'm gathering many facts as I can before posting my final question to make sure there be no better way to go. So I cant miss anything, anymore.
 
iirc=if i remember correctly.

I actually did the manual jail configuration without iocage, ezjail etc, because:
1st these still use the old jail approach, which is not only a bit clumsy and difficult to maintain manually, but also cursed with a deprecation warning, and
2nd I had the feeling that learning necessary to deal with those kinds of "jail frontends" is practically the same as the new jail approach (using jail.conf and rc.conf), so I saw little use in learning iocage, ezjail etc.

It's a bit sad that the handbook, which is otherwise very good, does not mention the new (quite easy-to-handle) jail.conf method. So it's best to read the jail.conf manpage in addition to the handbook.

On my server, every jail has its own private IP, runs its own web server environment, a jailed haproxy forwards the domains (which are in the http(s) header) to the appropriate private IPs (without decrypting while passing through, a strength of haproxy), and PF in turn forwards the packets to the appropriate jails and also takes care that the haproxy can only be accessed by cloudflare and local hosts, so foreign portscanners don't see my web servers.
 
. . . On my server, every jail has its own private IP, runs its own web server environment, a jailed haproxy forwards the domains (which are in the http(s) header) to the appropriate private IPs (without decrypting while passing through, a strength of haproxy), and PF in turn forwards the packets to the appropriate jails and also takes care that the haproxy can only be accessed by cloudflare and local hosts, so foreign portscanners don't see my web servers.

So that’s how it works! But haproxy don’t work along. It needs CloudFlare to do SSL termination and cloud flare provides some extras.

So that is why many threads speaks of using nginx to do reverse-proxy with SSL termination. Now I’m searching for the pros and cons of each, if any. This link made your most exclusive reply crystal clear for me. I thought CloudFlare was like rackspace or something, and I had no clue of how any of this worked, whatsoever, but in my gut I knew something should work as described. It was all in your wording, and then this link.

https://martensson.io/cloudflare-universal-ssl-with-haproxy/

This is getting more interesting by the minute.

Two Beers and a Button for you my guy :)
 
But haproxy don’t work along. It needs CloudFlare to do SSL termination and cloud flare provides some extras.
Huh? No it doesn't. HAProxy is perfectly capable of SSL terminating. It has done so since version 1.5.
 
Huh? No it doesn't. HAProxy is perfectly capable of SSL terminating. It has done so since version 1.5.

SirDice, I forgot about that. Most of what I’m seeking to do I read about in older threads where HAProxy was kind of new. However, passing data thru without decrypting, and leaving nothing behind to scan got to be a wonderful thing. I’ll make up for the addition overhead, if any, elsewhere. BTW: I think I was wrong about the port version worked. I'm going to try again just to make sure. I got the port version and the pkg version install information inside /var/cache and /var/db/ports. So I'm not sure, but I think the port version dependency python2-2_3 fail. I'll find out tonight. I never leave unknown things behind for something else. So it makes no since to start doing that now, even though I’ll trade the world for a Snurg setup.
 
Let me know what you're running into. I use HAProxy for my own stuff and I've set it up for a client. So I'm quite sure it builds correct.

Here's my own haproxy.conf with some bits changed, you can use it as an example config.

Code:
global
        maxconn 30000
        daemon

        log /dev/log local2

        user nobody
        group nobody

        stats socket /var/run/haproxy.socket mode 777 level admin

        tune.ssl.default-dh-param 2048

        ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
        ssl-default-bind-options no-sslv3 no-tls-tickets

defaults
        log global
        option httplog
        option dontlognull
        mode http

        option httpclose
        option abortonclose
        option forwardfor header X-Real-IP
        option http-server-close

        timeout connect 5000
        timeout client 50000
        timeout server 50000

        errorfile 400 /usr/local/www/haproxy/errors/400.http
        errorfile 403 /usr/local/www/haproxy/errors/403.http
        errorfile 404 /usr/local/www/haproxy/errors/404.http
        errorfile 500 /usr/local/www/haproxy/errors/500.http
        errorfile 503 /usr/local/www/haproxy/errors/503.http

        stats enable
        stats uri /haproxy?stats
        stats realm Statistics
        stats auth admin:changemenow

frontend http-in
        bind 1.2.3.4:80

        reqidel ^X-Real-IP:.*

        default_backend local

        # Letsencrypt
        acl is_letsencrypt path_beg /.well-known/acme-challenge/
        acl is_mail hdr_dom(host) -i mail.example.com
        acl is_webtrees hdr_dom(host) -i webtrees.example.com

        redirect scheme https if is_mail !{ ssl_fc }
        redirect scheme https if is_webtrees !{ ssl_fc }

        use_backend local if is_letsencrypt
        use_backend mail if is_mail
        use_backend webtrees if is_webtrees

frontend https-in
        bind 1.2.3.4:443 ssl crt /usr/local/etc/haproxy/ssl/

        http-response set-header Strict-Transport-Security "max-age=16000000; includeSubDomains; preload;"
        #http-response set-header X-Frame-Options DENY
        http-response set-header X-Content-Type-Options nosniff

        tcp-request content accept if { req_ssl_hello_type 1 }

        reqidel ^X-Real-IP:.*

        default_backend local

        # Letsencrypt
        acl is_letsencrypt path_beg /.well-known/acme-challenge/
        acl is_mail hdr_dom(host) -i mail.example.com
        acl is_webtrees hdr_dom(host) -i webtrees.example.com

        use_backend local if is_letsencrypt
        use_backend mail if is_mail
        use_backend webtrees if is_webtrees

backend local
        option httpchk GET /up.txt
        server localhost 127.0.0.1:80 check

backend webtrees
        option httpchk GET /up.txt
        server webtrees 192.168.21.3:80 check

backend mail
        option httpchk GET /up.txt
        server mail 192.168.21.4:80 check

SSL is terminated at HAProxy, traffic to the backends is always plain HTTP. The SSL settings also result in an A+ rating.

The X-Real-IP header can be used on the Apache or nginx backends to get the 'original' client's IP address (or else everything will appear to come from the HAProxy machine). A nginx running on localhost is used for Letsencrypt to automatically update the SSL certificates.
 
Let me know what you're running into. I use HAProxy for my own stuff and I've set it up for a client. So I'm quite sure it builds correct.

Here's my own haproxy.conf with some bits changed, you can use it as an example config.

Thank SirDice,

Its going to take me a minute to complete my setup and figure out this file. I don't want to munk this up.
 
I just did some checking, which don't seems so important now, at this moment. ... It was hatop that threw things off for me.
 
HAProxy doesn't have any run dependencies, it does have 2 build dependencies.
 
I think I'll write up some HOWTO these days, because the configuration I described is a thing apparently quite some people would like to have.

It is a bit complex, because the necessary configuration things regarding host and jails spread over more than a dozen files.
For example, it took me quite some time to figure out how to get SSL passthru, so HAproxy does not need to tamper with the actual data and preventing the actual web server doing the stuff (which limits security in the end). Aside of that, I would lie to my users if I would say them that they have a secure connection while actually using some decrypting and thus eavesdropping reverse proxy.

Regarding PF, there are still quite a few things in my pf.conf that I didn't yet figure out how to do correctly (traffic counting, bandwidth limiting/distributing, etc).
In the Howto I could then also explain the relationships between jail.conf and rc.conf, what parts of the configuration belongs where, of which I didn't find a complete documentation anywhere, and had to find out with much, much reading.
 
iirc=if i remember correctly. . . .

It's a bit sad that the handbook, which is otherwise very good, does not mention the new (quite easy-to-handle) jail.conf method. So it's best to

read the jail.conf manpage in addition to the handbook.

I come to realize that this is fair. Those folks invented or bettered those jail frameworks to make it easy for new and future FreeBSD users . . . One of them wrote the Handbook of Handbooks IMO, FreeBSD INSTALL GUIDE. However, it’s FreeBSD responsibility to improve upon great ideas for everyone. The authors would use those improvement wherever possible to make things friendlier for GUI type users. Us connoisseur mostly know we need to drill down a bit to find developers type information.

I think I'll write up some HOWTO these days, because the configuration I described is a thing apparently quite some people would like to have.
. . .
You have gotten too use to your own innovation, it's only second-nature to you. If it works the way you describe, even a bit, it would take the way jails are build or used to a whole new level. With that said …

Bring it on! Look at all the files it takes to make FreeBSD tick. A jail is like a system by itself that you build! However, most of use fail to use our imagination. Evidently, you did not! You did more with only a dozen plus files … that’s something to be admired.

About jail.conf and rc.conf; although so very simple, I have 8 files per jail. I use each set to start and do all for its own jail. Automation is my goal. I use my jail.conf and rc.conf only when I feel like it, mainly to make sure all startup information match. I am not limited when I use my own files. I can add more of what can be accomplished - - but I am restricted when I use the jail.conf and rc.conf. they both enforce limitations. So, when it comes to files, anything less then what it takes to run FreeBSD need not be that complicated, just more clever. Once all the files are organized, you realize the only complexity was learning how to set it up . . . though this is major *.

Three of the eight are .csh scripts, InstallWorld, start-jail, and kill-environment, Using .csh scripts are deem unsafe, but now I don’t think so. Once started it should never have to restart. Also consider the fact that it depends on how the entire system is being used … using my-jails on a LAN with un-trusted users, sure there’s risk firing up .csh scripts. For private intercommunication among my-jails running web-servers already started by csh, 1 hour – 20 years, I don’t think so. There are FreeBSD applications that use .csh script, or two, so why tell users who knows what they’re doing, for years, not to do it, with no evaluation of the situation, I believe the jail system use at least one .csh call under the hood, if not my way is stronger.

If you think its worth checking out, at your request I’ll post it as a HOW TO by Sunday night. This will give me time to clean it up a bit. I planned to do that someday anyway. It’s so cool to see how every single thing kicks in under the hood just like what you indicated above about jail.conf and rc.conf. What’s going to blow my mind is to see a FreeBSD server running remotely for many years just like in the old blogs I use to read. 5x – 6x They were loving it, geeking for life with fewer visits to the keyboard.

Snurg, now curiosity is killing the cat. I could be way off base but I think you also use a few .csh calls :) but don’t want to be bother with the “please don’t do that” replies, so you worked along, like myself. I also think some special device(s) may be involved, but that is only a shot in the dark. Whatever the case, I hope you find the time to create the HOW TO.

BTW: could you tell me how you set your alliances for your haproxy jail and your public web-jails. I’m not sure which to use even for a standard WEB-JAILS setup. I only been using the third one in the list below which allow me to build ports and packages.


.......................................................................................................................................................................................................
For those who don't know, .csh is no play toy. It will wipe out your entire system in a blink of an eye if you make a single mistake, or don’t know what you are doing. If you see it and don’t know it LEAVE IT ALONG.

So that why people warns us. But for those who do the warning, understand the situation first, and don't interfere with progress when you know it's productive. It's the user choice and it's his machine he risk during development.

No mistakes allow.

Now you know all your ABC's
.......................................................................................................................................................................................................



Now it’s time to study haProxy with SirDice example and learn something.

This is what I’m after:
https://forums.freebsd.org/threads/63261/

Skywalkers

over and out . . . >


1) for a remote dedicated or VPS public web server running Snurg jail solutions
Code:
cloned_interfaces="lo1"
ifconfig_lo1="inet 10.0.0.1/24"
ifconfig_lo1_alias0="inet 10.0.0.100/32"                      # haproxy
ifconfig_lo1_alias1="inet 10.0.0.101/32"                      # domain1
ifconfig_lo1_alias2="inet 10.0.0.102/32"                      # domain2

2) for port forwarding a way to connect to your home or work server from laptop while at McDonald's
Code:
cloned_interfaces="lo1"
ipv4_addrs_lo1=" inet 10.1.1.1 netmask 255.255.255.0"
ipv4_addrs_lo1_alias0="inet 10.1.1.100 netmask 255.255.255.255"
ipv4_addrs_lo1_alias1="inet 10.1.1.200 netmask 255.255.255.255"

3) for connection to web from jails to do such things as installing ports, ftp or use for virtual hosted websites
Code:
ifconfig_em0_alias0="inet 10.2.2.1/32"
ifconfig_em0_alias1="inet 10.2.2.102/32" 
ifconfig_em0_alias2="inet 10.2.2.103/32"

If this is correct, I'm really ready to roll?
 
I have slept over it and thought about it.
As I am configuring my desktop atm for use as desktop and development/testing server, it's probably best to partition the whole thing some way like this:
1. basic PF configuration
2. jailed secure DNS server (there is a good Howto already, to which I only will add some useful information)
3. Haproxy jail and a web server jail dedicated for obtaining Let'sEncrypt certificates
4. a template jail for the used certificates etc (or several, one for apache, one for nginx, ...)
5. maybe some scripts for routine things like cloning new jails from templates and preconfiguring them (ip, hostname, rc.conf etc)
6. finally some documentation about the routine processes in creating/managing jails

As this your thread is about haproxy, I post my haproxy.conf (anonymized and boiled down to two example web servers):

Code:
# 10.1.1.10 : jailed haproxy
# 10.1.1.11 : jailed webserver for site_without_https.org
# 10.1.1.12 : jailed webserver for site_with_http_and_https.org

global
maxconn 2000
user haproxy
group haproxy

defaults
timeout client 30s
timeout server 30s
timeout connect 10s

frontend ft_http
bind 10.1.1.10:80
mode http
acl http_site_with_http_and_https hdr(host) -i site_with_http_and_https.org
acl http_site_with_http_and_https_www hdr(host) -i www.site_with_http_and_https.org
acl http_site_without_https hdr(host) -i site_without_https.org
acl http_site_without_https_www hdr(host) -i www.site_without_https.org
use_backend backend_site_with_http_and_https_http if http_site_with_http_and_https
use_backend backend_site_with_http_and_https_http if http_site_with_http_and_https_www
use_backend backend_site_without_https_http if http_site_without_https
use_backend backend_site_without_https_http if http_site_without_https_www

frontend ft_https
bind 10.1.1.10:443
mode tcp
acl https_site_with_http_and_https req_ssl_sni -i site_with_http_and_https.org
acl https_site_with_http_and_https_www req_ssl_sni -i www.site_with_http_and_https.org
use_backend backend_site_with_http_and_https_https if https_site_with_http_and_https
use_backend backend_site_with_http_and_https_https if https_site_with_http_and_https_www

backend backend_site_without_https_http
mode http
server server_site_without_https_http 10.1.1.11:80

backend backend_site_with_http_and_https_http
mode http
server server_site_with_http_and_https_http 10.1.1.12:80

backend backend_site_with_http_and_https_https
mode tcp
server server_site_with_http_and_https_https 10.1.1.12:443

As you can see, the config is totally basic and primitive, but it works for me. (Suggestions are welcome!)
Hope it helps.
 
I have slept over it and thought about it.
As I am configuring my desktop atm for use as desktop and development/testing server, it's probably best to partition the whole thing some way like this:
...
...
5. maybe some scripts for routine things like cloning new jails from templates and preconfiguring them (ip, hostname, rc.conf etc)
...

As you can see, the config is totally basic and primitive, but it works for me. (Suggestions are welcome!)
Hope it helps.

Thanks Snurg

I like basic & primitive, that why I stuck with FreeBSD. Now I can do a few of the right things at the same time again. Cleaning up my jails scripts would be the perfect place to start. I’ll post my primitive way of building jails in within 36 hours. There might be something in there that you can use right now. Take your time to do all of what you need. All I needed were some facts. You guys provided me enough of that to keep me busy for a week.
 
As I am configuring my desktop atm for use as desktop and development/testing server, it's probably best to partition the whole thing some way like this:

More ideas:

ada0s1: 20 - 32GB

Code:
/dev/ada0s1a - /   512 - 2048
/dev/ada0s1b - swap   64 - 1024
/dev/ada0s1d - /tmp   64 - 1024
/dev/ada0s1e - /var     1024 - 4096
/dev/ada0s1f - /usr     17000 - 21504
FREE 1024MB
.......................................................
ada0s2:        for VM's and bk/restore ada0s1. Disaster Protection

/dev/ada0s2a /       20 - 32GB   install all of FBSD here
/dev/ada0s2b - swap   1024   only a swap is needed 
/dev/ada0s2d - /mydir/d   150GB
/dev/ada0s2e - /mydir/e   150GB
/dev/ada0s2f - /mydir/f   150GB
/dev/ada0s2g - /mydir/g   150GB
/dev/ada0s2h - /mydir/h   300GB
FREE 1024MB
.......................................................
ada0s3  ###GB for windows or more FBSD snaps (i,j,k,l,m)

.......................................................
ada0s4  EXTENDED   use Arch to recover from a disaster of disasters
ada0s5  1024mb       Arch swap
ada0s6  1536mb       Arch Linux

......................................................
ada0s7    msdos
ada0s8    msdos
ada0s9    msdos
ada0s10   msdos
ada0s11   msdos
ada0s12   msdos
ada0s13   msdos
ada0s14   msdos
ada0s15   ntfs
ada0s16   ntfs
ada0s17   ntfs
ada0s18   ntfs
ada0s19   msdos

I actually build my most important FreeBSD development VM on ntfs-16 and backup both FreeBSD host’s (ada0s1) and (ada0s2a) and all VM’s on the last partition, msdos-19. I also did all of this on a 500GB hard drive because I knew one day a flash drive will at least be available for that size in the near future. So my size is difference. It works for me, but these sizes are better. I wish I had plan for the 1GB flash-drive.

This way you don't need all of those extra partitions and can do everything within FreeBSD ufs or whatever. Anyway, it may be helpful. It's no better then the rest but no less than the best when it comes to doing everything possible on a single machine, pocket-size and cell ready.

I trust dd more, because he don't care.

This is why I keep the host’s and most of my vm’s as small as needed, but my main devel vm is 131GB.

BACKUP:
Code:
 dd if=/dev/ada0s16 bs=64k | gzip -c | split -b 3999m - /mydir/win/p/fbsd-devel/fbsd-devel.gz.
RESTORE:
Code:
cat /mydir/win/p/fbsd-devel/fbsd-devel.gz.* | gzip -dc | dd of=/dev/ada0s16 bs=64k
 
You guys are not going to believe this:
To make sure that there be no flaw in my upcoming jail demo, I updated to revision 326375 at SVN. I make buildworld, 7-hours. Once I got to this, make -DBATCH_DELETE_OLD_FILES delete-old-libs this is what I got:

By hand because Virtualbox terminal
has no select, so in this order:
Code:
/usr/lib/libbsnmptools.so.0
/usr/lib/libbsnmptools.so
/usr/lib/libgssapi.so.10
/usr/lib/debug/usr/lib/libgssapi.so.10.debug
plus 42 more files with .debug extensions
….
….
>>> Old libraries removed
Absolutely nothing else was in there to be removed.

So that’s why! I update with SVN weekly but had not buildworld for over a month. Evidently, the debug code caused the issues. However, if this did not happen, I’ll be struggling right now with the wrong reverse-proxy, forever. Well I guess this thread is SOLVE, with another happy camper prepared to kick *. My ears will be open, and you guys know where I will be ... and to think, I still got 24 hours to make my machine XMAS clean :)

See you latter Skywalkers.

Thank you FreeBSD.
 
Good point, I hadn't even considered that. Probably because it's almost automatic for me. I've done so many buildworld I don't even think about what I'm doing, it's mostly muscle memory.
 
Cleaning up after make buildworld... oh yes thats a thing I realize I practically never did... :rolleyes: Thanks for pointing at that!

And thanks for post #20 too!
Here is the background why:

When I was starting my reverse-proxied server experiments last year, I used squid as reverse proxy because I knew it a bit.
I did the first parts of my web project (sort of simple-to-use CMS) using only http.
When it was time to add https support, I found out that the claims of squid (and many other software which does reverse proxying) to be able to handle https must be interpreted in a different manner than I meant.
I had to learn that they mean it that way "you must make our proxy an eavesdropping man-in-the-middle!".

And that made me to investigate a bit how https works. I already knew for long time that the https protocol had been updated shortly after its introduction because it had a severe flaw - it lacked the unencrypted clear-text domain name. So it was necessary to decrypt the packets to find out which domain on that particular IP the packet was addressed to. And this flaw forced hosters to unencrypt packets at the server handling the IP - often the proxy/load balancer and not the actual web server. Which would break the so-praised "security"... So a thing called "SNI" was introduced.

So I had to search what software actually supports SNI, and this was why I ended up installing haproxy.
But when I was configuring, I found out that virtually all pages on the web dealing with haproxy https configuration are about the man-in-the-middle concept, and *not* about simple passthrough. And it took me quite a while to find the req_ssl_sni needle in the huge haystack which the haproxy documentation is...

When I read post #20 I thought, hmm is this still so rare to find, I did a search for "req_ssl_sni" again, this time on Bing. Why do I mention Bing, you might ask. The answer is that I am a lazy ass and just used google because it is the default search engine of Firefox. But Palemoon, which I use as my browser since FF Quantum made me change my default browser, does not offer Google in its search bar :)

And what jumps into my eyes then? One of the first hits was this page !
Quote:
I finally found a solution. It wasn't in the documentation though.

Use -m end instead of -i for wildcard

One thing what I felt like a sore in my haproxy.conf was the need to explicitly name each subdomain.
You put me into the tracks to find a solution for that just by chance :) This why my "Thanks" for post #20 :)
 
I might as well post this here. Put this in your jail-tool collection. The main problem is too many parts that have to be ran by hand. Some one once suggested a way to use .sh for all scripts but it didn’t work. Nevertheless, it’s great for testing, and runs great it but one too many script to total shutdown. Thanks Snurg … we ALL were right on time.

It’s great that more came out of this thread then expect, the good the bad and the ugly. Now even I know how they got the charlatan tweet. Tricky Dick, trying to be slick now getting hack by his own sponsor’s sticks. How else could Putin know about Hillary and the Trunkster love affair. . .



I now comment out openssl, then install sshguard and libressl for all jails. I really just build one ail, I give it only what all should have, then copy it with new name. but it's kind of tricky and confusing. It only take minutes to build a new one then customized each, might as well.

InstallWorld tells the whole story
A_InstallWorld

Code:
#!/bin/csh
#                                                                PROXY - INSTALL WORLD
#
# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        is there and directory empty
mkdir /mnt/z/proxy
cd /mnt/z/proxy && chflags -R noschg * && rm -Prf *; sleep 2;
# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        paste-in custom files
cp -pr /mnt/j/MAKE/proxy/config-files/REQUIRED/script1/.q /mnt/z/proxy/.q
cp -pr /mnt/j/MAKE/proxy/config-files/REQUIRED/script1/.s /mnt/z/proxy/.s
sleep .5
# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        set  environment
cd /mnt/z/;
setenv RESORT_0 /mnt/z/proxy;
# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        make system directories
mkdir -p $RESORT_0/dev
mkdir -p $RESORT_0/etc
mkdir -p $RESORT_0/usr/ports
mkdir -p $RESORT_0/usr/src
# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        installworld
cd /usr/src/
make installworld DESTDIR=$RESORT_0 # SRCCONF=/etc/src.conf
# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        mergemaster
sleep 2
mergemaster -a -C -D ${RESORT_0}
sleep 2
cd /usr/src/etc
cp /etc/resolv.conf $RESORT_0/etc
cp /etc/mnt/j/MAKE.conf $RESORT_0/etc
# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        distribution
sleep 2
make distribution DESTDIR=$RESORT_0 OPTIONS_UNSET=OPENSSH OPTIONS_UNSET=OPENSSL
sleep 2
rm -Pf $RESORT_0/etc/ssl/openssl.cnf
mkdir -p $RESORT_0/usr/local/openssl
cp /etc/ssl/openssl.cnf $RESORT_0/etc/ssl
cd $RESORT_0/usr/local/openssl/
sleep 2
ln -s ../../../etc/ssl/openssl.cnf openssl.cnf

# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        delete temproot on host

cd /var/tmp/temproot && chflags -R noschg * && rm -Prf * && cd .. && rm -Pr temproot

# . . . . . . . . . . . . . . . . . . . . . . . . . . . .        customize - add/remove
mkdir /mnt/z/proxy/usr/local/www
rm -Pr /mnt/z/proxy/sys

cp -pr /mnt/j/MAKE/proxy/config-files/with-dns/dhclient-enter-hooks /mnt/z/proxy/etc/dhclient-enter-hooks
cp -pr /mnt/j/MAKE/proxy/config-files/with-dns/dhclient.conf /mnt/z/proxy/etc/dhclient.conf
cp -pr /mnt/j/MAKE/proxy/config-files/with-dns/resolv.conf /mnt/z/proxy/etc/resolv.conf
cp -pr /mnt/j/MAKE/proxy/config-files/ssh/sshd_config /mnt/z/proxy/etc/ssh/sshd_config

cp -pr /mnt/j/MAKE/proxy/config-files/periodic.conf /mnt/z/proxy/etc/periodic.conf
cp -pr /mnt/j/MAKE/proxy/config-files/rc.shutdown /mnt/z/proxy/etc/rc.shutdown
cp -pr /mnt/j/MAKE/proxy/config-files/sysctl.conf /mnt/z/proxy/etc/sysctl.conf
cp -pr /mnt/j/MAKE/proxy/config-files/syslog.conf /mnt/z/proxy/etc/syslog.conf
cp -pr /mnt/j/MAKE/proxy/config-files/src.conf /mnt/z/proxy/etc/src.conf
cp -pr /mnt/j/MAKE/proxy/config-files/services /mnt/z/proxy/etc/services
cp -pr /mnt/j/MAKE/proxy/config-files/rc.conf /mnt/z/proxy/etc/rc.conf
cp -pr /mnt/j/MAKE/proxy/config-files/hosts /mnt/z/proxy/etc/hosts
sleep 1
ln -s /mnt/z/proxy/usr/local /mnt/z/proxy/Proxy_Local
cd /

echo "......."
echo " DONE. "
sleep 1000
exit


# FOR BIG-JAIL TO COMPLETE:  if folder is full = 4.2 min - if empty = 3 min
#      
# make buildworld
# make installworld DESTDIR=$RESORT_0

b_delete.cell
Code:
#!/bin/sh
# ................................................     delete PROXY jail
#
echo "deleting PROXY jail. 91 seconds to complete."

cd /mnt/z/proxy && chflags -R noschg * && rm -Prf *

echo "......."
echo " DONE. "
sleep 3

c_untar.cell
Code:
#!/bin/sh
# ................................................  reinstall PROXY jail
#
tar jxvf /mnt/j/MAKE/proxy.tgz -C /mnt/z/

echo "42 seconds"
echo ".........."
echo "   DONE   "
sleep 3

d_start
Code:
#!/bin/csh
# ................................................      Start PROXY jail
#
mount_nullfs -o /dev/ada0s1f /usr/ports /mnt/z/proxy/usr/ports;
mount_nullfs -o /dev/ada0s1f /usr/src /mnt/z/proxy/usr/src;
sleep 1;
setenv RESORT_0 /mnt/z/proxy
cd $RESORT_0
mount -t devfs devfs /mnt/z/proxy/dev
devfs -m /mnt/z/proxy/dev rule -s 4 applyset
devfs -m /mnt/z/proxy/dev rule apply path tun0 unhide

ln -s dev/null kernel
jail $RESORT_0 proxy.cell 10.0.0.1 /bin/sh

e_kill-environment
Code:
#!/bin/csh
# ................................................      Kill Environment
#
umount -A -t nullfs
umount -A -t devfs
sleep 2
unsetenv RESORT_0
unsetenv RESORT_1
unsetenv RESORT_2
unsetenv RESORT_3
unsetenv RESORT_4
unsetenv RESORT_5

echo "-----";
echo "-- jail is now Close!";
#cat /dev/null > /mnt/z/site1/var/log/nginx-access.log;
#cat /dev/null > /mnt/z/site1/var/log/nginx-error.log;
#cat /dev/null > /mnt/z/site1/var/log/nginx-header.log;
#cat /dev/null > /mnt/z/site1/var/log/php-fpm.log;
echo "-----";
echo "Logs are Clean";
echo "-----";
echo "-----";
sleep 3

tar-it
Code:
#!/bin/sh
# ................................................          Backup PROXY
#
echo "backing up PROXY. 74 seconds to complete."

rm -Pr /mnt/j/MAKE/proxy.tgz; sleep 2;

cd /mnt/z && tar cvzf /mnt/j/MAKE/proxy.tgz proxy/

echo "......."
echo " DONE. "
sleep 3

I once had it then I lost it. So, I had other things to do.
view-env

Code:
#!/bin/csh
# ................................................         View Commands
#
cd $RESORT_0
printenv $RESORT_0 /mnt/z/proxy

sleep 999
exit

z_RemoveFiles
Code:
#!/bin/sh
# ................................................ PROXY jail:
# ................................................ Remove un-needed files
rm -Pr /mnt/z/proxy/mnt
rm -Pr /mnt/z/proxy/net
rm -Pr /mnt/z/proxy/proc
rm -Pr /mnt/z/proxy/boot
rm -Pr /mnt/z/proxy/mntdia
rm -Pr /mnt/z/proxy/rescue
rm -Pr /mnt/z/proxy/etc/X11
rm -Pr /mnt/z/proxy/etc/zfs
rm -Pr /mnt/z/proxy/etc/motd
rm -Pr /mnt/z/proxy/etc/hosts
rm -Pr /mnt/z/proxy/etc/bluetooth
rm -Pr /mnt/z/proxy/etc/sysctl.conf
rm -Pr /mnt/z/proxy/etc/periodic.conf
rm -Pr /mnt/z/proxy/etc/pkg/FreeBSD.conf

#rm -Pr /mnt/z/proxy/etc/syslog.conf

rm -Pr /mnt/z/proxy/usr/obj
rm -Pr /mnt/z/proxy/usr/src
rm -Pr /mnt/z/proxy/usr/ports
rm -Pr /mnt/z/proxy/usr/tests

#rm -Pr /mnt/z/proxy/usr/share/openssl
rm -Pr /mnt/z/proxy/usr/share/examples
rm -Pr /mnt/z/proxy/usr/share/firmware
rm -Pr /mnt/z/proxy/usr/share/games
rm -Pr /mnt/z/proxy/usr/share/doc
rm -Pr /mnt/z/proxy/usr/share/man
rm -Pr /mnt/z/proxy/usr/local/man

cp -p /mnt/j/MAKE/proxy/config-files/hosts /mnt/z/proxy/etc/hosts
cp -p /mnt/j/MAKE/proxy/config-files/rc.conf /mnt/z/proxy/etc/rc.conf
cp -p /mnt/j/MAKE/proxy/config-files/sysctl.conf /mnt/z/proxy/etc/sysctl.conf # useless so far
cp -p /mnt/j/MAKE/proxy/config-files/resolv.conf /mnt/z/proxy/etc/resolv.conf
#cp -p /mnt/j/MAKE/proxy/config-files/syslog.conf /mnt/z/proxy/etc/syslog.conf
cp -p /mnt/j/MAKE/proxy/config-files/periodic.conf /mnt/z/proxy/etc/periodic.conf
cp -p /mnt/j/MAKE/proxy/config-files/FreeBSD.conf /mnt/z/proxy/etc/pkg/FreeBSD.conf

echo "......."
echo "......."
echo " DONE. "
sleep 3

.q = you MUST run this first to close terminal and such.
after closing run .e_kill-environment so /dev don't get stuck

Code:
#!/bin/sh
#  ..............         STOP all then quit, don't forget to kill-environment. I have
#  ..............         not figure a way to include it?  This terminal must be closed.
service cron stop
service syslogd stop
service sshguard stop
/usr/bin/pkill sshd
/usr/bin/pkill nginx
/usr/bin/pkill ld-elf
/usr/bin/pkill syslogd
/usr/bin/pkill ld-elf32
/usr/bin/pkill cron
/usr/bin/pkill clean_var
rm -Pr /var/run/*

kill -TERM -1
kill -KILL -1

#/usr/local/etc/rc.d/varnishd onestop;   sleep .5;
#/usr/local/etc/rc.d/nginx onestop;      sleep .5;
#/usr/local/etc/rc.d/php-fpm onestop;    sleep .5;
#/usr/local/etc/rc.d/mysql-server onestop;       sleep .5;
#
#cat /dev/null > /var/log/nginx-access.log
#cat /dev/null > /var/log/nginx-error.log
#cat /dev/null > /var/log/nginx-header.log
#cat /dev/null > /var/log/php-fpm.log

.s = run this after jail termial is open.
Code:
#!/bin/sh
#  ..................... System Varables
sh /etc/rc
sleep 1
exit
[/b]
 
Last edited:
# #####################################################
# #####################################################
# It seems that all of this runs in jail. sysctl makes
# make rc.conf tick but it dose nothing else for me, but
# it must be there even if empty or else jail don't work
# #####################################################
# #####################################################

If you want to see some action, go to
/var/run You will like it a lot inside there.
I wipe it completely out for fun just to see it all
come back. This is how I know when I screw something
up, elsewhere. I compair it to host /var/run for a doulble check,
but his count is largers. Well that's all the debugging you will ever need.

Add to jail /etc/rc.shutdown
Code:
# Insert other shutdown procedures here
service cron stop
service syslogd stop
service sshguard stop
/usr/bin/pkill sshd
/usr/bin/pkill nginx
/usr/bin/pkill ld-elf
/usr/bin/pkill syslogd
/usr/bin/pkill ld-elf32
/usr/bin/pkill cron
/usr/bin/pkill clean_var
#rm -Pr /var/run/*

dhclient.conf
Code:
supercede domain-name-servers 69.164.196.21, 96.90.175.167; # OpenNIC - sDice

dhclient-enter-hooks
Code:
# I use it on host and jails SirDice.
add_new_resolv_conf() {
        # We don't want /etc/resolv.conf changed
        # prevents dhclient from touching it
        return 0
}

#add_new_routes() {
        #route add -net 10.0.0.138 -iface $new_ip_address
        #route add default 10.0.0.138
#}

resolv.conf
Code:
#    OpenNIC  
nameserver 69.164.196.21
nameserver 96.90.175.167

rc.conf
Code:
network_interfaces=""
hostname="proxy.ka.cell"
ifconfig_em0_alias0="inet 10.0.0.1 netmask 255.255.255.255"

inetd_flags="-wW -a 10.0.0.1"

tcp_drop_synfin="YES"
icmp_drop_redirect="YES"
icmp_log_redirect="NO"

rpcbind_enable="NO"
cron_enable="YES"
cron_flags="$cron_flags -J 15"

sshd_enable="YES"

ip6addrctl_enable="NO"
ip6addrctl_policy="ipv4_prefer"
ipv6_activate_all_interfaces="NO"
auto_linklocal="NO"

virecover_enable="NO"
haproxy_enable="YES"

put this in the hosts rc.conf, and install sshguard in jail too.
Code:
# The boom-boom:  I thank SirDice for this one too.  I love watching it work.  Hope I did not mess up the pasting.
#
sshguard_watch_logs="/mnt/z/proxy/var/log/auth.log:/mnt/z/site1/var/log/auth.log:/mnt/z/site2/var/log/auth.log"

hosts
Code:
#::1                    localhost
127.0.0.1               localhost
10.0.0.1        proxy proxy.ha.cell
#
96.47.72.71             pkg.freebsd.org pkg

periodic.conf
Code:
# ........................................  ls -l /etc/periodic/*
# ........................................  self-create, touch and chmod 600
# ........................................ /etc/periodic.conf
daily_clean_hoststat_enable="NO"
daily_status_mail_rejects_enable="NO"
daily_status_include_submit_mailq="NO"
daily_submit_queuerun="NO"
#
##......................................................................  max21 search
#
daily_backup_pkgdb_enable="NO"
daily_status_mailq_enable="NO"

hourly_output="root"                # user or /file
weekly_show_success="NO"            # scripts returning 0
weekly_show_info="YES"                # scripts returning 1
weekly_show_badconfig="NO"            # scripts returning 2
.
.
.
 
Last edited:
Back
Top