Script help

Hello guys,

For the last few day I have been trying to teach myself how to do shell scripting but I am not really successful. Could someone show me how I should run the following as part of a .sh script?

Code:
cd /usr/ports/www/nginx
make config
set [x] IPv6 protocol (default)
set [x] Enable HTTP module (default)
set [x] Enable http_cache module (default)
set [x] Enable http_gzip_static module
set [x] Enable http_rewrite module (default)
set [x] Enable http_ssl module (default)
set [x] Enable http_stub_status module (default)
make install clean; rehash

mkdir -p /etc/nginx/sites-enabled/
mkdir -p /etc/nginx/sites-enabled/
mkdir -p /var/customers/webs/
mkdir -p /var/customers/tmp
mkdir -p /var/customers/logs/
echo 'nginx_enable="YES"' >> /etc/rc.conf

vi /usr/local/etc/nginx/nginx.conf
------------------------------------- ****** -------------------------------------
### Nginx configuration file ###

daemon on;
user www www;
worker_processes 4;

error_log /var/log/nginx/nginx.log crit;
pid /var/run/nginx.pid;


events {
	use	kqueue;
	worker_connections	64;
	accept_mutex_delay	50ms;
}


http {

	access_log	/var/log/nginx/access.log;

	include	mime.types;
	default_type	application/octet-stream;
	charset	utf-8;

	sendfile	on;
	tcp_nopush	on;
	tcp_nodelay	on;

	# Open File Cache
	open_file_cache	max=512 inactive=20m;
	open_file_cache_valid	60s;
	open_file_cache_errors	on;

	# FastCGI Cache
	fastcgi_cache_key	"$scheme$request_method$host$request_uri";
	fastcgi_cache_path	/var/tmp/nginx levels=1:2 keys_zone=cache:10m inactive=20m max_size=12m;
	fastcgi_temp_path	/var/tmp/nginx 1 2;
	fastcgi_cache	cache;
	fastcgi_cache_valid	200	20m;
	fastcgi_cache_valid	301 302 304	1h;
	fastcgi_cache_valid	any	1m;
	fastcgi_cache_use_stale	error timeout invalid_header http_500;

	# SSL
	#ssl_client_certificate	/path/to/example.pem;
	#ssl_crl	/path/to/example.crl;
	ssl_protocols	TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers	HIGH;
	ssl_prefer_server_ciphers	on;
	ssl_session_cache	builtin:256 shared:SSL:64k;
	ssl_session_timeout	20m;

	# GZip
	gzip	on;
	gzip_vary	on;
	gzip_comp_level	6;
	gzip_static	on;

	include /usr/local/etc/nginx/conf.d/*.conf;

}
------------------------------------- ****** -------------------------------------
/usr/local/etc/rc.d/nginx restart

So far this is what I came up with:
Code:
#!/bin/sh
#
# 
#     nginx.sh @ Version 0.10

echo "# install Nginx from port"
cd /usr/ports/www/nginx
make config

?? need help on how to select my options??
echo "# create nginx directory"
mkdir -p /etc/nginx/sites-enabled/
mkdir -p /etc/nginx/sites-enabled/
mkdir -p /var/customers/webs/
mkdir -p /var/customers/tmp
mkdir -p /var/customers/logs/

echo "# start Nginx at start up"

echo 'nginx_enable="YES"' >> /etc/rc.conf

cat << 'EOF' > /usr/local/etc/nginx/nginx.conf
### Nginx configuration file ###

daemon on;
user www www;
worker_processes 4;

error_log /var/log/nginx/nginx.log crit;
pid /var/run/nginx.pid;


events {
	use	kqueue;
	worker_connections	64;
	accept_mutex_delay	50ms;
}


http {

	access_log	/var/log/nginx/access.log;

	include	mime.types;
	default_type	application/octet-stream;
	charset	utf-8;

	sendfile	on;
	tcp_nopush	on;
	tcp_nodelay	on;

	# Open File Cache
	open_file_cache	max=512 inactive=20m;
	open_file_cache_valid	60s;
	open_file_cache_errors	on;

	# FastCGI Cache
	fastcgi_cache_key	"$scheme$request_method$host$request_uri";
	fastcgi_cache_path	/var/tmp/nginx levels=1:2 keys_zone=cache:10m inactive=20m max_size=12m;
	fastcgi_temp_path	/var/tmp/nginx 1 2;
	fastcgi_cache	cache;
	fastcgi_cache_valid	200	20m;
	fastcgi_cache_valid	301 302 304	1h;
	fastcgi_cache_valid	any	1m;
	fastcgi_cache_use_stale	error timeout invalid_header http_500;

	# SSL
	#ssl_client_certificate	/path/to/example.pem;
	#ssl_crl	/path/to/example.crl;
	ssl_protocols	TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers	HIGH;
	ssl_prefer_server_ciphers	on;
	ssl_session_cache	builtin:256 shared:SSL:64k;
	ssl_session_timeout	20m;

	# GZip
	gzip	on;
	gzip_vary	on;
	gzip_comp_level	6;
	gzip_static	on;

	include /usr/local/etc/nginx/conf.d/*.conf;

}
EOF

echo "# restart Nginx for the change to take effect"
/usr/local/etc/rc.d/nginx restart

sleep 3


Then I have more code....

Thank you all in advance.

Fred
 
I have a hard time understanding what it is you're exactly trying to do. My assumption is that you want to set a specific configuration for www/nginx and then install it.

For starters, even if you're going to "play" with Ports then it's still highly advisable to read the porters handbook. At least it gives you a good idea how this whole thing works.

In this case I'd forget all about # make config and instead set the options yourself. This could be a bit tricky, but when building a port you can use the BATCH build option, from the ports(7) manual page:

Code:
     BATCH         If defined, only operate on a port if it can be installed
                   100% automatically.
So the only thing left to do is to put all the options you need into /etc/make.conf. Then simply install the port and it will create the configuration ("options") file itself without any required interaction.
 
Hey guys,

Thank you for your reply. I've got to admit that you both lost me a bit. To clarify, what I want to do is set Nginx automatically via a script. Normally I would run each command manually. Just wonder how to do it via script. Could you guys give a me sample to work on?

Thank you
 
gpatrick said:
I was replying to your comment of


Because I thought you wanted to select one of those options, but looking again, I'm inclined to agree I don't know what you're trying to do.


I want to select all of the options. These are the option you can select when you install Nginx.

Thank you for your help :)
 
fred974 said:
To clarify, what I want to do is set Nginx automaticaly via a script.
In this case you need to use the previously mentioned BATCH build parameter which will make sure that the port will be build without any interaction.

All the required options need to be set in /etc/make.conf as I mentioned before.

fred974 said:
Just wonder how to do it via script.

Could you guys give a me sample to work on?
First you need to check which options can be set for the port you're working on, you can do so by checking out its Makefile. So, in case of www/nginx:

Code:
OPTIONS_DEFINE= \
        DEBUG \
        DEBUGLOG \
        FILE_AIO \
        IPV6 \
These are some of the options which can be set. Running # make config will confirm this. Let's concentrate on IPv6.

Another important aspect here is OPTIONS_DEFAULT, for obvious reasons. Because IPv6 is on by default I'm going to turn it off, should make a good example. Another important thing to keep in mind is bsd.options.mk. This is where you'll find a list of all variables which you can use (either in a ports Makefile, in /etc/make.conf or on the commandline (who said Ports weren't extensive? ]grep -A12 20130614 /usr/ports/UPDATING[/cmd].

So, turning IPv6 off...

Looking into bsd.options.mk gets you this:

Code:
# ${OPTIONS_NAME}_UNSET         - List of options to disable for a specific port.
And now for a bit of trial and error (or, in other words, good peeking ;)).

Run # make config for www/nginx (don't forget to use 'OK' to save the options) and go over /var/db/ports/www_nginx/options. Here I found this:

Code:
OPTIONS_FILE_SET+=IPV6
That tells me that I can easily use OPTIONS_FILE_UNSET=IPV6 in /etc/make.conf.

However, you want to set a specific list of options so I'll take things a bit further and instead of simply de-selecting IPv6 I'll define the whole lot of options I want (which is basically the default without IPv6):

Code:
OPTIONS_FILE_SET=HTTP HTTP_CACHE HTTP_REWRITE HTTP_STATUS WWW
OPTIONS_FILE_UNSET=IPV6
So what did I do here?

I took a look at OPTIONS_DEFAULT and put the whole list into OPTIONS_FILE_UNSET. Then I concentrated on the stuff I didn't want (IPv6) and moved all the other options into OPTIONS_FILE_SET.

Now all that's left to do is build or install www/nginx. For example by using: # make -C /usr/ports/www/nginx BATCH=yes install.

Oh, also important; how to get those 2 lines into /etc/make.conf and out of it again...

There are dozens of ways to do that, my approach would be something like this:

Code:
#!/bin/sh

## Add stuff to /etc/make.conf

echo "OPTIONS_FILE_SET=HTTP HTTP_CACHE HTTP_REWRITE HTTP_STATUS WWW" >> /etc/make.conf
echo "OPTIONS_FILE_UNSET=IPV6" >> /etc/make.conf
Getting this out again could be tricky because if this script is supposed to be run on other environments then you can't be sure that OPTIONS_FILE is something explicitly set by you. What you could do is simply cut the last 2 lines from make.conf. Which could be another liability itself (though somewhat neglectable) because in theory someone could have edited make.conf between inserting and removing.

Alas; you'll have to take an 'I owe you' on this because I approached this problem using tail and head and once again realized how far Linux has diverted me from straight POSIX. That's a "cool" way of saying "I'll need to dive into sed again to get this sorted out".

But I'm going to get back to you on that one, I love small problems like that.

Hope this can help you out somewhat.
 
Use the command-line version of the options instead of modifying the global /etc/make.conf:
make BATCH=yes OPTIONS_FILE_SET="HTTP HTTP_CACHE HTTP_REWRITE HTTP_STATUS WWW"
 
Plain and simple: that was simply stupid of me to completely overlook the possibility to specify those build options on the commandline.

Ignore my comments on make.conf, but I do suggest to keep the rest in mind since that could help whenever you're dealing with other ports.

And as a side note; I'm still going to look into the best option to remove the last two lines from a text file ;)
 
ShelLuser said:
And as a side note; I'm still going to look into the best option to remove the last 2 lines from a text file ;)

$ ( sed -e '$d' | sed -e '$d' ) < x.txt
or
$ count=$(cat x.txt | wc -l) ; head -$((count-2)) x.txt

Please define 'best' ;)
 
Wow @ShelLuser,

Thank you for taking the time to help me out :) I can't say that I got it all but then again, I need to go away and play/researched what you just explained. I'm sure as I play it it it will all fall into place :)

Just one more thing, is the way I used the
Code:
cat << 'EOF' >
correct?

Fred
 
Last edited by a moderator:
I'm trying to do the same as OP (select port options non-interactively).

When I run make BATCH=yes OPTIONS_FILE_SET=HEADERS_MORE and then make showconfig the output tells me [BGCOLOR=rgb(0, 0, 0)]HEADERS_MORE=off: 3rd party headers_more module[/BGCOLOR]

How come?
 
It is official: I might be psychic, because I somehow foresaw this post being posted! ;)

Ok, more seriously now:

When I run make BATCH=yes OPTIONS_FILE_SET=HEADERS_MORE and then make showconfig the output tells me HEADERS_MORE=off: 3rd party headers_more module
Which port are you trying to alter, and are you sure you used the right options?

Anyway, there has obviously been a misunderstanding on my part (now referring to your PM) because I can reproduce the issue, but I also have a valid explanation ;)

First: although OPTIONS_FILE_SET still works it's probably better (and easier (less typing)) to use OPTIONS_SET and/or OPTIONS_UNSET instead. So the same things you'd set up in /etc/make.conf.

Code:
root@psi:/usr/ports/ports-mgmt/poudriere # make showconfig
===> The following configuration options are available for poudriere-3.2.7:
     CERTS=on: Install checksum and SSL certificates for jail creation
     DIALOG4PORTS=on: Install dialog4ports for options command
     EXAMPLES=on: Build and/or install examples
     QEMU=off: Add qemu-user-static for non-x86 architectures
     ZSH=on: Install programmable completions for zsh
===> Use 'make config' to modify these settings
What I answered you in PM was wrong (sorry about that, this is also a reason why I prefer using the forums): changes applied through OPTIONS_(UN)SET are not automatically saved (which I assumed), but only apply to that specific building session. This is especially true if you use the BATCH option.

For example... (using ports-mgmt/poudriere as shown above):

If I run: # make BATCH=yes OPTIONS_UNSET+=EXAMPLES extract then nothing changes, so if I run # make showconfig again all the option remain as they were above.

However... when I actually build the port using these parameters then you will notice that the eventual package gets build without the EXAMPLES.

Just try running: # make OPTIONS_UNSET="EXAMPLES" extract and now you'll be taken to the options configuration where EXAMPLES is already deselected. Now you can actually save the changes if you want. This section gets skipped in the previous examples because of the BATCH setting.

For example:

Code:
root@psi:/usr/ports/ports-mgmt/poudriere # make BATCH=yes package > build.log
root@psi:/usr/ports/ports-mgmt/poudriere # cp work/pkg/poudriere-3.2.7.txz .
root@psi:/usr/ports/ports-mgmt/poudriere # make BATCH=yes OPTIONS_UNSET="EXAMPLES" clean package > build.log
root@psi:/usr/ports/ports-mgmt/poudriere # cp work/p
pkg/             poudriere-3.2.7/
root@psi:/usr/ports/ports-mgmt/poudriere # cp work/pkg/poudriere-3.2.7.txz ./poudriere2.txz
root@psi:/usr/ports/ports-mgmt/poudriere # ls -lh *txz
-rw-r--r--  1 root  wheel   693K Aug 13 22:44 poudriere-3.2.7.txz
-rw-r--r--  1 root  wheel   692K Aug 13 22:47 poudriere2.txz
root@psi:/usr/ports/ports-mgmt/poudriere #
Notice how both packages differ? That's purely because I didn't include the examples in the second run.
 
There is some new stuff in the ports system that you can use to check which options are selected and which are unselected:
Code:
make -V SELECTED_OPTIONS WITH=FOO WITHOUT=BAR
make -V DESELECTED_OPTIONS OPTIONS_SET+=FOO OPTIONS_UNSET+=BAR

I showed there both ways of overriding options on the command line, WITH/WITHOUT and OPTIONS_SET/OPTIONS_UNSET. The make -V invocation shows only the result of the variable expansion and doesn't execute any targets.
 
I think OPTIONS_UNSET doesn't actually work on the command line, at least what it shows for me:

Code:
% make -V DESELECTED_OPTIONS WITHOUT=ZSH  
QEMU ZSH

Code:
make -V DESELECTED_OPTIONS OPTIONS_UNSET=ZSH
QEMU

That's for the ports-mgmt/poudriere-devel port where the ZSH option installs the ZSH completions and I have the option set in my make.conf.
 
The source code is very clear on the matter, from /usr/ports/Mk/bsd.options.mk:

Code:
# These variables can be used in make.conf to configure options.  They are
# processed in the order listed below, i.e. later variables override the effects
# of previous variables.  Options saved using the options dialog are processed
# right before OPTIONS_SET_FORCE.  When building a port a dialog to configure
# options will only appear if there are new options, i.e. options which have not
# been configured before either using the option dialog in a previous build or
# using the variables below.  You can force the dialog to appear by running
# "make config".
#
# OPTIONS_SET                   - List of options to enable for all ports.
# OPTIONS_UNSET                 - List of options to disable for all ports.
# ${OPTIONS_NAME}_SET           - List of options to enable for a specific port.
# ${OPTIONS_NAME}_UNSET         - List of options to disable for a specific port.
#
# OPTIONS_SET_FORCE             - List of options to enable for all ports.
# OPTIONS_UNSET_FORCE           - List of options to disable for all ports.
# ${OPTIONS_NAME}_SET_FORCE     - List of options to enable for a specific port.
# ${OPTIONS_NAME}_UNSET_FORCE
#                               - List of options to disable for a specific port.
#
# These variables can be used on the command line. They override the effects of
# the make.conf variables above.
#
# WITH                          - Set options from the command line
# WITHOUT                       - Unset options from the command line

Sorry for the long quoted code but that explains the details well enough I think.
 
It helps for clarity sake to edit your posts, but apart from that: the option does work as shown above. If it didn't you also wouldn't be able to use it in /etc/make.conf.
 
It turns out that OPTIONS_UNSET on the command line doesn't override the port specific setting ports-mgmt_poudriere-devel_SET that is set in a make.conf file. If you use the command line to override the port specific option it's fine:

Code:
% make -V DESELECTED_OPTIONS ports-mgmt_poudriere-devel_UNSET=ZSH
QEMU ZSH

But again:
Code:
% make -V DESELECTED_OPTIONS OPTIONS_UNSET=ZSH
QEMU

As a conclusion, if you have to override selected/unselected options on the command line, use the WITH=FOO WITHOUT=BAR syntax because that seems to be the supported way (that's what the source says), everything else is suspect to break depending on your environment.
 
ShelLuser

This took me a while to digest but your explanation makes a lot of sense. After reading your post I was able to get it working (also I'll be sure not to PM you anymore :p)

kpa

If that is the recommended syntax that is what I will put in my little script.

Thanks guys !!
 
Back
Top