How to install package(s), which needs initial answers, unattended !?

Hello,

I am writing a script to automatically install and configure some programs (in a jail). For most packages that is no problem.
- pkg install -y <package name>
- automatically editing config files etc, can be done as well
- pkg install -y <package name next package>
- etc

However there are packages which needs answers after starting pkg install -y <that package>. As example
Code:
# pkg install -y mysql80-server
# lot of text .. followed by.
# Starting mysql
# mysql is running as pid 3748.
# Enter password:
Tja and there I have the problem. I would like to finish the mysql installation ^unattended^ so the minimal things I have to do is:
- Enter the password (which is just <CR>)
^CR^
- than enter the mysql shell, I need to provide there the (temporarily) root password, with a command like
^ALTER USER 'root' IDENTIFIED BY 'MySQLRootPW';<CR>^
- than I have to leave the mysql shell
^exit<CR>

And then the shell (in my case zsh) should continue installing the next package^
Code:
# pkg install -y nano (as example)
# etc
This kind of problem, is of cause not specific for mysql, but will occur in other cases as well.

I have been looking around for all kind of solutions:
- using ^yes^
- using echo <the answer> | <the actual script> (permission denied, and is it a good idea any how)
- considered the use of ^expect^ but I do not know how to use that in combination with pkg
(and also not in combination with the overall install script).
- perl ???
- some other tool??

Hopefully there are decend / clean methods. Suggestion and examples whould be highly appreciated!

Louis
 
However there are packages which needs answers after starting pkg install -y <that package>. As example
Code:
# pkg install -y mysql80-server
# lot of text .. followed by.
# Starting mysql
# mysql is running as pid 3748.
# Enter password:
It's not the installation of the package that triggers this.

Code:
root@fbsd-test:~ # pkg install -y databases/mysql80-server
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        mysql80-client: 8.0.27 [FreeBSD]
        mysql80-server: 8.0.27 [FreeBSD]

Number of packages to be installed: 2

The process will require 277 MiB more space.
[1/2] Installing mysql80-client-8.0.27...
[1/2] Extracting mysql80-client-8.0.27: 100%
[2/2] Installing mysql80-server-8.0.27...
===> Creating groups.
Using existing group 'mysql'.
===> Creating users
Using existing user 'mysql'.
===> Creating homedir(s)
[2/2] Extracting mysql80-server-8.0.27: 100%
=====
Message from mysql80-client-8.0.27:

--
This is the mysql CLIENT without the server.
for complete server and client, please install databases/mysql80-server
=====
Message from mysql80-server-8.0.27:

--
There is no initial password for first time use of MySQL.
Keep in mind to reset it to a secure password.

MySQL80 has a default /usr/local/etc/mysql/my.cnf,
remember to replace it with your own
or set `mysql_optfile="$YOUR_CNF_FILE` in rc.conf.
root@fbsd-test:~ #
As you can see, nothing is started. Nothing is asked. Even starting the service for the first time doesn't ask anything:
Code:
root@fbsd-test:~ # sysrc mysql_enable="YES"
mysql_enable:  -> YES
root@fbsd-test:~ # service mysql-server start
Starting mysql.
root@fbsd-test:~ #
 
Hum,

AT the botum of this replay the output related to my server if I execute ^pkg install -y mysql80-server^ .... which is different.
Note that I am trying to setup jails in a TrueNas core system having FreeBSD 12.2-RELEASE-p11

..... I do not know how to prevent this mysql dialog. But even if I could! I would have to solved the problem, being

1) I can configure MySQL from the command line, as long as there is a root password. Using commandlines like
^mysql -u root -p$rootpassword -Bse "command1;command2,commandn;"^
However that does not work if the root password is still empty ...... :'‑(

2) Further on, situations like this are not unique for mysql ....

Below the output I get
Code:
verdeler# zsh <scriptname>.sh
Updating FreeBSD repository catalogue...
[verdeler] Fetching packagesite.pkg: 100%    6 MiB   6.7MB/s    00:01
Processing entries: 100%
FreeBSD repository update completed. 31246 packages processed.
All repositories are up to date.
The following 13 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        cyrus-sasl: 2.1.27_2
        groff: 1.22.4_4
        icu: 69.1,1
        libedit: 3.1.20210216,1
        libevent: 2.1.12
        libpaper: 1.1.24.4
        libunwind: 20201110
        mysql80-client: 8.0.26
        mysql80-server: 8.0.26
        openldap24-client: 2.4.59_2
        protobuf: 3.17.3,1
        psutils: 1.17_5
        uchardet: 0.0.7

Number of packages to be installed: 13

The process will require 403 MiB more space.
40 MiB to be downloaded.

Proceed with this action? [y/N]: y
[verdeler] [1/13] Fetching mysql80-server-8.0.26.pkg: 100%   18 MiB  18.4MB/s    00:01
[verdeler] [2/13] Fetching groff-1.22.4_4.pkg: 100%    3 MiB   2.9MB/s    00:01
[verdeler] [3/13] Fetching uchardet-0.0.7.pkg: 100%  108 KiB 111.0kB/s    00:01
[verdeler] [4/13] Fetching psutils-1.17_5.pkg: 100%   58 KiB  59.8kB/s    00:01
[verdeler] [5/13] Fetching libpaper-1.1.24.4.pkg: 100%   24 KiB  24.6kB/s    00:01
[verdeler] [6/13] Fetching protobuf-3.17.3,1.pkg: 100%    3 MiB   2.9MB/s    00:01
[verdeler] [7/13] Fetching libunwind-20201110.pkg: 100%  128 KiB 130.6kB/s    00:01
[verdeler] [8/13] Fetching libevent-2.1.12.pkg: 100%  320 KiB 327.6kB/s    00:01
[verdeler] [9/13] Fetching libedit-3.1.20210216,1.pkg: 100%  135 KiB 138.4kB/s    00:01
[verdeler] [10/13] Fetching icu-69.1,1.pkg: 100%   10 MiB  10.9MB/s    00:01
[verdeler] [11/13] Fetching mysql80-client-8.0.26.pkg: 100%    4 MiB   4.1MB/s    00:01
[verdeler] [12/13] Fetching cyrus-sasl-2.1.27_2.pkg: 100%  960 KiB 982.7kB/s    00:01
[verdeler] [13/13] Fetching openldap24-client-2.4.59_2.pkg: 100%    1 MiB   1.1MB/s    00:01
Checking integrity... done (0 conflicting)
[verdeler] [1/13] Installing libpaper-1.1.24.4...
[verdeler] [1/13] Extracting libpaper-1.1.24.4: 100%
[verdeler] [2/13] Installing uchardet-0.0.7...
[verdeler] [2/13] Extracting uchardet-0.0.7: 100%
[verdeler] [3/13] Installing psutils-1.17_5...
[verdeler] [3/13] Extracting psutils-1.17_5: 100%
[verdeler] [4/13] Installing cyrus-sasl-2.1.27_2...
*** Added group `cyrus' (id 60)
*** Added user `cyrus' (id 60)
[verdeler] [4/13] Extracting cyrus-sasl-2.1.27_2: 100%
[verdeler] [5/13] Installing groff-1.22.4_4...
[verdeler] [5/13] Extracting groff-1.22.4_4: 100%
[verdeler] [6/13] Installing protobuf-3.17.3,1...
[verdeler] [6/13] Extracting protobuf-3.17.3,1: 100%
[verdeler] [7/13] Installing libunwind-20201110...
[verdeler] [7/13] Extracting libunwind-20201110: 100%
[verdeler] [8/13] Installing libevent-2.1.12...
[verdeler] [8/13] Extracting libevent-2.1.12: 100%
[verdeler] [9/13] Installing libedit-3.1.20210216,1...
[verdeler] [9/13] Extracting libedit-3.1.20210216,1: 100%
[verdeler] [10/13] Installing icu-69.1,1...
[verdeler] [10/13] Extracting icu-69.1,1: 100%
[verdeler] [11/13] Installing openldap24-client-2.4.59_2...
[verdeler] [11/13] Extracting openldap24-client-2.4.59_2: 100%
[verdeler] [12/13] Installing mysql80-client-8.0.26...
[verdeler] [12/13] Extracting mysql80-client-8.0.26: 100%
[verdeler] [13/13] Installing mysql80-server-8.0.26...
===> Creating groups.
Creating group 'mysql' with gid '88'.
===> Creating users
Creating user 'mysql' with uid '88'.
===> Creating homedir(s)
[verdeler] [13/13] Extracting mysql80-server-8.0.26: 100%
=====
Message from cyrus-sasl-2.1.27_2:

--
You can use sasldb2 for authentication, to add users use:

        saslpasswd2 -c username

If you want to enable SMTP AUTH with the system Sendmail, read
Sendmail.README

NOTE: This port has been compiled with a default pwcheck_method of
      auxprop.  If you want to authenticate your user by /etc/passwd,
      PAM or LDAP, install ports/security/cyrus-sasl2-saslauthd and
      set sasl_pwcheck_method to saslauthd after installing the
      Cyrus-IMAPd 2.X port.  You should also check the
      /usr/local/lib/sasl2/*.conf files for the correct
      pwcheck_method.
      If you want to use GSSAPI mechanism, install
      ports/security/cyrus-sasl2-gssapi.
      If you want to use SRP mechanism, install
      ports/security/cyrus-sasl2-srp.
      If you want to use LDAP auxprop plugin, install
      ports/security/cyrus-sasl2-ldapdb.
=====
Message from groff-1.22.4_4:

--
In order to be able to use the html driver, you need to install the following
packages:
 - ghostscript
 - netpbm
=====
Message from openldap24-client-2.4.59_2:

--
The OpenLDAP client package has been successfully installed.

Edit
  /usr/local/etc/openldap/ldap.conf
to change the system-wide client defaults.

Try `man ldap.conf' and visit the OpenLDAP FAQ-O-Matic at
  [URL]http://www.OpenLDAP.org/faq/index.cgi?file=3[/URL]
for more information.
=====
Message from mysql80-client-8.0.26:

--
This is the mysql CLIENT without the server.
for complete server and client, please install databases/mysql80-server
=====
Message from mysql80-server-8.0.26:

--
There is no initial password for first time use of MySQL.
Keep in mind to reset it to a secure password.

MySQL80 has a default /usr/local/etc/mysql/my.cnf,
remember to replace it with your own
or set `mysql_optfile="$YOUR_CNF_FILE` in rc.conf.
mysql_enable:  -> YES
mysql not running? (check /var/db/mysql/verdeler.pid).
Starting mysql.
mysql is running as pid 3748.
Enter password:
 
Tja and there I have the problem. I would like to finish the mysql installation ^unattended^ so the minimal things I have to do is:
- Enter the password (which is just <CR>)
^CR^
- than enter the mysql shell, I need to provide there the (temporarily) root password, with a command like
^ALTER USER 'root' IDENTIFIED BY 'MySQLRootPW';<CR>^
- than I have to leave the mysql shell
^exit<CR>
Not a good practice to supply the root password automatically like that. Way too easy to get careless and leak it at some point. Automation is convenient, but you gotta pay attention to what it even does. 😩
--
Easily restoring from backup is a valid argument for automating stuff, but you gotta think things through, because half-baked ideas can very well backfire.
 
I did try installation of mysql80-server and php80-mysqli again on a clean jail (I did roll back to a snapshot not yet containing mysql)
and I have to admit the I did not get the password question .... do not know why. Do not know why ... I did a lot of trying and testing .....
That does not solve my problem, but changes it a bit ???? perhaps.

I still have to find a solution to set the default root password via the shell ....
 
Not a good practice to supply the root password automatically like that. Way too easy to get careless and leak it at some point. Automation is convenient, but you gotta pay attention to what it even does. 😩
--
Easily restoring from backup is a valid argument for automating stuff, but you gotta think things through, because half-baked ideas can very well backfire.
I agree of cause and I will change the temporarily root password after the initial installation. The involved DB is also empty at that moment and is also not accessible from the internet.
 
TrueNAS is not supported here. The FreeBSD packages don't start the service automatically when you install them.

I can imagine that it was a left over test, I did earlier. Something like ^sysrc mysql_enable="YES"^.
I did a jail roll backs between tests a couple of times, however not after each try / test.
 
I can imagine that it was a left over test, I did earlier. Something like ^sysrc mysql_enable="YES"^.
That only enables the service. Installing or removing the package still isn't going to 'automagically' stop or start the service itself.
 
I still have to find a solution to set the default root password via the shell ....
I think that this manpage might help: passwd(1). Even the root account can change it's own password, IIRC. For MySQL specifically, that RDBMS also uses 'root' as the name for its admin account, so it's important to keep track of that. MySQL uses a different config file (to store the passwords) than the base system. For more details, see the mysql(1) manpage.
 
That only enables the service. Installing or removing the package still isn't going to 'automagically' stop or start the service itself.
I do not know why that happened. I did cut and paste it from the command line. I did see that multiple times, probably for the same reason.

What I do now as a ^work around^ is that I install mysql-server as first package in the script.
That is not logical, however ..... that way do not have to wait a long time, before I have to do the manual actions to define the root password!

pkg install -y mysql80-server

cd /usr/local/etc/mysql
mv my.cnf $now"my.cnf"
cp my.cnf.sample my.cnf

service mysql-server onestart
service mysql-server status
# mysql is running as pid <someno>

mysql -u root -p
> Enter <CR> (password is still ^nothing^
> ALTER USER 'root'@'localhost' IDENTIFIED BY '$MySQLRootPW';FLUSH PRIVILEGES;"<CR>
> quit<CR>

I should be possible to do that "unattended", however despite lots of google-ing and try's ..... I do not yet manage :rude:
 
I do not know why that happened. I did cut and paste it from the command line. I did see that multiple times, probably for the same reason.

What I do now as a ^work around^ is that I install mysql-server as first package in the script.
That is not logical, however ..... that way do not have to wait a long time, before I have to do the manual actions to define the root password!

pkg install -y mysql80-server

cd /usr/local/etc/mysql
mv my.cnf $now"my.cnf"
cp my.cnf.sample my.cnf

service mysql-server onestart
service mysql-server status
# mysql is running as pid <someno>

mysql -u root -p
> Enter <CR> (password is still ^nothing^
> ALTER USER 'root' IDENTIFIED BY $MySQLRootPW;FLUSH PRIVILEGES;"<CR>
> quit<CR>

I should be possible to do that "unattended", however despite lots of google-ing and try's ..... I do not yet manage :rude:
Now that I think about it a bit... it looks like it may be better to install MySQL as the very last package. Dunno how possible that is, because MySQL is specified as a dependency in lots of packages, and it's next to impossible to correct that without switching to ports. Installing MySQL as the very last package has the advantage that you can get to manual config pretty quickly when your script completes.
--
Also, it looks like your MySQL installation doesn't like empty password for admin connections. When I was playing with MySQL, setting up admin password in the RDBMS was one of the setup chores that I did not neglect.
 
Dunno how possible that is, because MySQL is specified as a dependency in lots of packages
Only the client, not the server.

But you are going to run into issues because MySQL 5.7 is the standard. So everything is going to depend on mysql57-client and that's going to conflict with mysql80-client (which is a dependency of mysql80-server).
 
No, No, (IMHO)
- People should always use the latest releases. It is IMHO a bad idea, not to update applications to the latest or minimal the latest -1 release. That would also help against: many versions to maintain, security bugs, other bugs, etc.

On my private system (or PC), every package not compatible with the actual OS release of php or whatever will be removed!

- In my opinion you should start with the base platform, which does include MySQL. And than the applications which depends on MySQL. Just to give a well know example, IMHO you should first install apache (and test that), then php (test again) then mysql (test) and then wordpress.
 
No, No, (IMHO)
- People should always use the latest releases. It is IMHO a bad idea, not to update applications to the latest or minimal the latest -1 release. That would also help against: many versions to maintain, security bugs, other bugs, etc.

On my private system (or PC), every package not compatible with the actual OS release of php or whatever will be removed!

- In my opinion you should start with the base platform, which does include MySQL. And than the applications which depends on MySQL. Just to give a well know example, IMHO you should first install apache (and test that), then php (test again) then mysql (test) and then wordpress.
🤣 I use ports, I turned MySQL off completely (switching to SQLite where needed). Oh, and I didn't bother installing Apache until I discovered that I need it for Poudriere.
--
But it's your machine, and your time and efforts. Yeah, it's nice to have a base and a way to make recovering from a disaster easy.
 
I think that this manpage might help: passwd(1). Even the root account can change it's own password, IIRC. For MySQL specifically, that RDBMS also uses 'root' as the name for its admin account, so it's important to keep track of that. MySQL uses a different config file (to store the passwords) than the base system. For more details, see the mysql(1) manpage.

Hum, yes and no beware ^root^ <> ^root^. In the given situation there is
- the root of the host computer
- the root of the jail
- the root of mysql. Which has absolutely no technical relation with the ^root^ from the ^host^ or the ^jail^. It could be the same human of course :)

To be honest how that is between ^host^ and ^jail^ I still have to discover. That the ^host-root^ has access to the Jail is clear.
Jail users, I do not yet know how to define them (I am new to jails), should not have any possibility to access any thing outside the jail

So the pw storage of the host is completely independent of the password storage of mysql.
 
People should always use the latest releases. It is IMHO a bad idea, not to update applications to the latest or minimal the latest -1 release.
MySQL 5.7 is a supported version until October 2023. And that's the only thing that matters.

In my opinion you should start with the base platform, which does include MySQL.
The base platform (FreeBSD itself) doesn't include MySQL. It's a third party application. The default MySQL is set to MySQL 5.7, you can change this default but you will need to build from ports (or set up your own repository). The packages from the FreeBSD repositories however will be set according to certain defaults (and you can't change those after the package has been created).

Just to give a well know example, IMHO you should first install apache (and test that), then php (test again) then mysql (test) and then wordpress.
Order of installation is actually completely irrelevant.
 
Back
Top