HOWTO: Puppet and Puppet Dashboard on top of Nginx/Passenger

junovitch@

Developer
CONTENTS
  1. Intro
  2. Assumptions
  3. MariaDB Database Jail Prep
  4. Creating and Configurating a MariaDB Database for Puppet Dashboard


Intro
I've been using Puppet for most of this year using the standard sysutils/puppet with its included WEBrick server. While that works for me, I was interested in having the Puppet Dashboard and figured it would be an excellent opportunity to work with something new and learn something in the process. My goal here is to take what I have learned and cover how to set up a production quality Puppetmaster/Puppet Dashboard installation from scratch. Many of the steps will follow along closely with Puppet Lab's online instructions so I would encourage you to read along with their detailed explanation as you do the steps.


Assumptions
I've started with a pair of bare FreeBSD 9.2-RELEASE VMs to test out the procedure on to ensure I didn't miss anything from my live setup. We'll need to do a few things to stage our environment. Our server VM will also run both the Dashboard and Puppet master. With enough clients this may have to be on different systems. Much of the same configuration will still apply. Adapt accordingly if this doesn't fit your environment.

For these two VMs, I'll use the following /etc/hosts file. Ensure that DNS resolves appropriately in a live environment.
Code:
10.100.82.10		client.example.com client
10.100.82.2		jailhost.example.com jailhost
10.100.82.3		mariadb.example.com mariadb
10.100.82.4		puppet.example.com dashboard.example.com puppet dashboard

On jailhost.example.com we'll set up a basic environment:
pkg_add -r ezjail
ezjail-admin install
ezjail-admin update -P


MariaDB Database Jail Prep
databases/mariadb55-server will handle the back end requirements of Puppet Dashboard. MySQL can also be used as well.

Configure the jail on jailhost.example.com:
ezjail-admin create [highlight]mariadb.example.com 10.100.82.3[/highlight]
cp /etc/resolv.conf /usr/jails/[highlight]mariadb.example.com/etc/[/highlight]
cp /etc/hosts /usr/jails/[highlight]mariadb.example.com/etc/[/highlight]
ezjail-admin console -f [highlight]mariadb.example.com[/highlight]

On mariadb.example.com:
pkg_add -r portmaster
portmaster databases/mariadb55-server


Creating and Configurating MariaDB Database for Puppet Dashboard
http://docs.puppetlabs.com/dashboar...tml#creating-and-configuring-a-mysql-database

Tuning: Copy one of the default config files and change the max packet size to allow for the 17 MB data rows that Dashboard can occasionally send.
Code:
cp /usr/local/share/mysql/[highlight]my-huge.cnf[/highlight] /var/db/mysql/my.cnf
patch /var/db/mysql/my.cnf << 'EOF'
32c32,33
< max_allowed_packet = 1M
---
> # Allow 32MB packet Size for ~17MB size rows Puppet dashboard sends
> max_allowed_packet = 32M
'EOF'

Enable and start MariaDB
echo 'mysql_enable="YES"' >> /etc/rc.conf
service mysql-server start

Prepare Database for use by running the secure installation.
Choose a root password and answer yes to all questions.
mysql_secure_installation

Login to MariaDB and create appropriate tables for Dashboard.
mysql -u root -p
Code:
CREATE DATABASE dashboard_production CHARACTER SET utf8;
CREATE DATABASE dashboard_development CHARACTER SET utf8;
CREATE DATABASE dashboard_test CHARACTER SET utf8;
CREATE USER 'dashboard'@'[highlight]10.100.82.4[/highlight]' IDENTIFIED BY '[highlight]dashboard_password[/highlight]';
GRANT ALL PRIVILEGES ON dashboard_production.* TO 'dashboard'@'[highlight]10.100.82.4[/highlight]';
GRANT ALL PRIVILEGES ON dashboard_development.* TO 'dashboard'@'[highlight]10.100.82.4[/highlight]';
GRANT ALL PRIVILEGES ON dashboard_test.* TO 'dashboard'@'[highlight]10.100.82.4[/highlight]';
flush privileges;
quit;
 
CONTENTS
  1. Puppet Jail Prep
  2. Installing Dependencies: Install Puppet, Git, Nginx, and Ruby Components
  3. Puppet Initial Testing
  4. Puppet Dashboard Installation
  5. Puppet Dashboard Configuration
  6. Testing That Dashboard is Working
  7. Configuring Puppet
  8. Starting and Managing Delayed Job Workers


Puppet Jail Prep

On jailhost.example.com we'll set up a basic environment:
ezjail-admin create [highlight]puppet.example.com 10.100.82.4[/highlight]
cp /etc/hosts /usr/jails/[highlight]puppet.example.com/etc/[/highlight]
cp /etc/resolv.conf /usr/jails/[highlight]puppet.example.com/etc/[/highlight]
ezjail-admin console -f [highlight]puppet.example.com[/highlight]


Installing Dependencies: Install Puppet, Git, Nginx, and Ruby Components
http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#installing-dependencies
echo 'rubygem-passenger_UNSET+=APACHE22' >> /etc/make.conf
echo 'rubygem-passenger_SET+=NGINX' >> /etc/make.conf
echo 'nginx_SET+=PASSENGER HTTP_SSL' >> /etc/make.conf
echo 'DEFAULT_VERSIONS= ruby=1.9' >> /etc/make.conf
pkg_add -r portmaster
portmaster sysutils/puppet sysutils/puppet-lint devel/git www/nginx devel/ruby-gems devel/rubygem-rake converters/ruby-iconv www/rubygem-passenger sysutils/rubygem-bundler databases/mysql55-client textproc/libxslt www/node databases/sqlite3


Puppet Initial Testing
At this point, Puppet needs to be started so that all its SSL keys can be generated. This gives the chance to test that Puppet does work before anything else gets stacked on as well as ensures the SSL keys referenced by Nginx's config file are in place before that step.

service puppetmaster onestart

On client.example.com - start Puppet on the client system
service puppet onestart

On puppet.example.com - sign client.example.com's SSL key on the Puppetmaster
puppet cert sign client.example.com

On client.example.com - Run a test on the client to ensure it works and do a onestop afterwards
puppet agent --test
service puppet onestop


Puppet Dashboard Installation
http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#installing-puppet-dashboard

Installing from Git
cd /usr/local/share
git clone [url=git://github.com/sodabrew/puppet-dashboard.git]git://github.com/sodabrew/puppet-dashboard.git[/url]

Manually create the 'puppet-dashboard' user and provide required permissions.
pw groupadd -n puppet-dashboard -g 800
pw useradd -n puppet-dashboard -c "Puppet Dashboard,,," -u 800 -g puppet-dashboard -s /usr/sbin/nologin
chown -R puppet-dashboard:puppet-dashboard /usr/local/share/puppet-dashboard


Puppet Dashboard Configuration
http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#configuring-dashboard
database.yml: Copy the example database YAML file. Update with database information.

cd /usr/local/share/puppet-dashboard/config
cp database.yml.example database.yml
Code:
patch database.yml << 'EOF'
49c49
<   password:
---
>   password: [highlight]dashboard_password[/highlight]
51c51,52
<   adapter: postgresql
---
>   adapter: mysql2
>   host: [highlight]10.100.82.3[/highlight]
56c57
<   password:
---
>   password: [highlight]dashboard_password[/highlight]
58c59,60
<   adapter: postgresql
---
>   adapter: mysql2
>   host: [highlight]10.100.82.3[/highlight]
63c65
<   password:
---
>   password: [highlight]dashboard_password[/highlight]
65,66c67,68
<   adapter: postgresql
<
---
>   adapter: mysql2
>   host: [highlight]10.100.82.3[/highlight]
51a52
'EOF'
chown puppet-dashboard:puppet-dashboard database.yml
chmod 660 database.yml

settings.yml: Copy the example settings YAML file. No changes needed.
cd /usr/local/share/puppet-dashboard/config
cp settings.yml.example settings.yml
chown puppet-dashboard:puppet-dashboard settings.yml
chmod 660 settings.yml

Fix shebang line in External Node Classifier Script.
sed -i '' -e 's/#! \/usr\/bin\/ruby/#!\/usr\/local\/bin\/ruby/' /usr/local/share/puppet-dashboard/bin/external_node

Install gems required in the 'Gemfile' via the Rubygem Bundler. If the postgresql gem bundles is required additional dependencies are needed.
cd /usr/local/share/puppet-dashboard
bundle install --path vendor/bundle --without postgresql

Generate secret_token. Cleanup any errors and the default token after generating the new one.
echo "secret_token: `bundle exec rake secret`" >> config/settings.yml
vi config/settings.yml

Creating and Configuring a MariaDB Database - Preparing Schema
http://docs.puppetlabs.com/dashboar...tml#creating-and-configuring-a-mysql-database
At this point the database was already installed in another jail with some blank tables. We need to run rake to finish the process with the database structure needed.
cd /usr/local/share/puppet-dashboard
env RAILS_ENV=production bundle exec rake db:setup
env RAILS_ENV=development bundle exec rake db:setup


Testing That Dashboard is Working
http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#testing-that-dashboard-is-working

Run Dashboard using Ruby's built-in WEBrick server to validate functionality. It will be available at http://dashboard:3000
cd /usr/local/share/puppet-dashboard
su -m puppet-dashboard -c 'bundle exec rails server'

Before going into a production environment, Dashboard 2.0 must precompile assets for production.
env RAILS_ENV=production bundle exec rake assets:precompile

Chown any files created up until now to the right owner.
chown -R puppet-dashboard:puppet-dashboard /usr/local/share/puppet-dashboard


Configuring Puppet
http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#configuring-puppet
Configuring Puppet
All agent nodes have to be configured to submit reports to the master. The master has to be configured to send reports to Dashboard. If you already have a working Puppet installation you can configure it to distribute the updated puppet.conf to your hosts.

Examples:
puppet.conf (on each agent)
Code:
[agent]
  report = true

puppet.conf (on the Puppetmaster)
Code:
[master]
  reports = store, http
  reporturl = http://[highlight]dashboard.example.com:3000[/highlight]/reports/upload
  node_terminus = exec
  external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=[highlight]http://dashboard.example.com:3000[/highlight] /usr/local/share/puppet-dashboard/bin/external_node

Testing Puppet's Connection to Dashboard
From a Puppet agent, run puppet agent --test. A new background task should show in the Dashboard UI at http://dashboard:3000


Starting and Managing Delayed Job Workers
http://docs.puppetlabs.com/dashboar...tml#starting-and-managing-delayed-job-workers
Using the monitor script
Dashboard ships a worker process maanager under script/delayed_job. It can manually start delayed jobs via the following command:
su -m puppet-dashboard -c 'env RAILS_ENV=production bundle exec script/delayed_job -p dashboard -n 2 -m start'

However, rather than manually triggering background workers, this rc script will accomplish the same thing and ensure the background jobs get started on the next reboot.

Code:
cat > /usr/local/etc/rc.d/dashboard_workers << 'EOF'
#!/bin/sh

# PROVIDE: dashboard_workers
# REQUIRE: LOGIN
# KEYWORD: shutdown

# By default dashboard_workers uses flags '-n 1' for 1 worker.  This should be 
# adjusted to the number of CPU cores.
dashboard_workers_enable=${dashboard_workers_enable:-"NO"}
dashboard_workers_flags=${dashboard_workers_flags:-"-n 1"}
# The default rails environment is set to production
dashboard_workers_env=${dashboard_workers_env:-"/usr/bin/env PATH=${PATH}:/usr/local/bin RAILS_ENV=production"}
# The default user is set to puppet-dashboard and install location is set to
# /usr/local/share/puppet-dashboard.
dashboard_workers_user=${dashboard_workers_user:-"puppet-dashboard"}
dashboard_workers_chdir=${dashboard_workers_chdir:-"/usr/local/share/puppet-dashboard"}

. /etc/rc.subr

name="dashboard_workers"
rcvar="dashboard_workers_enable"
load_rc_config $name
extra_commands="reload run zap status"

# All commands call the same function and strip the fast|one|quiet prefix
# to deliver to the bundler.
reload_cmd="f_dashboard_workers reload"
restart_cmd="f_dashboard_workers restart"
run_cmd="f_dashboard_workers run"
start_cmd="f_dashboard_workers start"
status_cmd="f_dashboard_workers status"
stop_cmd="f_dashboard_workers stop"
zap_cmd="f_dashboard_workers zap"

# Use the function's ARVG $1 as the bundler program's '-m' flag
f_dashboard_workers() {
    cd $dashboard_workers_chdir && \
    su -m "$dashboard_workers_user" \
        -c "${dashboard_workers_env} bundle exec script/delayed_job ${rc_flags} -p dashboard -m $1" || \
    echo "Failed to $1 dashboard_workers"
}

run_rc_command "$1"
'EOF'
chmod 555 /usr/local/etc/rc.d/dashboard_workers

With that in place, we need to override the defaults and enable the script along with setting '-n 4' workers to match the number of processor cores and ensure it's ready for a production workload.

echo 'dashboard_workers_enable="YES"' >> /etc/rc.conf
echo 'dashboard_workers_flags="-n 4"' >> /etc/rc.conf
service dashboard_workers start
 
CONTENTS
  1. Running Dashboard in a Production-Quality Server (Nginx/Passenger)
  2. Configuring Dashboard - Advanced Features
  3. Conclusion and To Do List


Running Dashboard in a Production-Quality Server (Nginx/Passenger)
http://docs.puppetlabs.com/dashboar...ning-dashboard-in-a-production-quality-server

Configuring Nginx and Passenger
Since Puppet Lab's documentation is focused on Apache, I found the following two links extremely helpful for information about Nginx/Passenger configuration. They refer to CentOS but it wasn't too difficult to translate what was needed.

http://z0mbix.github.io/blog/2012/03/01/use-nginx-and-passenger-to-power-your-puppet-master/
http://www.watters.ws/mediawiki/index.php/Configure_puppet_master_using_nginx_and_mod_passenger

Our /usr/local/etc/nginx/nginx.conf file:
Code:
cat > /usr/local/etc/nginx/nginx.conf << 'EOF'
user  www www;
worker_processes  4;
error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    passenger_root /usr/local/lib/ruby/gems/1.9/gems/passenger-4.0.29;
    passenger_ruby /usr/local/bin/ruby;
    passenger_max_pool_size 15;
    passenger_pool_idle_time 300;
    #passenger_spawn_method direct; # Uncomment on Ruby 1.8 for ENC to work

    include       mime.types;
    default_type  application/octet-stream;
    sendfile      on;
    tcp_nopush    on;
    keepalive_timeout  65;
    tcp_nodelay        on;

    server {
        listen       3000;
        server_name  [highlight]dashboard.example.com[/highlight];

        passenger_enabled on;
	passenger_user             puppet-dashboard;
	passenger_group            puppet-dashboard;

        access_log /var/log/nginx/dashboard_access.log;

        root /usr/local/share/puppet-dashboard/public;
    }
    server {
        listen       8140 ssl;
        server_name  [highlight]puppet.example.com[/highlight];

        passenger_enabled          on;
        passenger_set_cgi_param    HTTP_X_CLIENT_DN $ssl_client_s_dn;
        passenger_set_cgi_param    HTTP_X_CLIENT_VERIFY $ssl_client_verify;
	passenger_user             puppet;
	passenger_group            puppet;

        access_log                 /var/log/nginx/puppet_access.log;

        root                       /usr/local/etc/puppet/rack/public;
        ssl_certificate            /var/puppet/ssl/certs/[highlight]puppet.example.com[/highlight].pem;
        ssl_certificate_key        /var/puppet/ssl/private_keys/[highlight]puppet.example.com[/highlight].pem;
        ssl_crl                    /var/puppet/ssl/ca/ca_crl.pem;
        ssl_client_certificate     /var/puppet/ssl/certs/ca.pem;
        ssl_ciphers                SSLv2:-LOW:-EXPORT:RC4+RSA;
        ssl_prefer_server_ciphers  on;
        ssl_verify_client          optional;
        ssl_verify_depth           1;
        ssl_session_cache          shared:SSL:128m;
        ssl_session_timeout        5m;
    }
}
'EOF'

Create the log directory to prevent issues on startup.
mkdir /var/log/nginx

Enable a daily log file rotation via newsyslog.conf
printf "/var/log/nginx/*.log\t\t\t644 7\t * @T00 JG /var/run/nginx.pid 30\n" >> /etc/newsyslog.conf

If the puppetmaster service is still running from earlier testing, stop it now
service puppetmaster onestop

With initial setup of the Puppetmaster done, a RACK file that Nginx will use to start the Ruby application will be needed. Copy/paste the example.

Code:
mkdir -p /usr/local/etc/puppet/rack/public
cat > /usr/local/etc/puppet/rack/config.ru << 'EOF'
# Trimmed back FreeBSD Version of https://github.com/puppetlabs/puppet/blob/master/ext/rack/files/config.ru
$0 = "master"
ARGV << "--rack"
ARGV << "--confdir" << "/usr/local/etc/puppet"
ARGV << "--vardir" << "/var/puppet"
require 'puppet/util/command_line'
run Puppet::Util::CommandLine.new.execute
'EOF'
chown -R puppet:puppet /usr/local/etc/puppet/rack

Enable nginx service and start it. At this point basic functionality is online.
echo 'nginx_enable="YES"' >> /etc/rc.conf
service nginx start


Configuring Dashboard - Advanced Features
http://docs.puppetlabs.com/dashboard/manual/1.2/configuring.html

Generating Certs and Connecting to the Puppet Master
With separate Puppet/Dashboard systems the puppet cert sign dashboard will be on the Puppetmaster.

cd /usr/local/share/puppet-dashboard
su -m puppet-dashboard -c 'bundle exec rake cert:create_key_pair'
su -m puppet-dashboard -c 'bundle exec rake cert:request'
puppet cert sign dashboard
su -m puppet-dashboard -c 'bundle exec rake cert:retrieve'

Enabling Inventory Support
Example auth.conf (on Puppet master)
Code:
path /facts
auth yes
method find, search
allow dashboard

Enabling the Filebucket Viewer
Example site.pp (on Puppet master)
Code:
filebucket { "main":
  server => "{your puppet master}",
  path => false,
}

In either site.pp, in an individual init.pp, or in a specific manifest.
Code:
File { backup => "main" }

Go back and add the line for Inventory Support.

cd /usr/local/share/puppet-dashboard/config
Code:
patch settings.yml << 'EOF'
35c35
< enable_inventory_service: false
---
> enable_inventory_service: true
45c45
< use_file_bucket_diffs: false
---
> use_file_bucket_diffs: true
54c54
'EOF'

With all the updates made, restart so that it takes effect.
service nginx restart

For future maintenance, periodic jobs to prune old reports and run DB optimization.
Code:
mkdir -p /usr/local/etc/periodic/monthly
cat > /usr/local/etc/periodic/monthly/clean_dashboard_database.sh << 'EOF'
#!/bin/sh
cd /usr/local/share/puppet-dashboard && \
    echo "Pruning Old Reports from Puppet Dashboard Database" && \
    /usr/bin/su -m puppet-dashboard -c '/usr/local/bin/bundle exec rake RAILS_ENV=production reports:prune upto=3 unit=mon'  && \
    echo "Optimizing Database" && \
    /usr/bin/su -m puppet-dashboard -c '/usr/local/bin/bundle exec rake RAILS_ENV=production db:raw:optimize'
'EOF'
chmod 755 /usr/local/etc/periodic/monthly/clean_dashboard_database.sh

Code:
mkdir -p /usr/local/etc/periodic/weekly
cat > /usr/local/etc/periodic/weekly/clean_puppet_reports.sh << 'EOF'
#!/bin/sh
echo "Pruning Puppetmaster Reports greater than 7 days old"
echo -n "  Reports Removed:"
find /var/puppet/reports -mtime 7 | xargs rm -v | wc -l
'EOF'
chmod 755 /usr/local/etc/periodic/weekly/clean_puppet_reports.sh


Conclusion and To Do List

When all is said and done, you'll have a Puppetmaster accessible from https://puppet.example.com:8140 and a Dashboard accessible from http://dashboard.example.com:3000.

  1. Update to the next Puppet Dashboard when it's released and update to Ruby 1.9 in the process
  2. Add Password Login to Dashboard that doesn't break reports/upload URL
  3. Add HTTPS support to Dashboard
  4. Perhaps find a cleaner way to start delayed workers than the rc script I wrote.
  5. Maybe recompile Git with the Gitweb browser and activate another virtual host for Nginx
 
Hi,

First of all thank you for working that out. I do run it since a while together with theforeman (http://theforeman.org) as dashboard. I just wonder if you did get the external_node script to run (you need to change the ruby part). I get a strange error but I can't put my finger on what it is. Do you have any idea?
Code:
puppet-master[31729]: (//xxxxxxxx.com/Puppet) Error 400 on SERVER: Failed to find xxxxxxx.om via exec: Execution of '/usr/local/etc/puppet/node.rb xxxxxxx.com' returned 1

The funny part is when I run Puppet with the rc.d script it works just fine.
 
Thank you @junovitch for the extremely detailed how-to.

@miwi@, are you running foreman on FreeBSD? It looks very interesting and I might be able to make use of it. It has a lot of features the Puppet Dashboard doesn't seem to have, features I very much like to incorporate in a project I'm currently working on. I also like how it includes DNS, something I was looking for. But I have to be able to run it on FreeBSD.
 
Last edited by a moderator:
@miwi@,
I hadn't turned on the ENC until now. That was a real bugger to figure out. I'll be fixing the instructions shortly. This should do the trick.

Fix the shebang line so it knows to use the right path for ruby.
sed -i'' -e 's/#! \/usr\/bin\/ruby/#!\/usr\/local\/bin\/ruby/' /usr/local/share/puppet-dashboard/bin/external_node

And append the passenger_spawn_method line /usr/local/etc/nginx/nginx.conf
Code:
 http {
     passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-4.0.17;
     passenger_ruby /usr/local/bin/ruby;
     passenger_max_pool_size 15;
     passenger_pool_idle_time 300;
+    passenger_spawn_method direct;
 
Last edited by a moderator:
I've gone ahead and updated the instructions to cover using the community supported Dashboard 2.0 and Ruby 1.9. Based off this video from the recent Puppet Conference http://puppetlabs.com/presentations/story-dashboard-20, they are right around the corner from tagging the community supported Dashboard as version 2.0 since Puppet Labs stopped supporting version 1.2 in lieu of other projects. With lang/ruby18 being removed getting over to Ruby 1.9 seemed important so I've gone through and updated it from scratch to cover the new Ruby and Dashboard versions. Not too many significant changes, mainly the use of sysutils/rubygem-bundler to support self-contained gem dependencies inside the application and outside of the ports tree required installing a few packages to support using it. In addition, multiple commands change to reflect using the Bundler commands instead of other Ruby commands directly. Enjoy.
 
Re: HOWTO: Puppet and Puppet Dashboard on top of Nginx/Passe

Hi @junovitch!

Super thank you for this, Puppet Dashboard is awesome!

Three questions:

Dashboard was a completely new concept to me so bear with me here, I've just been programming my master from CLI up until now:)

  1. How come NGINX is preferred over Apache? I myself have no personal preference on the matter and know too little to know the difference, so that's why I'm just curious why it was chosen over the "standard" web service.
  2. When all is said and done, you don´t need to (or are even able to) start the regular Puppetmaster service any more right? Because you configure NGINX to listen to 8140 which is the normal Puppetmaster port, I'm guessing they are mutually exclusive, at least on the same server.
  3. When you say that the Puppetmaster is accessible from port 8140, you mean just like a normal Puppetmaster, or is there even more coolness to it, like a GUI or something? From what I've gathered, there is nothing to control through this, this is just the monitoring aspect from Dashboard, right? Just to be clear, I think the monitoring is cool enough for me, I just need to ask so that I haven't missed something even cooler:)
Oh, and one more thing, don't you think the SQL client always should be the MariaDB version?

portmaster sysutils/puppet sysutils/puppet-lint devel/git www/nginx devel/ruby-gems devel/rubygem-rake converters/ruby-iconv www/rubygem-passenger sysutils/rubygem-bundler [b]databases/mysql55-client[/b] textproc/libxslt www/node databases/sqlite3 should perhaps be changed to portmaster sysutils/puppet sysutils/puppet-lint devel/git www/nginx devel/ruby-gems devel/rubygem-rake converters/ruby-iconv www/rubygem-passenger sysutils/rubygem-bundler [b]databases/mariadb55-client[/b] textproc/libxslt www/node databases/sqlite3

/Sebulon
 
Last edited by a moderator:
Re: HOWTO: Puppet and Puppet Dashboard on top of Nginx/Passe

Hello @Subulon, I'm glad you found this useful.
  1. Personal preference. I went with Nginx because of its reputation for performance. Additionally, since I had only worked a little with Apache in the past I felt it would be a good learning opportunity to test out something new.
  2. That is correct. You don't need to start the standalone Puppetmaster anymore. Since both listen on the same port, you won't be able to anyway. The standalone Puppetmaster starts using Ruby's built in single-threaded WebRICK HTTP server listening on port 8140. Once it's on a dedicated server, be that Nginx or Apache, the request on port 8140 gets sent to Passenger instead and Passenger spawns Puppet as a rack application. If you watch top right after restarting the web server you can see this in action. There will be some activity with Passenger followed by a new Ruby process using the UID of the puppet user.
  3. It's mainly monitoring, although you do get some ability to change things with Dashboard as an External Node Classifer (ENC). You can assign the classes that already exist on the Puppetmaster on the fly via the Dashboard rather than dropping to the command line to put them in a manifest. It's not terribly useful since changing those modules themselves requires a command line, but nonetheless it could be useful in some situations.
As far as the client package, with Puppet Dashboard 1.2 I had originally installed databases/rubygem-mysql, which brought in databases/mysql55-client. Rather than try to force that particular port to use the databases/mariadb55-client version I just left it as is and didn't change it after updating to Puppet Dashboard 2.0. They are supposed to be drop in replacements so either one should provide the necessary libraries needed to run the bundler and install all the needed Ruby gems. You are absolutely right though, and for consistency I should change it. I will just try it out to be sure and fix it later on this week.
 
Last edited by a moderator:
Re: HOWTO: Puppet and Puppet Dashboard on top of Nginx/Passe

Hi @junovitch!

I just wanted to chip in that I tried getting authentication working in Nginx that I suspect went equally well as for you, since you stated in your to-do list that you needed to find a way to enable password login that doesn't break Dashboard (it broke mine as well). After trying both basic- and ldap-auth, and getting a lot of 403, I tried switching to Apache instead, recompiled rubygem-passenger and after reading a lot of PuppetLabs´s documentation I now have both secure SSL connectivity and password login from our Active Directory. I can share the configuration if you are interested.

/Sebulon
 
Last edited by a moderator:
Re: HOWTO: Puppet and Puppet Dashboard on top of Nginx/Passe

@Sebulon,
Sorry I didn't reply sooner. Work has kept me busy this week. I toyed around with the password authentication for a bit before putting it on the back burner. Having a password for the root directory always inherited and I couldn't quite figure out how to prevent that. It's not that important to me but would be nice to figure out. I'm about to update my post to use databases/mariadb55-client as that does work as expected. The Rubygem bundler has no issues using those libraries when it installs the Rubygem MySQL2 gem. I'm also going to clean this up for the the new forum style. I would appreciate any information you have on your Apache setup and would be glad to modify my instructions where appropriate to refer to what you have came up with for anybody interested in using Apache.
 
Last edited by a moderator:
Re: HOWTO: Puppet and Puppet Dashboard on top of Nginx/Passe

Hi @junovitch!

Oh no worries, we all seem to be very busy these days. OK, so here comes the 'How' in Apache (sorry, couldn´t resist)

databases/mariadb55-client
databases/mariadb55-server
Code:
(X) OPENSSL

www/apache22
Code:
(X) AUTHNZ_LDAP
(X) LDAP

And normally you wouldn´t need to do anything here but since www/rubygem-passenger now has been compiled for Nginx:
# cd /usr/ports/www/rubygem-passenger
# make clean config reinstall clean
Code:
(X) APACHE22

Then it´s just a matter of placing out a vhost-definition for apache and you´re all set. This file is actually included in dashboard, the original is in ${DASHBOARD_DIR}/ext/passenger/dashboard-vhost.conf.

/usr/local/etc/apache22/Includes/dashboard-vhost.conf
Code:
# UPDATE THESE PATHS TO SUIT YOUR ENVIRONMENT
LoadModule passenger_module /usr/local/lib/ruby/gems/1.9/gems/passenger-4.0.27/buildout/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.9/gems/passenger-4.0.27
PassengerDefaultRuby /usr/local/bin/ruby19

# you may want to tune these settings
PassengerHighPerformance on
PassengerMaxPoolSize 12
PassengerPoolIdleTime 1500
# PassengerMaxRequests 1000
PassengerStatThrottleRate 120
PassengerEnabled On

<VirtualHost *:80>
        ServerName puppet.foo.bar
        DocumentRoot /usr/local/share/puppet-dashboard/public/
        <Directory /usr/local/share/puppet-dashboard/public/>
                Options None
                Order allow,deny
                allow from all
        </Directory>
  ErrorLog /var/log/apache2/puppet.foo.bar_error.log
  LogLevel warn
  CustomLog /var/log/apache2/puppet.foo.bar_access.log combined
  ServerSignature On

# Uncomment this section to enable basic auth. This section can also be copied
# to the HTTPS VirtualHost example below.
   # For report submission from masters.
   <Location /reports/upload>
       <Limit POST>
           # Configuration restricts HTTP actions to POST only
           Order allow,deny
           # Allow from localhost
           # Allow from localhost.localdomain
           # Allow from 127.0.0.1
           # Allow from example.com
           # This can be locked down to just your puppet master if required
           # See examples above, or http://httpd.apache.org/docs/2.2/howto/access.html
           Allow from all
           Satisfy any
       </Limit>
   </Location>

   # For node definitions from masters.
   <Location /nodes>
       <Limit GET>
           # Configuration restricts HTTP actions to GET only
           Order allow,deny
           # Allow from localhost.localdomain
           # Allow from localhost
           # Allow from 127.0.0.1
           # Allow from example.com
           # This can be locked down to just your puppet master if required
           # See examples above, or http://httpd.apache.org/docs/2.2/howto/access.html
           Allow from all
           Satisfy any
       </Limit>
   </Location>

   <Location "/">
   ### I actually prefer _not_ being able to log in unencrypted ###
   #         AuthType basic
   #         AuthName "Puppet Dashboard"
   #         AuthBasicProvider ldap
   #         AuthLDAPBindDN "CN=someuser,OU=Users,DC=ad,DC=foo,DC=bar"
   #         AuthLDAPBindPassword VerySecretPassword
   #         AuthLDAPURL ldap://ad.foo.bar:3268/DC=ad,DC=foo,DC=bar?sAMAccountName?sub?(objectClass=*)
   #         AuthLDAPGroupAttributeIsDN off
   #         Require valid-user
       Order deny,allow
       Deny from all
       Allow from localhost.my.domain
       Allow from localhost
       Allow from 127.0.0.1
       Allow from puppet.foo.bar
   </Location>

</VirtualHost>

# Uncomment this section to enable HTTPS (SSL)
Listen 443
<VirtualHost *:443>
        SSLEngine on
        SSLProtocol -ALL +SSLv3 +TLSv1
        SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP

        SSLCertificateFile        /var/puppet/ssl/certs/puppet.foo.bar.pem
        SSLCertificateKeyFile     /var/puppet/ssl/private_keys/puppet.foo.bar.pem
        SSLCACertificateFile      /var/puppet/ssl/certs/ca.pem

        # If Apache complains about invalid signatures on the CRL, you can try disabling
        # CRL checking by commenting the next line, but this is not recommended.
        SSLCARevocationFile       /var/puppet/ssl/ca/ca_crl.pem

        SSLVerifyClient optional
        SSLVerifyDepth  1
        SSLOptions +StdEnvVars

        ServerName puppet.foo.bar
        DocumentRoot /usr/local/share/puppet-dashboard/public
        <Directory   /usr/local/share/puppet-dashboard/public>
                Options None
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

        ErrorLog /var/log/apache2/ssl_puppet.foo.bar_error.log
        LogLevel warn
        CustomLog /var/log/apache2/ssl_puppet.foo.bar_access.log combined
        ServerSignature On

        <Location /reports/upload>
            <Limit POST>
                Order allow,deny
                Allow from all
               Satisfy any
            </Limit>
        </Location>

        <Location /nodes>
            <Limit GET>
                Order allow,deny
                Allow from all
                Satisfy any
            </Limit>
        </Location>

        <Location "/">
            AuthType basic
            AuthName "Puppet Dashboard"
            AuthBasicProvider ldap
            AuthLDAPBindDN "CN=someuser,OU=Users,DC=ad,DC=foo,DC=bar"
            AuthLDAPBindPassword VerySecretPassword
            AuthLDAPURL ldap://ad.foo.bar:3268/DC=ad,DC=foo,DC=bar?sAMAccountName?sub?(objectClass=*)
            AuthLDAPGroupAttributeIsDN off
            Require valid-user
        </Location>

</VirtualHost>

Listen 8140
<VirtualHost *:8140>

    SSLEngine on
    SSLCipherSuite SSLv2:-LOW:-EXPORT:RC4+RSA
    SSLCertificateFile        /var/puppet/ssl/certs/puppet.foo.bar.pem
    SSLCertificateKeyFile     /var/puppet/ssl/private_keys/puppet.foo.bar.pem
    SSLCACertificateFile      /var/puppet/ssl/certs/ca.pem
    SSLCARevocationFile       /var/puppet/ssl/ca/ca_crl.pem
    SSLVerifyClient optional
    SSLVerifyDepth  1
    SSLOptions +StdEnvVars

    # The following client headers allow the same configuration to work with Pound.
    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

    DocumentRoot /usr/local/etc/puppet/rack/public
    <Directory /usr/local/etc/puppet/rack/>
        Options None
        AllowOverride None
        Order allow,deny
        allow from all
    </Directory>

    ErrorLog /var/log/apache2/ssl_puppet.foo.bar_error.log
    LogLevel warn
    CustomLog /var/log/apache2/ssl_puppet.foo.bar_access.log combined
    ServerSignature On

</VirtualHost>

You´ll end up with the dashboard accessible from https://puppet.foo.bar, no non-standard port that will always be a source of misunderstanding otherwise, but that´s just a matter of taste really. For Active Directory login you need a user that is used for LDAP-searches, so make it very unprivileged, only being able to read what it needs to read, since the password is written here in plain text. For smaller environments you can use "AuthBasicProvider file" instead, creating local apache accounts with htpasswd, instructions can found at http://httpd.apache.org/docs/current/howto/auth.html. The standard root URL http://puppet.foo.bar:80 is blocked with only localhost allowed. For systems that are "out there" on the web, that is a must, in my opinion.

/Sebulon
 
Last edited by a moderator:
Back
Top