Poudriere repository

balanga · May 19, 2019

I have manged to build a few pkgs using poudriere and can install them using pkg add after copying them to somewhere on my LAN. I'd like to be able to install them using pkg install but can't figure out what is involved...

I guess I need to change POUDRIERE_DATA= in /usr/local/etc/poudriere.conf to something like /mnt/pkg-repository which is a share which will have been mounted at boot and then have a /usr/local/etc/pkg/repos/myrepo.conf
consisting of:-

Code:

myrepo: {
    url             : "file:///mnt/pkg-repository",
    enabled         : yes,
    mirror_type     : NONE
}

Am I close?

mfaridi · May 19, 2019

Can this happen of hardware compatibility?

malavon · May 19, 2019

What I use for my repo, though I don't use a file: scheme anymore (because I wanted to host my repo publicly online without having to use a VPN or such):

Code:

server: {
  url: "pkg+http://packages.malavon.com/${ABI}-default-server",
  mirror_type: "srv",
  enabled: no
}

Check manpages to be sure.

Lamia · May 20, 2019

What error do you get when you run "pkg install?" What version of FBSD are you running? Is it up-to-date? Can you install pkgs from the FBSD quarterly repo using "pkg install?" All of these might give you some clues.

SirDice · May 20, 2019

Set up www/nginx for example and share the packages with it. Then you can easily use it as a remote repository.

Poudriere has configuration examples for both Apache and nginx in /usr/local/share/examples/poudriere.

Here's my nginx.conf:

Code:

load_module /usr/local/libexec/nginx/ngx_http_fancyindex_module.so;

user  www;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

        # Allow gzipping js, css, log, svg and json files.
        gzip on;
        gzip_http_version 1.0;
        gzip_comp_level 6;
        gzip_proxied any;
        gzip_min_length  1100;
        gzip_buffers 16 8k;
        gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript image/g
if image/jpeg image/png application/json image/svg+xml;

        types {
                text/plain                            log;
        }


    server {
        listen          80;
        server_name     ports.example.com;
        root            /usr/local/share/poudriere/html;

        # Allow caching static resources
        location ~* ^.+\.(jpg|jpeg|gif|png|ico|svg|woff|css|js|html)$ {
                add_header Cache-Control "public";
                expires 2d;
        }

        location /data {
                alias /usr/local/poudriere/data/logs/bulk;

                # Allow caching dynamic files but ensure they get rechecked
                location ~* ^.+\.(log|txz|tbz|bz2|gz)$ {
                        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
                }

                # Don't log json requests as they come in frequently and ensure
                # caching works as expected
                location ~* ^.+\.(json)$ {
                        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
                        access_log off;
                        log_not_found off;
                }

                # Allow indexing only in log dirs
                location ~ /data/?.*/(logs|latest-per-pkg)/ {
                        autoindex on;
                }

                break;
        }

        location /packages {
                alias /usr/local/poudriere/data/packages;

                fancyindex on;
                }

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/local/www/nginx-dist;
        }

    }
}

This shares the packages on http://ports.example.com/packages but note that things are split up according to your jails and port names.
For example I have multiple repositories:

I have three jails in poudriere:

Code:

root@molly:~ # poudriere jail -l
JAILNAME    VERSION         ARCH  METHOD                                    TIMESTAMP           PATH
112-release 11.2-RELEASE-p9 amd64 ftp                                       2019-03-03 14:04:03 /usr/local/poudriere/jails/112-release
120-release 12.0-RELEASE-p3 amd64 ftp                                       2019-04-13 17:32:11 /usr/local/poudriere/jails/120-release
12-stable   12.0-STABLE     amd64 url=file:///storage/release/12-stable/ftp 2019-05-05 19:23:51 /usr/local/poudriere/jails/12-stable

With two ports trees:

Code:

root@molly:~ # poudriere ports -l
PORTSTREE METHOD    TIMESTAMP           PATH
desktop   svn+https 2019-05-18 22:40:11 /usr/local/poudriere/ports/desktop
server    svn+https 2019-05-18 22:37:50 /usr/local/poudriere/ports/server

This gives me a total of 6 different repositories.

Lamia · May 20, 2019

SirDice said:

Set up www/nginx for example and share the packages with it. Then you can easily use it as a remote repository.

Poudriere has configuration examples for both Apache and nginx in /usr/local/share/examples/poudriere.

Here's my nginx.conf:

Code:

load_module /usr/local/libexec/nginx/ngx_http_fancyindex_module.so;

user  www;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

        # Allow gzipping js, css, log, svg and json files.
        gzip on;
        gzip_http_version 1.0;
        gzip_comp_level 6;
        gzip_proxied any;
        gzip_min_length  1100;
        gzip_buffers 16 8k;
        gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript image/g
if image/jpeg image/png application/json image/svg+xml;

        types {
                text/plain                            log;
        }


    server {
        listen          80;
        server_name     ports.example.com;
        root            /usr/local/share/poudriere/html;

        # Allow caching static resources
        location ~* ^.+\.(jpg|jpeg|gif|png|ico|svg|woff|css|js|html)$ {
                add_header Cache-Control "public";
                expires 2d;
        }

        location /data {
                alias /usr/local/poudriere/data/logs/bulk;

                # Allow caching dynamic files but ensure they get rechecked
                location ~* ^.+\.(log|txz|tbz|bz2|gz)$ {
                        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
                }

                # Don't log json requests as they come in frequently and ensure
                # caching works as expected
                location ~* ^.+\.(json)$ {
                        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
                        access_log off;
                        log_not_found off;
                }

                # Allow indexing only in log dirs
                location ~ /data/?.*/(logs|latest-per-pkg)/ {
                        autoindex on;
                }

                break;
        }

        location /packages {
                alias /usr/local/poudriere/data/packages;

                fancyindex on;
                }

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/local/www/nginx-dist;
        }

    }
}

This shares the packages on http://ports.example.com/packages but note that things are split up according to your jails and port names.
For example I have multiple repositories:

I have three jails in poudriere:

Code:

root@molly:~ # poudriere jail -l
JAILNAME    VERSION         ARCH  METHOD                                    TIMESTAMP           PATH
112-release 11.2-RELEASE-p9 amd64 ftp                                       2019-03-03 14:04:03 /usr/local/poudriere/jails/112-release
120-release 12.0-RELEASE-p3 amd64 ftp                                       2019-04-13 17:32:11 /usr/local/poudriere/jails/120-release
12-stable   12.0-STABLE     amd64 url=file:///storage/release/12-stable/ftp 2019-05-05 19:23:51 /usr/local/poudriere/jails/12-stable

With two ports trees:

Code:

root@molly:~ # poudriere ports -l
PORTSTREE METHOD    TIMESTAMP           PATH
desktop   svn+https 2019-05-18 22:40:11 /usr/local/poudriere/ports/desktop
server    svn+https 2019-05-18 22:37:50 /usr/local/poudriere/ports/server

This gives me a total of 6 different repositories.

Don't forget to change the settings in your myrepo.conf config from file when done. You will need a few additional lines too. An example is shown above but it is not enabled...

balanga · May 21, 2019

SirDice said:
Set up www/nginx for example and share the packages with it. Then you can easily use it as a remote repository.

Poudriere has configuration examples for both Apache and nginx in /usr/local/share/examples/poudriere.
I have three jails in poudriere:

Code:

root@molly:~ # poudriere jail -l JAILNAME VERSION ARCH METHOD TIMESTAMP PATH 112-release 11.2-RELEASE-p9 amd64 ftp 2019-03-03 14:04:03 /usr/local/poudriere/jails/112-release 120-release 12.0-RELEASE-p3 amd64 ftp 2019-04-13 17:32:11 /usr/local/poudriere/jails/120-release 12-stable 12.0-STABLE amd64 url=file:///storage/release/12-stable/ftp 2019-05-05 19:23:51 /usr/local/poudriere/jails/12-stable

With two ports trees:

Code:

root@molly:~ # poudriere ports -l PORTSTREE METHOD TIMESTAMP PATH desktop svn+https 2019-05-18 22:40:11 /usr/local/poudriere/ports/desktop server svn+https 2019-05-18 22:37:50 /usr/local/poudriere/ports/server

This gives me a total of 6 different repositories.

You're obviously a heavy duty user and I appreciate the advice. Maybe your post could be included in the Howtos section...

Poudriere repository

balanga

mfaridi

malavon

Lamia

SirDice

Administrator

Lamia

balanga