Solved How much disk space does a package repository take?

Anybody building all packages have any idea how much storage all the FreeBSD packages take up.
http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/

I just need a rough estimate for feasibility. 10 Gigs -100Gigs-1TB? Single Arch.
I am building NanoBSD images and would like the whole shebang for a personal repository.
Perhaps poudeire to build all. What will it take for diskspace?
Thanks
 
OK there is a part 2 for the package experts. Maybe I don't need my own repository but learn pkg.

The way I build now is pkg clean and scrape from /var/cache/pkg all the files I chose for NanoBSD.
It uses an /Pkg folder under NanoBSD. Any packages in there get built (if script uses pkg_ng function).

Problem is I am chasing alot of dependencies. How can I use pkg fetch and pull all dependencies?
And dependancies of dependancies...you know the routine.

How can I do this smarter. Right now its all handwork through freshports.org and their excellent work. I made lists.
 
Phishfry
Ooo, Ooo. I know this one! ;)
Give a couple moments to whip up some data for you. I'll even throw in a quick HOWTO to make a repo of only the packages you're interested in! :)

--Chris
 
Thanks Chris_H
I read the manual and part #2 is pkg fetch -do
That is all I need for now. I do want to setup a Poudeiere repository for this project.

So how big is a whole arch repository, everything? Just wondering..
 
OK. Found the stats.
If I export all the packages on a running system I want to upgrade to a new version; 93-STABLE==>11.2-STABLE, for example. The grand total (after building the new ones) is 466MB. Do note; this is for a full (XFCE-4 desktop, and asst. utilities). Thess stats are for the built packages only. :)
So there are your "rough" stats. :)

I know this. Because I build a custom FreeBSD install DVD inside a jail(8), and install afresh. Packages, and all. :)

HTH!

--Chris
 
I can't tell tou how much time I wasted chasing failed dependencies on the first run.
Now I'm cooking with fire.
 
OH. You have NO IDEA how frustrated I got trying to manage (configuring) dependencies on the ports I was trying to build. Or maybe you do. :)
make config recursive doesn't quite get it. But you knew that. ;)
I have a rant about my initial experience(s) with all this, on these forums somewhere.

--Chris
 
OK what about a part #3) Extra Extra credit.

My embedded machine will get pkg updates from 'only' its custom repository set in FreeBSD.conf .

How can I add a layer of security here. Authenticate the package server to client.
Any key methods baked in to the package system?
 
Looks like a snippet from bsdnow.tv and thier Poudeiere instructions have me covered! Is 4096 still a good key length?
# mkdir -p /usr/local/etc/ssl/keys /usr/local/etc/ssl/certs
# chmod 600 /usr/local/etc/ssl/keys
# openssl genrsa -out /usr/local/etc/ssl/keys/pkg.key 4096
# openssl rsa -in /usr/local/etc/ssl/keys/pkg.key -pubout > /usr/local/etc/ssl/certs/pkg.cert
Be sure to copy the pkg.cert file to your client systems via a secure
method like SCP or sneakernet. Now we move over to those client systems
and set the appropriate pkg config options to download from the server
you (hopefully) setup.
 
Sorry to be ignorant but I can't believe how easy the above implementation is.

Could someone explain to me this. How does the 'embedded client' machine pkg know to require this key.
Is the magic all in the name of the cert file in the /ssl/keys/ directory or are there more settings needed for /etc/pkg/FreeBSD.conf that are missing in the tutorial?
This seems too easy.
Looking further at the instructions this is probably needed on the client too in its pkg configuration?:
Code:
signature_type: "pubkey",
pubkey: "/usr/local/etc/ssl/certs/pkg.cert",
 
OK. IMHO you're trying to make this more complicated than it need be. That should sound like a relief. In case it didn't ;)
Is it remotely possible you could either:
1) use pf to restrict (every|any)one but your connecting hosts?
Code:
. . .
set block-policy drop
. . .
table <trusted> persist file /etc/TRUSTED # contains ONLY MY trusted hosts, and the NSAs IPs
. . .
pass in quick on { <your-if> } proto tcp from <trusted> to port <some-port-maybe-80?> keep state

# I think you're done here :)
Dump your packages onto a USB stick, or CD/DVD.

While you can easily generate a key, and pkg(8) on the host(s) will accept it.
It just seems like a lot of unnecessary work, for a somewhat one-off affair. Unless this is all
done remotely. In which case option 1 (pf) can handle it. No?

I'll post the (key) incantation for you in a moment.

--Chris
 
For pkg(8) there are 2 places you're concerned with:
/etc/pkg/
For a strictly LOCAL repo on the (receiving) host:
LocalPKG.conf
Code:
LocalPKG: {
  url: "file:///mnt/packages/",
  mirror_type: "none",
  enabled: yes
}

FreeBSD: {
  enabled: no
}
Also in /etc/pkg/ even if not from LOCAL media:
Code:
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: no
}
I left it commented. As it hints to your other concern: /usr/local/pkg/repos/
NOTE:
The FreeBSD: {enabled:no} is important!

OK. I failed to find my notes on the "key" incantation. But I responded to another related thread some time ago. Where I make some other interesting notes you might care about here:
https://forums.freebsd.org/threads/install-some-packages-from-local-repository.49280/

I'll keep looking. :)

--Chris
 
like I needed to say that..
Unfortunatly you do. I have no idea what I am doing and I am really-really winging it.
Hopefully I get kudos for starting with security first!!!
Bryans page mentions using the cloud for a custom repository. That sounds mighty convenient.
Maybe I could setup Poudriere in the cloud too? Farm it out to Amazon. No long term package server equipment needed then.
 
Hmm. I think 4096 should be more than enough.
You should be able to do it thusly:
Code:
mkdir -p /usr/local/etc/ssl/keys /usr/local/etc/ssl/certs
chmod 600 /usr/local/etc/ssl/keys
openssl genrsa -out /usr/local/etc/ssl/keys/pkg.key 4096
openssl rsa -in /usr/local/etc/ssl/keys/pkg.key -pubout > /usr/local/etc/ssl/certs/pkg.cert
 
What are the strengths of a LetsEncrypt based system versus the SSL approach as noted above?
Is it still a cert key system but stronger encryption or private versus public key? Perhaps active key swapping instead of static keys?
 
One. I think the cert you're talking about verifies your repo. Whereas letsencrypt verifies your (web) site. Like I said; just another added layer of security. :)
If you're too stressed just getting things working. You may want to forgo the web cert for now. Then once you've established a working repo. Move on to the www cert. :)

TLDR; one cert (the one you're already working on) verifies your *packages*. Letsencrypt verifies you're *web site*. :)

--Chris
 
This should work as a basic template for your poudriere repo:
Code:
# file: /usr/local/etc/pkg/repos/poudriere.conf
# system: Clients using the package repositories

poudriere: {
url: "http://your.url/here",
mirror_type: "http",
signature_type: "pubkey",
pubkey: "/usr/local/etc/ssl/certs/pkg.cert",
enabled: yes
}
rename as desired. :)
HTH!

--Chris
 
OK I solved my original question with this command: pkg fetch -a

Code:
Number of packages to be fetched: 32268

The process will require 76 GiB more space.
76 GiB to be downloaded.

Proceed with fetching packages? [y/N]:
 
OK I solved my original question with this command: pkg fetch -a

Code:
Number of packages to be fetched: 32268

The process will require 76 GiB more space.
76 GiB to be downloaded.

Proceed with fetching packages? [y/N]:
O. I clearly misunderstood.
I thought you'd be using some sort of builder to whip up only the ports you were interested in for the respective target.
But looks like you wanted the whole ball-o-wax! :eek:

--Chris
 
Not sure if this is relevant to the OP:

~]$ pkg stats
Local package database:
Installed packages: 82
Disk space occupied: 701 MiB

Remote package database(s):
Number of repositories: 1
Packages available: 31972
Unique packages: 31972
Total size of packages: 87 GiB
 
Back
Top