Copy/install packages from one to other server

R

RusDyr

Are there any proper method to copy already installed packages from one server to other?
I've got a bunch of FreeBSD servers (10.1 and 10.2 releases), and sometimes need to setup new one that differs only in some network settings.
Currently I do some customized setup script from USB flash with modified FreeBSD "memstick edition", that untar base system to disk and copy over changed files. For packages copying I doing pkg create -a on a production server and pkg add *.txz in chroot of the new one. But I believe it's a wrong way since it install some packages whatever it's already have been installed earlier. Moreover ( and it's more important), it can't install some dependencies due it doesn't know that it's in catalog.
So I decided to create local repo:
pkg repo / && scp /*.txz root@NEW_SERV:/pkg/ && scp /var/cache/pkg/*.txz root@NEW_SERV:/pkg/var/cache/pkg/

/etc/pkg/FreeBSD.conf
Code: 
# $FreeBSD: release/10.0.0/etc/pkg/FreeBSD.conf 258710 2013-11-28 14:24:26Z gjb $
myrepo: {
url: "file:////pkg",
mirror_type: "file",
enabled: yes
}

FreeBSD: {
url: "pkg+[URL]http://pkg.FreeBSD.org/${ABI}/latest[/URL]",
mirror_type: "srv",
signature_type: "fingerprints",
fingerprints: "/usr/share/keys/pkg",
enabled: yes
}

But when I trying to install it on new server, it gives me many duplicate or even "required shared library ... not found":

On # pkg upgrade
Code: 
...
pkg: ruby-2.1.5_4,1: duplicate dependency listing: libyaml, ignoring
pkg: ruby-2.1.5_4,1: duplicate dependency listing: libffi, ignoring
pkg: ruby-2.1.5_4,1: duplicate dependency listing: libedit, ignoring
pkg: ruby-2.1.5_4,1: duplicate dependency listing: libyaml, ignoring
pkg: ruby-2.1.5_4,1: duplicate dependency listing: libffi, ignoring
pkg: ruby-2.1.5_4,1: duplicate dependency listing: libedit, ignoring
pkg: ruby-2.1.6,1: duplicate dependency listing: libyaml, ignoring
pkg: ruby-2.1.6,1: duplicate dependency listing: libffi, ignoring
pkg: ruby-2.1.6,1: duplicate dependency listing: libedit, ignoring
pkg: ruby-2.1.6,1: duplicate dependency listing: libyaml, ignoring
pkg: ruby-2.1.6,1: duplicate dependency listing: libffi, ignoring
pkg: ruby-2.1.6,1: duplicate dependency listing: libedit, ignoring
pkg: ruby-2.1.7,1: duplicate dependency listing: libyaml, ignoring
pkg: ruby-2.1.7,1: duplicate dependency listing: libffi, ignoring
pkg: ruby-2.1.7,1: duplicate dependency listing: libedit, ignoring
pkg: ruby-2.1.7,1: duplicate dependency listing: libyaml, ignoring
pkg: ruby-2.1.7,1: duplicate dependency listing: libffi, ignoring
pkg: ruby-2.1.7,1: duplicate dependency listing: libedit, ignoring
...

# pkg check -aBds
Code: 
...
Checking all packages: 93%
(svnup-1.07_1) /usr/local/bin/svnup - required shared library libmd.so.6 not found
(svnup-1.07_1) /usr/local/bin/svnup - required shared library libssl.so.7 not found
(svnup-1.07_1) /usr/local/bin/svnup - required shared library libc.so.7 not found
Checking all packages: 100%
# locate libc.so.7
Code: 
/lib/libc.so.7
# file /lib/libc.so.7
Code: 
/lib/libc.so.7: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, stripped

So, the question is - how should I copy installed packages to other server or install same?
 
[root@INSTALL-VR ~/INSTALL_SCRIPTS]# SHELL="/bin/tcsh" chroot /mnt/gm-rootfs/
root@INSTALL-VR:/ # ls /var/db/pkg/
Code: 
local.sqlite myrepo.meta repo-myrepo.sqlite
root@INSTALL-VR:/ # rm /var/db/pkg/*
root@INSTALL-VR:/ # pkg update -r myrepo
Code: 
Updating myrepo repository catalogue...
Fetching meta.txz: 100% 264 B 0.3kB/s 00:01
Fetching packagesite.txz: 100% 27 KiB 27.2kB/s 00:01
Processing entries: 100%
myrepo repository update completed. 218 packages processed.
pkg: sqlite error while executing INSERT INTO pkg_search SELECT id, name || '-' || version, origin FROM packages;CREATE INDEX packages_origin ON packages(origin COLLATE NOCASE);CREATE INDEX packages_name ON packages(name COLLATE NOCASE);CREATE INDEX packages_uid_nocase ON packages(name COLLATE NOCASE, origin COLLATE NOCASE);CREATE INDEX packages_version_nocase ON packages(name COLLATE NOCASE, version);CREATE INDEX packages_uid ON packages(name, origin);CREATE INDEX packages_version ON packages(name, version);CREATE UNIQUE INDEX packages_digest ON packages(manifestdigest); in file pkgdb.c:2394: UNIQUE constraint failed: packages.manifestdigest
 
I highly recommend setting up your own repository with ports-mgmt/poudriere. Then point all your servers to this repository.

Note, never edit /etc/pkg/FreeBSD.conf. Changes will be overwritten next time you update the server. Instead create:
/usr/local/etc/pkg/repos/FreeBSD.conf:
Code: 
FreeBSD: {
  enabled: no
}
/usr/local/etc/pkg/repos/myrepo.conf
Code: 
myrepo: {
  enabled: yes
  url: file:///storage/pkg/
}
 
RusDyr said:
root@INSTALL-VR:/ # rm /var/db/pkg/*
Click to expand...
Never do this! This also removes /var/db/pkg/local.sqlite which is the database with currently installed packages. If you remove this file the system will think there are no packages installed any more.
 
I appreciate your answers but they are slightly irrelevant to my question.
I installed poudriere, but still can't import existing packages to it or make new one - poudriere requires in "bind" set port name as relative pathname, e.g. www/nginx, while pkg info shows only port name itself. And there is no way, AFAIK, to just copy or other way import from installed existing packages. That's why I tried to use local repo as a easiest way to do it.
 
Just give poudriere a list of ports you need. It'll build the dependencies automatically. For a typical FAMP server this would be all that's needed:
Code: 
www/apache24
www/mod_php56
lang/php56
databases/mysql55-server
 
SirDice , it's a pretty clear.
Anyway, it's a pity what there isn't a simple and a quick way to backup and restore already installed packages from one server to other, without additional software - for cases occured from time to time, it's a overhead. Isn't it?
 
I rarely have a need for it. If you have a proper repository it's easy to install the packages you need, why back them up at all? Settings are all saved and won't be packaged any way. So you still have to configure things. The only backups I make are for configuration files and data. The rest is easily installed from scratch.

And if you're looking for something to automate the installation and configuration I can highly recommend sysutils/puppet.
 
SirDice , what you need in rarely is not a all need it rarely also ;) Sometimes it really helpful. As I said before, there was working way - make package-recursive and pkg_add -v *.tgz after on the new server. So it would be nice to have similar ability. Moreover, it has with pkg create -a and pkg add *.txz.
In short - what about using local file repository? What is proper way?
BTW, sysutils/puppet is poorly documented in FreeBSD part - I have one puppet installation, and there is a bunch of problems with it: user management, rc.conf and so on.
 
I know where you're coming from, RusDyr, and what's "handy" to one, does not guarantee "handy" to another. ;)
The answer was actually already provided. But perhaps just not the way you were looking for it. :)
I've already gone through this same situation. Although I've changed my method to dump(), and restore(). If all your packages are centrally located, and have been made into a "proper" repo. The answer is as simple as pointing your client to the location of the repo. Either as part of a mount()ed filesystem, an ftp() server, or even by way of a mount()ed USB stick. The "magic" to all this, is simply creating a proper "repo", as you indicate you've already done. So all you have to do, is point the client you want to up(grade|date) to it. Create the necessary .conf entries on your client, as SirDice indicated, and you're off! It need'nt be complicated; you can make it an ftp() server, by simply entering an IP address, even if that address is a local/private IP. It could be a file path to a mount()ed NFS() filesystem, or even a mount()ed USB stick. I hope this helps, RusDyr!

--Chris
 
You must log in or register to reply here.
Back
Top