How do I create a FreeBSD software repository server with packages

Discussion related to network/web services such as Apache, BIND Sendmail, etc.

How do I create a FreeBSD software repository server with packages

Postby baronobeefdip » 28 Jan 2012, 22:31

I work in a networking lab and those who use it including myself are diving into using FreeBSD. The way we are set up is we are isolated from the network running on the school for safety purposes since most of the activities we do might damage the network so we run on our own, But we don't have internet so we can't access packages.

I am able to create a software repository for Debian and Fedora just fine to where the internet isn't required to install software that wasn't installed on the os by default. But with making the FreeBSD package repository server I am at a barrier with it. What do I need to do in order to have a local package repository server that we can access whenever we need software that wasn't installed by default in a typical FreeBSD install. And how do I direct the FreeBSD machines to retrieve index files from the server by default so the local package repository is the default server the computer will retrieve software from?
baronobeefdip
Member
 
Posts: 123
Joined: 04 Nov 2010, 22:49

Postby da1 » 28 Jan 2012, 22:53

I would first create a publicly available FTP that will contain all the necessary packages. Then I would make sure that my isolated FreeBSD machine has network access to that FTP server and then I would set the PKG_PATH shell environment to point to that server.

More or less, I think this is all you need to do.
da1
Member
 
Posts: 738
Joined: 02 Sep 2009, 09:35
Location: Berlin,DE

Postby baronobeefdip » 28 Jan 2012, 23:08

I did do that but I get the same error every time. I am doing the tests with the package collection that is on the DVD,I added the entire packages into the [FILE]/home/ftp[/FILE] directory. I am using the installation utility for packages in [FILE]sysinstall[/FILE], I ran [FILE]sysinstall[/FILE] and chose an alternative ftp site, Entered my url for the ftp server "ftp://192.168.1.101". It connected to the ftp server fine but then it tells me that
Code: Select all
9.0 RELEASE Couldn't be located
what does this mean and how do I get rid of this issue so that I can finish setting up the package repository server?
baronobeefdip
Member
 
Posts: 123
Joined: 04 Nov 2010, 22:49

Postby fonz » 28 Jan 2012, 23:39

baronobeefdip wrote:It connected to the ftp server fine but then it tells me that "9.0 RELEASE Couldn't be located"

Did you mimic the directory structure? It's probably looking for [FILE]ftp://192.168.1.101/pub/FreeBSD/releases/i386/9.0-RELEASE/packages[/FILE] or something along those lines.

Fonz
ΔS>0 ⇒ Entropy is here to stay.

"Lawyer" is just "liar" pronounced in a regional accent.
User avatar
fonz
Giant Locked
 
Posts: 2236
Joined: 16 Nov 2008, 16:32
Location: in dreamland, swooning over a woman I can't have

Postby jb_fvwm2 » 29 Jan 2012, 02:12

I had a similar question. A few weeks ago began using a thumbdrive:
( seven years ago or so mucked around with NFS, may try that again...)
Code: Select all
pkg_delete -f /var/db/pkg/gcc-4.6.3.20111201 && pkg_add [file]/mnt/packages/[/file]gcc-4.6.3.20120113.tbz
*GREATLY* seems to simplify the process across multiple machines, if your shell has tab completion, history recall, etc etc. One can have the [file] /usr/ports/packages/All [/file] *mostly* vacant in fact on each machine, transferring packages to the thumbdrive. (At least it is simpler to setup initially). While I'm at it I may as well show a method to prune the [file] /mnt/packages/ [/file] from extra duplicates which will accumulate...: (requires two additional ports installed: )
Code: Select all
gnuls -oSr | sort -k 4 | sort -k 8 | lookat
... one can easily check for duplicates in the pane shown at the result...
jb_fvwm2
Senior Member
 
Posts: 1503
Joined: 16 Nov 2008, 19:19

Postby SirDice » 30 Jan 2012, 07:43

If you have your own network why not use that? No need to fumble about with thumb drives :e

Just NFS export [file]/usr/ports/[/file] and/or [file]/usr/ports/packages/[/file]. If you're worried about security you can export them read-only.

I recommend not using [file]sysinstall[/file] at all. But I think it's looking for an [file]INDEX[/file] file. Not sure if you can just copy the one in [file]/usr/ports/[/file].
Senior UNIX Engineer at Unix Support Nederland
Experience is something you don't get until just after you need it.
User avatar
SirDice
Old Fart
 
Posts: 16185
Joined: 17 Nov 2008, 16:50
Location: Rotterdam, Netherlands

Postby baronobeefdip » 30 Jan 2012, 20:32

fonz wrote:Did you mimic the directory structure? It's probably looking for [FILE]ftp://192.168.1.101/pub/FreeBSD/releases/i386/9.0-RELEASE/packages[/FILE] or something along those lines.

Fonz


It just worked! After I mimicked the filesystem you posted it went to the ftp server with the packages no problem. Thanks for your help.

Now I can download all of the FreeBSD packages and create my own FreeBSD packages repository.

Now how do you make it the main web site, I don't have to tell it where to go every time I want to install packages. Weather it may be from the sysinstall method or just by using the command line.
baronobeefdip
Member
 
Posts: 123
Joined: 04 Nov 2010, 22:49

Postby baronobeefdip » 07 Feb 2012, 17:18

Alright I have a problem. When I mimicked the file structure like I posted with the packages I downloaded from the ftp server for FreeBSD. It's telling me that nothing can be found. Here's what I made as the file structure.
Code: Select all
/usr/home/ftp/pub/FreeBSD/releases/i386/8.2-RELEASE/packages

This worked with the packages that I found on the DVD, but when I tried it with the packages from the ftp server it didn't work. What could be the problem here?
baronobeefdip
Member
 
Posts: 123
Joined: 04 Nov 2010, 22:49

Postby SirDice » 07 Feb 2012, 17:42

Most likely it's the symbolic links in [file]${PKGDIR}/Latest/[/file] or the various categories.

Everything points to the real file that's in [file]${PKGDIR}/All/[/file], for example:

Code: Select all
dice@williscorto:/usr/ports/packages>ls -l Latest/firefox.tbz
lrwxr-xr-x  1 root  wheel  25 Feb  4 23:28 Latest/firefox.tbz -> ../All/firefox-10.0,1.tbz
dice@williscorto:/usr/ports/packages>ls -l www/firefox-10.0,1.tbz
lrwxr-xr-x  1 root  wheel  25 Feb  4 23:28 www/firefox-10.0,1.tbz -> ../All/firefox-10.0,1.tbz
dice@williscorto:/usr/ports/packages>ls -l All/firefox-10.0,1.tbz
-rw-r--r--  1 root  wheel  31583454 Feb  4 23:28 All/firefox-10.0,1.tbz


You normally set the [file]PACKAGESITE[/file] variable to the [file]/Latest/[/file] directory. Also note that the files in [file]${PKGDIR}/Latest/[/file] don't include a version number. This is done so you can do [file]pkg_add -r firefox[/file] instead of having to do [file]pkg_add -r firefox-10.0,1[/file].

If you have this stuff mounted with NFS you can just do:
[cmd=#]pkg_add /usr/ports/packages/All/firefox-10.0,1[/cmd]
Command/file completion will take the grunt of the typing ;)
Senior UNIX Engineer at Unix Support Nederland
Experience is something you don't get until just after you need it.
User avatar
SirDice
Old Fart
 
Posts: 16185
Joined: 17 Nov 2008, 16:50
Location: Rotterdam, Netherlands

Postby baronobeefdip » 16 Feb 2012, 21:15

Alright I just downloaded the [FILE]INDEX[/FILE] files to my ftp server (Running FreeBSD 9) with anonymous access. The client was able to pick up the [FILE]INDEX[/FILE] file and read it then display the contents of the package repository. So all is left is to download the packages but there is just one problem. In the directories inside the [FILE]packages[/FILE] folder, besides the folder named [FILE]ALL[/FILE] the other folders contain symbolic links instead of actual packages. Whenever I tried downloading them with Filezilla, it will download the actual packages instead of just pulling down the symbolic links. This is a problem because of memory preservation, I want these directories to have the symbolic links like the real server so I don't have to take up as much space, And I will have room to store programs for previous and possibly future releases of FreeBSD
baronobeefdip
Member
 
Posts: 123
Joined: 04 Nov 2010, 22:49

Postby Beeblebrox » 18 Feb 2012, 14:02

most of the activities we do might damage the network

An example of how to gather intel :)
11-CURRENT_amd64_root-on-zfs_RadeonKMS
User avatar
Beeblebrox
Member
 
Posts: 838
Joined: 03 Sep 2010, 04:45

Postby baronobeefdip » 22 Feb 2012, 02:22

How do you download all the packages from the ftp site with command line, Plus is it possible to make this command display a progress bar?
baronobeefdip
Member
 
Posts: 123
Joined: 04 Nov 2010, 22:49

Postby baronobeefdip » 02 Mar 2012, 18:12

Alright I sort of found a solution to my symbolic links problem but there isn't any easy to follow or helpful sources on how to do this. I now want to mirror the FreeBSD ftp server for the packages so that I can create a local software repo server. What I am planning to accomplish with with method is that unlike the Filezilla method, Instead of downloading the symbolic links as the packages them selves which takes up more memory than it should. It will download the symbolic links as symbolic links instead of the actual package. This is what I plan on doing but only if I can figure out how to do it.

I have heard of people having some luck with [FILE]rsync[/FILE]. As I mentioned earlier is that there is very little documentation on it, Nor there is anything useful and easy to follow on how to mirror the FreeBSD repo with [FILE]rsync[/FILE]. I even read what was in the FreeBSD handbook and there isn't anything that is easy to understand. I at least want a dissection of the command when you download the packages for use as a local repo server. This is the url of the repo server I plan on trying to download
ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/9.0-RELEASE/packages
This is the url I plan on downloading the packages and symbolic links from with [FILE]rsync[/FILE], I want it to contain the symbolic links like the real server on the FreeBSD official web site.
baronobeefdip
Member
 
Posts: 123
Joined: 04 Nov 2010, 22:49


Return to Web & Network Services

Who is online

Users browsing this forum: No registered users and 0 guests