1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How do I create a FreeBSD software repository server with packages

Discussion in 'Web and Network Services' started by baronobeefdip, Jan 28, 2012.

  1. baronobeefdip

    baronobeefdip New Member

    Messages:
    123
    Likes Received:
    0
    I work in a networking lab and those who use it including myself are diving into using FreeBSD. The way we are set up is we are isolated from the network running on the school for safety purposes since most of the activities we do might damage the network so we run on our own, But we don't have internet so we can't access packages.

    I am able to create a software repository for Debian and Fedora just fine to where the internet isn't required to install software that wasn't installed on the os by default. But with making the FreeBSD package repository server I am at a barrier with it. What do I need to do in order to have a local package repository server that we can access whenever we need software that wasn't installed by default in a typical FreeBSD install. And how do I direct the FreeBSD machines to retrieve index files from the server by default so the local package repository is the default server the computer will retrieve software from?
     
  2. da1

    da1 Member

    Messages:
    756
    Likes Received:
    0
    I would first create a publicly available FTP that will contain all the necessary packages. Then I would make sure that my isolated FreeBSD machine has network access to that FTP server and then I would set the PKG_PATH shell environment to point to that server.

    More or less, I think this is all you need to do.
     
  3. baronobeefdip

    baronobeefdip New Member

    Messages:
    123
    Likes Received:
    0
    I did do that but I get the same error every time. I am doing the tests with the package collection that is on the DVD,I added the entire packages into the /home/ftp directory. I am using the installation utility for packages in sysinstall, I ran sysinstall and chose an alternative ftp site, Entered my url for the ftp server "ftp://192.168.1.101". It connected to the ftp server fine but then it tells me that
    Code:
    9.0 RELEASE Couldn't be located
    what does this mean and how do I get rid of this issue so that I can finish setting up the package repository server?
     
  4. fonz

    fonz Moderator Staff Member Moderator

    Messages:
    2,437
    Likes Received:
    5
    Did you mimic the directory structure? It's probably looking for ftp://192.168.1.101/pub/FreeBSD/releases/i386/9.0-RELEASE/packages or something along those lines.

    Fonz
     
  5. jb_fvwm2

    jb_fvwm2 Member

    Messages:
    1,565
    Likes Received:
    0
    I had a similar question. A few weeks ago began using a thumbdrive:
    ( seven years ago or so mucked around with NFS, may try that again...)
    Code:
    pkg_delete -f /var/db/pkg/gcc-4.6.3.20111201 && pkg_add [file]/mnt/packages/[/file]gcc-4.6.3.20120113.tbz 
    *GREATLY* seems to simplify the process across multiple machines, if your shell has tab completion, history recall, etc etc. One can have the /usr/ports/packages/All *mostly* vacant in fact on each machine, transferring packages to the thumbdrive. (At least it is simpler to setup initially). While I'm at it I may as well show a method to prune the /mnt/packages/ from extra duplicates which will accumulate...: (requires two additional ports installed: )
    Code:
    gnuls -oSr | sort -k 4 | sort -k 8 | lookat 
    ... one can easily check for duplicates in the pane shown at the result...
     
  6. SirDice

    SirDice Moderator Staff Member Moderator

    Messages:
    17,375
    Likes Received:
    3
    If you have your own network why not use that? No need to fumble about with thumb drives :e

    Just NFS export /usr/ports/ and/or /usr/ports/packages/. If you're worried about security you can export them read-only.

    I recommend not using sysinstall at all. But I think it's looking for an INDEX file. Not sure if you can just copy the one in /usr/ports/.
     
  7. baronobeefdip

    baronobeefdip New Member

    Messages:
    123
    Likes Received:
    0
    It just worked! After I mimicked the filesystem you posted it went to the ftp server with the packages no problem. Thanks for your help.

    Now I can download all of the FreeBSD packages and create my own FreeBSD packages repository.

    Now how do you make it the main web site, I don't have to tell it where to go every time I want to install packages. Weather it may be from the sysinstall method or just by using the command line.
     
  8. baronobeefdip

    baronobeefdip New Member

    Messages:
    123
    Likes Received:
    0
    Alright I have a problem. When I mimicked the file structure like I posted with the packages I downloaded from the ftp server for FreeBSD. It's telling me that nothing can be found. Here's what I made as the file structure.
    Code:
    /usr/home/ftp/pub/FreeBSD/releases/i386/8.2-RELEASE/packages
    This worked with the packages that I found on the DVD, but when I tried it with the packages from the ftp server it didn't work. What could be the problem here?
     
  9. SirDice

    SirDice Moderator Staff Member Moderator

    Messages:
    17,375
    Likes Received:
    3
    Most likely it's the symbolic links in ${PKGDIR}/Latest/ or the various categories.

    Everything points to the real file that's in ${PKGDIR}/All/, for example:

    Code:
    dice@williscorto:/usr/ports/packages>ls -l Latest/firefox.tbz 
    lrwxr-xr-x  1 root  wheel  25 Feb  4 23:28 Latest/firefox.tbz -> ../All/firefox-10.0,1.tbz
    dice@williscorto:/usr/ports/packages>ls -l www/firefox-10.0,1.tbz 
    lrwxr-xr-x  1 root  wheel  25 Feb  4 23:28 www/firefox-10.0,1.tbz -> ../All/firefox-10.0,1.tbz
    dice@williscorto:/usr/ports/packages>ls -l All/firefox-10.0,1.tbz 
    -rw-r--r--  1 root  wheel  31583454 Feb  4 23:28 All/firefox-10.0,1.tbz
    
    You normally set the PACKAGESITE variable to the /Latest/ directory. Also note that the files in ${PKGDIR}/Latest/ don't include a version number. This is done so you can do pkg_add -r firefox instead of having to do pkg_add -r firefox-10.0,1.

    If you have this stuff mounted with NFS you can just do:
    # pkg_add /usr/ports/packages/All/firefox-10.0,1
    Command/file completion will take the grunt of the typing ;)
     
  10. baronobeefdip

    baronobeefdip New Member

    Messages:
    123
    Likes Received:
    0
    Alright I just downloaded the INDEX files to my ftp server (Running FreeBSD 9) with anonymous access. The client was able to pick up the INDEX file and read it then display the contents of the package repository. So all is left is to download the packages but there is just one problem. In the directories inside the packages folder, besides the folder named ALL the other folders contain symbolic links instead of actual packages. Whenever I tried downloading them with Filezilla, it will download the actual packages instead of just pulling down the symbolic links. This is a problem because of memory preservation, I want these directories to have the symbolic links like the real server so I don't have to take up as much space, And I will have room to store programs for previous and possibly future releases of FreeBSD
     
  11. Beeblebrox

    Beeblebrox Member

    Messages:
    868
    Likes Received:
    1
    An example of how to gather intel :)
     
  12. baronobeefdip

    baronobeefdip New Member

    Messages:
    123
    Likes Received:
    0
    How do you download all the packages from the ftp site with command line, Plus is it possible to make this command display a progress bar?
     
  13. baronobeefdip

    baronobeefdip New Member

    Messages:
    123
    Likes Received:
    0
    Alright I sort of found a solution to my symbolic links problem but there isn't any easy to follow or helpful sources on how to do this. I now want to mirror the FreeBSD ftp server for the packages so that I can create a local software repo server. What I am planning to accomplish with with method is that unlike the Filezilla method, Instead of downloading the symbolic links as the packages them selves which takes up more memory than it should. It will download the symbolic links as symbolic links instead of the actual package. This is what I plan on doing but only if I can figure out how to do it.

    I have heard of people having some luck with rsync. As I mentioned earlier is that there is very little documentation on it, Nor there is anything useful and easy to follow on how to mirror the FreeBSD repo with rsync. I even read what was in the FreeBSD handbook and there isn't anything that is easy to understand. I at least want a dissection of the command when you download the packages for use as a local repo server. This is the url of the repo server I plan on trying to download
    ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/9.0-RELEASE/packages
    This is the url I plan on downloading the packages and symbolic links from with rsync, I want it to contain the symbolic links like the real server on the FreeBSD official web site.