Solved freebsd-update to new release behind proxy

Hi there,

I have a FreeBSD for developing purposes running as a client in an Oracle Virtualbox with version 11.0
From time to time I do freebsd-update(8) to get the kernel up-to-date, but for some weeks now it urges me to upgrade to release-11.1.

This is the result:
Code:
root@cookiebox # freebsd-update -r 11.1-RELEASE upgrade 
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... none found.
Fetching metadata signature for 11.0-RELEASE from update.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files... done.
Inspecting system... done.

The following components of FreeBSD seem to be installed:
kernel/generic world/base world/lib32

The following components of FreeBSD do not seem to be installed:
kernel/generic-dbg world/base-dbg world/doc world/lib32-dbg


Fetching metadata signature for 11.1-RELEASE from update.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files... done.
Inspecting system... done.
Fetching files from 11.0-RELEASE for merging... done.
Preparing to download files... done.
Fetching 149 patches... done.
Applying patches... done.
Fetching 367 files... failed.

root@cookiebox:/usr/home/hennes #

I tried again and again, but I can't download the required files.

I set the company proxy in the environment variables ftp_proxy, http_proxy and https_proxy. I tried in the early morning, at noon, and just before I leave the office.

Any chance to make it work OR to do the process step-by-step manually?

Hennes
 
From time to time I do freebsd-update(8) to get the kernel up-to-date
Excellent. But note that you're not only updating the kernel, you're updating the OS (which includes the kernel). FreeBSD is more than just a kernel, it's a complete OS.

In any case, freebsd-update(8) shouldn't have a problem with a proxy. It basically uses fetch(1) (plain HTTP GET) to get a bunch of files to apply. But if the proxy has some kind of content scanning it may detect false positives and hence fail to correctly download a patch file. Have you talked to the admins of the proxy? Perhaps they can shed some light on the fetch failures as they should be able to look through the logs of the proxy.

Another possibility is that the patch file got corrupted during download and the proxy has now cached the 'broken' file. Every time you try to get the file you get the corrupted one from the cache. Your proxy admins should be able to help with that too.
 
Our proxy has no authentication enabled, but I set HTTP_PROXY and HTTP_PROXYS as well. No change :(
Our admins were so kind to clear the cache, but that also didn't help.
I tried to download some files with "fetch", via http:// and ftp:// as well. Successful !!

Maybe I'm using a "bad" mirror?
 
Code:
root@cookiebox:/usr/home/hennes # /usr/libexec/phttpget www.openbsd.org /index.html
http://www.openbsd.org//index.html: 200 OK
root@cookiebox:/usr/home/hennes #

Strange enough, I still can install any package like
Code:
root@cookiebox:/usr/home/hennes # pkg install bind-tools
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 3 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
    bind-tools: 9.11.2
    idnkit: 1.0_6
    json-c: 0.12.1

Number of packages to be installed: 3

The process will require 39 MiB more space.
4 MiB to be downloaded.

Proceed with this action? [y/N]: y
[1/3] Fetching bind-tools-9.11.2.txz: 100%    4 MiB   4.0MB/s    00:01
[2/3] Fetching idnkit-1.0_6.txz: 100%  194 KiB 198.3kB/s    00:01
[3/3] Fetching json-c-0.12.1.txz: 100%   38 KiB  39.2kB/s    00:01
Checking integrity... done (0 conflicting)
[1/3] Installing idnkit-1.0_6...
[1/3] Extracting idnkit-1.0_6: 100%
[2/3] Installing json-c-0.12.1...
[2/3] Extracting json-c-0.12.1: 100%
[3/3] Installing bind-tools-9.11.2...
Extracting bind-tools-9.11.2: 100%

Okay, here is a first clue:
freebsd-update(8) calls /usr/libexec/phttpget with a zillion of .gz file parameters. phttpget terminates with "Connection failure".

I tried to reduce to only one parameter:
Code:
root@cookiebox:/usr/home/hennes # /usr/libexec/phttpget update.freebsd.org 11.1-RELEASE/amd64/f/08baff81d2cabb88d7c636c62997af842ea7260035866dd2130fcd85cf6cc4b1.gz
phttpget: Connection failure
root@cookiebox:/usr/home/hennes #

Now I give fetch a try:

Code:
root@cookiebox:/usr/home/hennes # fetch http://update.freebsd.org/11.1-RELEASE/amd64/f/08baff81d2cabb88d7c636c62997af842ea7260035866dd2130fcd85cf6cc4b1.gz
08baff81d2cabb88d7c636c62997af842ea7260035866d100% of   10 MB 6094 kBps 00m02s
root@cookiebox:/usr/home/hennes #

Surprise....

phttpget(8) reveals:

Code:
ENVIRONMENT
     HTTP_PROXY       URL of the proxy to use for HTTP requests.

     HTTP_PROXY_AUTH  Authorization parameters for the HTTP proxy.

     HTTP_USER_AGENT  The User-Agent string to use for HTTP requests.  The
                      default is ``phttpget/0.1''.

     HTTP_TIMEOUT     Timeout for HTTP request in seconds.

So I set the timeout to 1h (3600s):

Code:
root@cookiebox:/usr/home/hennes # setenv HTTP_TIMEOUT 3600
root@cookiebox:/usr/home/hennes # /usr/libexec/phttpget update.freebsd.org 11.1-RELEASE/amd64/f/08baff81d2cabb88d7c636c62997af842ea7260035866dd2130fcd85cf6cc4b1.gz
http://update.freebsd.org/11.1-RELEASE/amd64/f/08baff81d2cabb88d7c636c62997af842ea7260035866dd2130fcd85cf6cc4b1.gz: 200 OK
root@cookiebox:/usr/home/hennes #

Now I'm going to try to upgrade again...
Tadaaaaa...

Code:
Fetching metadata signature for 11.1-RELEASE from update.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files... done.
Inspecting system... done.
Fetching files from 11.0-RELEASE for merging... done.
Preparing to download files... done.
Fetching 149 patches.....10....20....30....40....50....60....70....80....90....100....110....120....130....140.... done.
Applying patches... done.
Fetching 230 files... done.
Attempting to automatically merge changes in files... done.

Although it's awfully slow!! Pity there's no European mirror :(

But it works. So thanks for pointing me in the right direction.
 
Back
Top