FreeBSD + internet connection frustrations

[harishankar]

Not meant to be a flame or rant or any of that sort, but I cannot seem to figure out ANY package or port management tool that will resume a broken downloads or upgrades at the correct point where it was broken in a convenient manner.

It's hellish to see a massive binary upgrade fail because fetch couldn't "resolve" the package URL even though it works on the next run of the command. I keep setting FTP_TIMEOUT for a long duration but it just simply does not wait. My internet connection is slow and not very responsive so timeout

using alias fetch='fetch -a'

also doesn't work.

Any idea HOW to get the damn system to keep retrying the connection before failing?

 

[harishankar]

Here's an error message an example of why I

harishankar# pkg_upgrade -a
fetch: [url]ftp://ftp4.freebsd.org/pub/FreeBSD/ports/amd64/packages-8-stable/All/libopenraw-0.0.8_3.tbz:[/url] No route to host
pkg_upgrade: The package <libopenraw-0.0.8_3.tbz> could not be fetched.

I simply want the damn thing to keep retrying and retrying and retrying until it works.

 

[SirDice]

"No route to host" means you don't have a connection. Or your default gateway isn't set correctly.

 

[harishankar]

"No route to host" means you don't have a connection. Or your default gateway isn't set correctly.

No. Re-trying the same URL within a few seconds works. But by then the damn command is failed and there's no way I can resume the process from where it left off.

This is a poor internet connection but I have no options. I simply want some level of "fault tolerance" by these "fetch" commands to ensure that it is retried ad infinitum or at least several times before failing.

Can you imagine the frustration when my update is almost done and then the command fails due to fetch not waiting long enough or retrying a few times?

 

[graudeejs]

mmm, search internet how to replace fetch with axel for ports fetching.
I think axel will auto resume unfinished downloads next time

 

[graudeejs]

a finally I found it
http://scratching.psybermonkey.net/...rts-simultaneously-with-multiple-connections/

 

[harishankar]

Thanks killasmurf. I definitely want to replace fetch with some other robust download manager.

But since I'm not using ports, but packages could there be an equivalent configuration.

Is FETCH_CMD an environment variable I can use?

 

[graudeejs]

I'm afraid that's not possible (however I'm not 100% sure), unless you hack pkg_add source

 

[harishankar]

I was pretty sure that fetch -a should be the trick. But either the alias is not working or the switch does not deal with this issue.

 

[harishankar]

I'm pretty sure I need to hack the source code of pkg_upgrade if I have to get it working properly. There is no way it seems to control which download manager is being used.

The damn download failed again for the umpteenth time today

 

[ondra_knezour]

Just quick idea - what about get required packages list, download them with tool which can meet your requirements better, put them in PKGDIR and then pkg_add them locally?

% pkg_add -r -n gnome2

Will run "dry run" (doesn't do anything in fact) with following output:

Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable/Latest/gnome2.tbz... Done.
Package dependency xineramaproto-1.1.2 for ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable/Latest/gnome2.tbz not found!
[...]

You can grep packages URLs from this list, fetch them and then use pkg_add without the -r option.

 

[harishankar]

Yes, but that will require writing a script which I'm not very comfortable at the moment.

Also how does a local upgrade replicate the complexity of package dependencies, or could that be built into the script somehow using external tools?

 

[ondra_knezour]

I'm leaving office just now, so only quick response.

I don't think, you will need script for this, just grep URL for each port, forward output to file using > sign and use tool, which can get download list from text file, for example ftp/wget may be able to do this.

If I'm not wrong, each package contains information about its dependencies, you can see

Package dependency xineramaproto-1.1.2 for [...] gnome2.tbz not found!

in output above. Thanks this you can get full list of required packages to download.

 

[harishankar]

I'm afraid you've got me there. I simply want to do a massive binary upgrade using pkg_upgrade -a

Any way to replace "fetch" with wget for the above script?

 

[mix_room]

Would running # portupgrade -a several times in a row work? Then all you would have to do is write a script that does the following:

for countingVariable = 1 : 10 
portupgrade -a 
end

That should be sufficiently easy that you can write it yourself.

 

[harishankar]

I tried that. It's too simplistic a solution and besides it restarts the whole process. I simply wish to find a solution whereby the download manager will not fail simply because its first attempt to fetch a package failed.

 

[ondra_knezour]

I'm afraid you've got me there. I simply want to do a massive binary upgrade using pkg_upgrade -a

No, I didn't. In fact, I was answering for similar question regarding problems with pkg_add -r. Sorry for confusion.

 

[harishankar]

Never mind. I fixed it by using HTTP repositories instead of FTP passive. Now the whole process went off fine and I've successfully installed Gnome 2.28 from FreeBSD-8-STABLE

 

[psybermonkey]

a finally I found it
http://scratching.psybermonkey.net/...rts-simultaneously-with-multiple-connections/

Hi, the link is broken. The page is at :
http://scratching.psybermonkey.net/2009/09/freebsd-download-ports-simultaneously.html

 

[davidgurvich]

Have you had any issues with the network seemingly unable to get a connection? I've had problems with some network cards where freebsd can't maintain a connection but linux or opensolaris can.