ports: parallel fetch

[Beju]

Well, this is just an idea that might be helpful to all users who don't have access to a broadband Internet connection. I don't even know if it is "implementable", but I think It's worth of trying
The idea is simple:
When You are installing more than one app from ports (i.e. the dependencies), it's a sequence of downloading and building alternately. Because some ports have very long build time especially on slower machines, that's an unnecessary waste of time. So why don't fetch the next required package while building the current? This feature is implemented i.e. in Gentoo portage. I think It'd be useful.

 

[avilla@]

of course it would be. actually, it's already in the wiki: http://wiki.freebsd.org/PortsToDo#head-0f9c53b6934169a564bd772bbee63aab6b1dfa71

 

[MG]

Then there must be some sort of mechanism that watches the fetching proces to pass information to the build proces about the downloads that are in progress. Otherwise make sees a partially downloaded distfile and tries to complete it while the other fetching process is still running. It might result in corrupted distfiles.

 

[MG]

of course it would be. actually, it's already in the wiki: http://wiki.freebsd.org/PortsToDo#head-0f9c53b6934169a564bd772bbee63aab6b1dfa71

Nice, haven't seen this doc before.

 

[avilla@]

Then there must be some sort of mechanism that watches the fetching proces to pass information to the build proces about the downloads that are in progress. Otherwise make sees a partially downloaded distfile and tries to complete it while the other fetching process is still running. It might result in corrupted distfiles.

or, you can set a "fetching" status (creating work/.fetching.port._usr_local... ports are already using those files to mark the steps of the build), and then make make(1) poll the pending ports and wait for one of them to complete and be ready for the build
and there are probably plenty of better options than this

and yes, that's a pretty interesting document... i hope some of those ideas get implemented in the future

 

[jb_fvwm2]

 wget -c -nd --limit-rate=1k http://...file...

if you have a 4k dial up connection, you can download
4 at once, each 1/4 the speed, in seperate, for
example, xterms, and if only downloading 3 at once, that
leaves 1k for browsing, (pages will load slower so you
can load them in the tabs -- background with the right
browser(s). )

 

[VictorM]

this could be interesting when deploying VMs based on NetBSD - thanks given

 

[lme@]

Thread moved to Ports section.

 

[0mp]

There is now an experimental tool called parfetch which can fetch FreeBSD ports in parallel and with connection pipelining. See https://github.com/t6/parfetch.

 

[Cath O'Deray]

http://wiki.freebsd.org/PortsToDo#head-0f9c53b6934169a564bd772bbee63aab6b1dfa71

does not exist yet