Something More Revelant then BSDSTATS.org

vermaden

vermaden

... recalling the recent thread about BSD usage statistics:
http://forums.freebsd.org/showthread.php?t=14522

Generally http://bsdstats.org is only known by BSD users (and definitely not by all of them), so even having BIG stats out there is more or less pointless.

But there is other way to 'impress' other people with BSD stats ... http://distrowatch.com portal. It mainly focuses on Linux distributions but it also gathers stats for BSDs and OpenSolaris/Solaris 'distributions'.

First we would need to add www/lynx package (it does not need x11 dependency as www/links):
# pkg_add -r lynx

Having this simple line in all machines attached to network for 24/7 will add actual BSD users/machines count to DistroWatch portal stats:
[CMD=""]0 4 * * * /usr/local/bin/lynx -dump -useragent="Lynx $( uname -spr )" http://distrowatch.com > /dev/null[/CMD]

Laptop users will propably have their system up and running in different times of the day/night, so it would be better to put such line into the /etc/rc.local file, so their system will 'send' stats at boot:
[CMD=""]/usr/local/bin/lynx -dump -useragent="Lynx $( uname -spr )" http://distrowatch.com > /dev/null &[/CMD]

Such 'lynx dump' will create that line in the logs:
[cmd=""]8.8.8.8 - - [30/Jun/2010:14:15:24 +0200] "GET / HTTP/1.0" 200 75 "-" "Lynx FreeBSD 8.0-RELEASE-p3 amd64"[/CMD]

... so it definitely works.

We can even put our favorite browser to increase its 'stats':
% lynx -dump -useragent="Opera/9.80 (X11; $( uname -spr ); U; en) Presto/2.2.15 Version/10.00" [url]http://distrowatch.com[/url]

... and here is the 'opera' result:
[CMD=""]8.8.8.8 - - [30/Jun/2010:14:19:21 +0200] "GET / HTTP/1.0" 200 75 "-" "Opera/9.80 (X11; FreeBSD 8.0-RELEASE-p3 amd64; U; en) Presto/2.2.15 Version/10.00"[/CMD]

Some may say 'isnt that cheating?', definitely not, we only appear one enter per one machine a day, and this DistroWatch stats are about that, 'hits per day' count about various operating systems.
 
Good one, Vermaden. Those wanting to run 'links' on servers: don't use the package (which has X11 'on'), use the port: remove the X11 options (which are on by default).
 
DutchDaemon said:
Good one, Vermaden. Those wanting to run 'links' on servers: don't use the package (which has X11 'on'), use the port: remove the X11 options (which are on by default).
Click to expand...

Other sollution may be just adding the www/links package without dependencies:
[cmd='']# pkg_add -i ftp://ftp.freebsd.org/pub/FreeBSD/ports/$( uname -m )/packages-8.0-release/Latest/links.tbz[/CMD]

The best sollution seems to be www/lynx with that arguments:
% lynx -dump -useragent="Lynx $( uname -spr )" [url]http://distrowatch.com[/url]

... which will left this in logs:
[CMD=""]8.8.8.8 - - [30/Jun/2010:14:15:24 +0200] "GET / HTTP/1.0" 200 75 "-" "Lynx FreeBSD 8.0-RELEASE-p2 amd64"[/CMD]

Going to edit the 1st post to make it 'lynx by default'.

... we can even put our favorite browser to increase its 'stats':
% lynx -dump -useragent="Opera/9.80 (X11; $( uname -spr ); U; en) Presto/2.2.15 Version/10.00" [url]http://distrowatch.com[/url]
 
Yeah, tested fetch too, but that was equally useless. Not sure what wget reports, don't use it.
 
ftp/wget does a wonderful job of the same task

[cmd=""]wget -O /dev/null http://server/file.html[/cmd]

Output from httpd-access.log
Code: 
MYIP - - [30/Jun/2010:14:41:11 +0200] "GET /file.html HTTP/1.0" 301 239 "-" "Wget/1.12 (freebsd8.0)"
MYIP - - [30/Jun/2010:14:41:12 +0200] "GET /file.html HTTP/1.0" 200 1012858

[cmd=""]wget -O /dev/null --user-agent="`uname -spr`" http://server/file.html[/cmd]

Output from httpd-access.log
Code: 
MYIP - - [30/Jun/2010:14:44:27 +0200] "GET /file.html HTTP/1.0" 301 239 "-" "FreeBSD 8.1-PRERELEASE amd64"
MYIP - - [30/Jun/2010:14:44:27 +0200] "GET /file.html HTTP/1.0" 200 1012858
 
mix_room said:
Code: 
MYIP - - [30/Jun/2010:14:41:11 +0200] "GET /file.html HTTP/1.0" 301 239 "-" "Wget/1.12 (freebsd8.0)"
MYIP - - [30/Jun/2010:14:41:12 +0200] "GET /file.html HTTP/1.0" 200 1012858
Click to expand...

But there is a question if 301 RETURN values are evaluated into stats, cause 200 for sure are, but there is no 'os data' by the 200 value with wget, wget can also be 'blocked for stats' since its not a browser.
 
You'll need to set HTTP_USER_AGENT for /usr/bin/fetch to report something useful. From fetch(3):
Code: 
     HTTP_USER_AGENT     Specifies the User-Agent string to use for HTTP
                         requests.  This can be useful when working with HTTP
                         origin or proxy servers that differentiate between
                         user agents.
 
crsd said:
You'll need to set HTTP_USER_AGENT for /usr/bin/fetch to report something useful. From fetch(3):
Code: 
     HTTP_USER_AGENT     Specifies the User-Agent string to use for HTTP
                         requests.  This can be useful when working with HTTP
                         origin or proxy servers that differentiate between
                         user agents.
Click to expand...

tested
Code: 
$ export HTTP_USER_AGENT="Test Agent"; fetch http://www.bsdroot.lv

logs show
Code: 
83.241.11.135 - - [30/Jun/2010:16:52:51 +0300] "GET / HTTP/1.1" 200 1342 "-" "Test Agent"

FOR crontab -e:
Code: 
0 20 * * * /bin/sh -c "export HTTP_USER_AGENT=\"Opera/9.80 (X11; $( uname -spr ); U; en) Presto/2.2.15 Version/10.10\"; /usr/bin/fetch -o /dev/null http://distrowatch.com > /dev/null 2> /dev/null"
or /etc/crontab
Code: 
0 20 * * * nobody /bin/sh -c "export HTTP_USER_AGENT=\"Opera/9.80 (X11; $( uname -spr ); U; en) Presto/2.2.15 Version/10.10\"; /usr/bin/fetch -o /dev/null http://distrowatch.com > /dev/null 2> /dev/null"
something like this
 
@crsd

It seems that best sollution is even in the FreeBSD's base system, thanks for great suggestion.
 
The distrowatch site expects to be monitoring the web browser their visitors use. Not a bunch of $X fanboys artificially pushing their OS/browser to the highest possible position.
IMHO this is stats poisoning ...
 
let's make botnet and change every browser to report that is run on fbsd machine.
why you want those 'hits per day' ?
 
Carpetsmoker said:
The distrowatch site expects to be monitoring the web browser their visitors use. Not a bunch of $X fanboys artificially pushing their OS/browser to the highest possible position.
IMHO this is stats poisoning ...
Click to expand...

I agree, but still enjoy this.

maybe integrate this in bsdstats port??
Click to expand...

I agree with this and we should let the maintainers of bsdstats know about it.
 
vermaden, to figure one's own User-Agent you don't have use www server
$ true | nc -lk 3333 &
and visit http://0:3333. It would display smth like this
Code: 
$ fetch http://0:3333
GET / HTTP/1.1
Host: 0:3333
User-Agent: fetch libfetch/2.0
Connection: close
or from a browser
Code: 
GET / HTTP/1.1
Host: 0:3333
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; en-US; rv:2.0b2pre) Gecko/20100704 Conkeror/0.9.2 Firefox/4.0b2pre
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
 
You must log in or register to reply here.
Back
Top