Need script to count earthquakes

J

jonfr

It is clear that my script to count earthquakes are not working with cron. So I need new scripts to do the job, I am sadly not a programmer (or really bad one at best).

The problem is that mrtg is showing zero value when I run the script in cron. The script, when run manual shows current number, here is the script in question.

Code: 
#!/bin/sh

#earthquake count shell V0.0.3

URL="http://hraun.vedur.is/ja/englishweb/eqlist.html"
TODAYSDATE=`date +%Y-%m-%d`
#TODAYSDATE=`date +%Y/%m/%d`
#TODAYSDATE=`date +%Y-%m-%d`

COUNT=`lynx -dump ${URL} | grep ${TODAYSDATE} | grep -vi updated | wc -l`
#COUNT=`wget -O ${URL} | grep ${TODAYSDATE} | grep -vi updated | wc -l`
echo ${COUNT}
echo ${COUNT}
#echo ${COUNT}
#echo "uptime"
#echo "hostname"

exit 0

When I run the script manually, I get this data from it.

Code: 
./eq.imo.sh 
320
320

When I run it with cron, I get the result shown in the image attachment. Here is also the config file for mrtg.

Code: 
#---------Apache hits-----------------------
WorkDir: /usr/local/www/apache24/data/mrtg/
Target[localhost_eq]: `/usr/local/etc/mrtg/eq.imo.sh`
Options[localhost_eq]: gauge, integer ,perminute, nopercent, growright, noinfo
MaxBytes[localhost_eq]: 100000
YLegend[localhost_eq]: count
#Colours[localhost_eq]: GREEN#458B00
ShortLegend[localhost_eq]: quakes   
LegendO[localhost_eq]: Earthquakes:
LegendI[localhost_eq]: Earthquakes:
Legend2[localhost_eq]: Earthquakes pr hour
Legend4[localhost_eq]: Max number of earthquakes
Title[localhost_eq]: Number of earthquakes Iceland
WithPeak[localhost_eq]: wmy
PageTop[localhost_eq]: <h1>Earthquakes Iceland Automatic </h1>
#------------End Apache Hits------------------

I was thinking it would be a good idea to get the data into a simple text file, rather then to have mrtg read it from the shell, as that might be the actual problem. I am however not sure on that detail.

Thanks for the help.
 

Attachments

  • localhost_eq-day.png
    localhost_eq-day.png
    1.6 KB · Views: 573
This seems to be working as expected. It's showing the delta between the previous reading and the current reading. What output did you expect?
gauge
Treat the values gathered from target as 'current status' measurements and not as ever incrementing counters. This would be useful to monitor things like disk space, processor load, temperature, and the like ...

In the absence of 'gauge' or 'absolute' options, MRTG treats variables as a counters and calculates the difference between the current and the previous value and divides that by the elapsed time between the last two readings to get the value to be plotted.
Click to expand...
Interesting graph output, then. ><
 
When I was running Gentoo Linux the counter did go up, not just spike as it is now doing. I need to have it that way, so I can track the increase in earthquakes during the day. This is important since it allows me to track earthquake swarms when they happen and how fast they are taking place.

I just need to know what to fix and how to fix it. Thanks.

It also needs to start at 0 at midnight every day, then add the number up as earthquakes take place during the day.
 
No, it #6. A new version was issued with changed functionality. Now it shows me earthquakes during 1 hour in spikes. Rather then in build up graph as before. I do not know the difference, but something was clearly changed and there is nothing I can do about it.

So I adjust to the change that was made, or I try to find a solution that works.
 
You have to use full paths, and you might want to make sure to check /usr/local/bin/ also. Third party installations from ports install with --prefix=/usr/local/ as opposed to Gentoo Linux which uses --prefix=/usr/.
 
jnbek said:
you have to use full paths, and you might want to make sure to check /usr/local/bin/ also. 3rd party installations from ports install with
--prefix=/usr/local/
as opposed to Gentoo Linux which uses:
--prefix=/usr/
Click to expand...

Then in mrtg script part? From the cron part, everything is running correctly and I use full paths in that, I use webmin to set up cron jobs.
 
I have been looking closer into this problem, it seems that mrtg is ignoring the output from the shell, I use sh for the shell in this case. I do not know why it is, and I am not sure I can fix it. If that is actually the case, I might have to give up using FreeBSD for this.
 
I see no other option than to use Gentoo Linux again. I have not found any solution for this issue, even after some heavy search into this issue. This counter is important for my work (watching earthquakes), so I must have it up and running.

Thanks for all the help.
 
I was half way trough a Gentoo Linux install when I did just quit it. The mess was the same, even on a new install (the Linux install process, not the script I am speaking about here).

I am not sure how I am going to solve this with FreeBSD, but I am going to solve it. After I re-install FreeBSD on the server computer again, it is going to take me few hours to get all the programs that I use back in and configure them.

I was thinking about using maybe Python or some other good script language for this job, rather then a shell program that does clearly not work in FreeBSD.
 
wblock@ said:
You just proved that the problem was not FreeBSD-specific.
Click to expand...

Sure, this problem is related to sh/MRTG.

@jonfr,

Check if eq.imo.sh is executable and run it once to test % usr/local/apache2/htdocs/mrtg ./eq.imo.sh. No output is good, this means that the script ran MRTG and updated the data and log files with no problems.
 
Last edited by a moderator:
wblock@ said:
You just proved that the problem was not FreeBSD-specific.
Click to expand...
It is actually, not sure where the issue is. This problem comes around because sh does not pass the information to mrtg to read them. Earlier I was speaking about state of Gentoo Linux as I see it, not the script problem that I have been dealing with here.

I am not sure if making a log file is going to solve this issue, but I am going to look at it at later time, once I have everything set up again (working on it now).
 
Judging from your script and sample graph, your script is getting total number of earthquakes in one day, when in fact, you want 1 hour precision, so your error is semantic.

I think you don't really need cron to accomplish your tasks.

In shell scripting, there are many different ways to perform a certain tasks, I'm not saying mine is the best one, but I tried it now, and it seems to work. I took the liberty of modifying your script a little, and this is what I got.

The earthcount.sh script:
Code: 
# earthcount.sh
#!/usr/local/bin/bash

URL="http://hraun.vedur.is/ja/englishweb/eqlist.html"
TODAY=`date +%Y-%m-%d`
HOURS=`date +%H`
MINUTES=`date +%M`
OUT="$TODAY.txt"

NOW=`echo "($HOURS * 60) + $MINUTES" | bc`
EARTHNUM=`wget --quiet -O - $URL | grep $TODAY | grep -vci updated`

touch $OUT
echo "$NOW     $EARTHNUM" >> $OUT

exit 0

so this will create file with today's date (timestamp is always good), the file will have two colums:
Code: 
<current daytime in minutes>    <cumulative number of earthquakes>

I think this is sort of what you wnat. You can later process this file. If you want you can create another script to run it or you can run it inside this simple script, called runner.sh,

Code: 
# runner.sh -- runs the earthcount.sh script
#!/bin/csh

while (1)
   ./earthcount.sh
   sleep 3600
end

exit 0

You can actually do it all in one script if you prefer. Every 3600 seconds, or every hour, it will check and update the file.

When the new day comes, you it will automatically create a new file, and the web site will reset the EARTHNUM to zero anyway. You can modify the earthcount.sh a little, to get the number of earthquakes in that particular hour, not all the earthquakes up to that hour (to get rid of the cumulative number, and get "bins" instead).

To run the script, just use $ ./runner.sh &.
 
I am getting errors with the script.

Code: 
./eq.imo.sh
./eq.imo.sh: command substitution: line 10: unexpected EOF while looking for matching `"'
./eq.imo.sh: command substitution: line 11: syntax error: unexpected end of file
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
./eq.imo.sh: line 17: $OUT: ambiguous redirect

I am not sure what the fault is, since I am poor programmer at the moment.
 
Thanks, but I am still getting errors when I run the script.

Code: 
./eq.imo.sh
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
./eq.imo.sh: line 17: $OUT: ambiguous redirect
 
Code: 
# earthcount.sh
#!/bin/sh

URL="http://hraun.vedur.is/ja/englishweb/eqlist.html"
TODAY=`date +%Y-%m-%d`
HOURS=`date +%H`
MINUTES=`date +%M`
OUT="$TODAY.txt"

NOW=`echo "($HOURS * 60) + $MINUTES" | bc`
EARTHNUM=`wget --quiet -O - $URL | grep $TODAY | grep -vci updated`

if [ $OUT ]; then
   touch $OUT
fi

echo "$NOW     $EARTHNUM" >> $OUT

exit 0

Code: 
[CMD]% cat 2013-05-14.txt[/CMD] 
520     3
 
@ondra_knezour, thanks :D didn't see I put a buggy version there :D

@cpu82, I put that `if' clause there, in case there is no such file. Some shells will create one, if you use `>>', and some won't, so I wasn't sure how is it for bash. I know the `-a' flag in the `if' clause is present for the bash and zsh, but definitely not for sh. However, I guess it can be omitted completely, since `touch' won't do anything to the data.

@jonfr I tried it at home letting it run for a few hours and it was working normally, plus the end result (plot) was something like you suggested. I edited my previous post, and hopefully the script will run normally for you now. Make sure the shells are the same as I suggested in the script, so the first one is actually a bash and the second one is a csh script.
 
Last edited by a moderator:
Thanks, it works now without a issue. I just have one more question. Is there any good way to remove the file once it hits midnight, or is it replaced automatically after midnight by the script and the old file deleted?

Thanks again. :)
 
The filename is "$TODAY.txt". So it creates a new file for each day. If you want, you can keep your measurements all in one file too, but then you'd have to figure out a different way to keep data in your file -- my way only has two columns and I guess you're going to do some statistics research for each month as well ...

Still, like I said before, the script is giving you a cumulative distribution of earthquakes over time, rather than just a plain time distribution (you can see this in your old graphs too) which in my opinion is unwise, so you may want to fix that.

I gave you the way, but I recommend you improve on it yourself:
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://www.bolthole.com/solaris/ksh.html
http://www.csem.duke.edu/Cluster/csh_basics.htm

There is really a ton of documentation for shell scripting, be it ksh, zsh, bash, csh ... It will do you more good if you read through it, than have me baby feed you ;) you know, Sun Tzu and stuff :D -- Give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime.
 
I am not using this data for any statistical research, that is the field of Icelandic Meteorological Office. I do use this to keep track of earthquake swarms in Iceland. As such it is highly useful to me, it tells me how big the swarms are and how long they last. Too bad for me am a really bad programmer, even if I am good at computers. I lack any math skills, even if I am good at logical thinking. i am not going to go into it here for many reasons.

I changed this in the script and it seems to work. I guess the file would be overwritten at midnight.

Code: 
OUT="today.is.txt"

I have tested the script and it works when I run it from the command line, I have not yet tested it with mrtg. I just have not had the time to set it up again. I am hoping that is going to happen tonight.

Thanks for the help. I did read the bash link, but I did not find anything about this type of script there. I am going to read it again later tonight.
 
Oh, nice, your work sounds really exciting! Well the thing is, if you already know your way in the command line, you just have to know how to use `for' and `while' loops, plus the `if' statement, to know how to write basic shell scripts like these. The rest will come with practice, or by reading other people's scripts (it's one of the benefits of the open source operating systems ;)).

Anyway, good luck, and happy hacking :beer

PS: if you want to learn more about computers, and not just shells, and write more portable programs, then you might want to try good old C. ;)
 
You must log in or register to reply here.
Back
Top