vps problem

cbrace · #1 August 8th, 2012, 13:12

Hi all,

I have set up a small (256M RAM) VPS at Tilaa.nl for test purposes (it has no load to speak of).

Something keeps periodically breaking. When I reboot it, it runs fine for awhile, but then things stop working. I can ssh into the server, and everything seems to be running, but server log activity has stopped nor are the nightly status mails aren't sent. When I try updating some of the installed packages using portupgrade but typically at the same two points the make command just hangs there until I put it out of its misery:

Code:

[...]
--->  Backing up the old version
--->  Uninstalling the old version
--->  Deinstalling 'portupgrade-2.4.9.5,2'
^C

Code:

[...]
Configuring OpenLDAP 2.4.32-Release ...
checking build system type... i386-portbld-freebsd9.0
checking host system type... i386-portbld-freebsd9.0
checking target system type... i386-portbld-freebsd9.0
checking for a BSD-compatible install... /usr/bin/install -c -o root -g wheel
checking whether build environment is sane... 
^C

It seems so, ah... random, I am almost embarassed to ask about it here. The server appears to be sorta working (the webserver serves up pages) yet also broken.

Any ideas what might be going on here?

$ uname -r
9.0-RELEASE

**SirDice** · #2 August 8th, 2012, 14:22

I also have a VPS with Tilaa. It's the slightly larger version than yours. I have absolutely no issues there. I started with a 8.3-RELEASE but upgraded it to 9.0-RELEASE when it came out. What version of FreeBSD did you install?

da1 · #3 August 8th, 2012, 15:20

When you get another freeze like that, try CTRL+T. It might shed some light as to why the process behaves like that.

cbrace · #4 August 8th, 2012, 17:17

@da1: thanks, will give that a try.

@SirD: I installed 9-release. Perhaps you can answer another question I have: I notice there are two network interfaces, em0 and em1. The latter has a private IP address (172.17.*.*). What is the purpose of this interface on VPS? What would be the correct firewall configuration? On my home LAN, I get it; just not sure what to do with it here. TIA

jb_fvwm2 · #5 August 8th, 2012, 18:20

With a very few ports, /usr/local/bin/grep halts the configure phase, which expects the base grep only, maybe. I don't see that in these posts, but one more thing to consider... (Similar for gcc vs clang...)

**SirDice** · #6 August 9th, 2012, 07:30

Quote:

Originally Posted by cbrace

@SirD: I installed 9-release. Perhaps you can answer another question I have: I notice there are two network interfaces, em0 and em1. The latter has a private IP address (172.17.*.*). What is the purpose of this interface on VPS?

If you have more than one VPS with them you can use the 'internal' interface to communicate with your other VPS. Data send internally isn't counted for your datalimit. So if you have two VPS you can use the internal connection for for example MySQL replication, or net/rsync.

Quote:

What would be the correct firewall configuration? On my home LAN, I get it; just not sure what to do with it here.

Pretty much the same way. Except your VPS is more likely to receive incoming traffic whereas your home LAN is more likely to have outgoing traffic.

cbrace · #7 August 10th, 2012, 16:42

Just what I needed to know, thanks. Since I only have one VPS at Tilaa, I can safely leave em1 traffic blocked in my PF config.

cbrace · #8 August 16th, 2012, 09:17

I rebooted my VPS server at Tilaa a week or so ago. It was running fine until today. Now it is once again acting weirdly. Here is something strange: the date is way off; it is running approx. 47 hours behind. Comparing local time with that on the VPS:

Code:

[colin@venus ~]$ ssh tilcara date;date
Tue Aug 14 11:42:19 CEST 2012
Thu Aug 16 10:18:53 CEST 2012

I have enabled ntdate at startup and ntpd to run continously, configuring it for the servers at nl.pool.ntp.org. Here are the ntpd entries from /var/log/messages since the last reboot:

Code:

Aug  8 14:30:32 tilcara ntpd[975]: ntpd 4.2.4p5-a (1)
Aug  8 14:36:20 tilcara ntpd[976]: time reset +2.722595 s
Aug  8 14:36:20 tilcara ntpd[976]: kernel time sync status change 2001
Aug  8 15:12:18 tilcara ntpd[976]: time reset +263.146824 s
Aug  8 15:12:18 tilcara ntpd[976]: kernel time sync status change 6001
Aug  8 15:24:51 tilcara ntpd[976]: time reset +3.570556 s
Aug  8 15:24:51 tilcara ntpd[976]: kernel time sync status change 2001
Aug  8 15:49:57 tilcara ntpd[976]: time reset +49.266041 s
Aug  9 03:33:02 tilcara ntpd[976]: time reset +4.770299 s
Aug  9 03:33:02 tilcara ntpd[976]: kernel time sync status change 6001
Aug  9 03:33:09 tilcara ntpd[976]: kernel time sync status change 2001
Aug  9 11:34:06 tilcara ntpd[976]: time reset +0.190643 s
Aug 10 00:17:56 tilcara ntpd[976]: time reset +0.201308 s
Aug 10 03:15:15 tilcara ntpd[976]: time reset +1.534728 s
Aug 10 03:53:09 tilcara ntpd[976]: time reset +2.687787 s
Aug 10 08:26:14 tilcara ntpd[976]: time reset +0.141079 s
Aug 10 12:21:18 tilcara ntpd[976]: time reset +324.847564 s
Aug 10 12:21:18 tilcara ntpd[976]: kernel time sync status change 6001
Aug 10 12:22:04 tilcara ntpd[976]: kernel time sync status change 2001
Aug 10 14:39:38 tilcara ntpd[976]: time reset +4.478887 s
Aug 11 03:14:23 tilcara ntpd[976]: time reset +1.087398 s
Aug 11 03:50:36 tilcara ntpd[976]: time reset +2.628583 s
Aug 11 04:30:39 tilcara ntpd[976]: time reset +0.322837 s
Aug 12 03:20:35 tilcara ntpd[976]: time reset +1.274951 s
Aug 12 03:37:48 tilcara ntpd[976]: time reset +2.690640 s
Aug 13 03:16:51 tilcara ntpd[976]: time reset +1.536481 s
Aug 13 03:43:30 tilcara ntpd[976]: time reset +2.580338 s
Aug 13 13:22:52 tilcara ntpd[976]: time reset +194.007063 s
Aug 13 13:22:52 tilcara ntpd[976]: kernel time sync status change 6001
Aug 13 13:23:09 tilcara ntpd[976]: kernel time sync status change 2001
Aug 14 03:12:22 tilcara ntpd[976]: time reset +1.424797 s
Aug 14 03:57:39 tilcara ntpd[976]: time reset +2.415529 s
Aug 14 06:53:08 tilcara ntpd[976]: time reset +0.181532 s
Aug 14 10:02:48 tilcara ntpd[976]: time reset +7.148770 s
Aug 14 11:30:03 tilcara ntpd[976]: time reset +0.132012 s

First, I am surprised by the big resets of 200 or 300 seconds. I am even more surprised that my server is now running ca. 47 hours behind with no indication of ntpd either having done this or corrected it.

Any thoughts on what might be going on here?

TIA

break19 · #9 August 16th, 2012, 11:51

Sounds like a problem for the vps provider to handle.. possibly hardware related.

cbrace · #10 August 16th, 2012, 20:34

Here is what Tilaa support said:

Quote:

We've heard it before a few times, FreeBSD time handling is flakey, and 9
seems worse than 8. I hope that changes if it ever gets a paravirtualized
clock.

There shouldn't be any need to run ntpd and ntpdate, we run it already on
the hypervisor and that time is correct.

Can you please check if the following sysctl is set?
kern.timecounter.hardware=i8254

I checked this and it was set.

Quote:

That's strange, because that setting fixed the problem for all customers
who told us about this issue (and we made it default after that).

I don't know if it helps, but it does seem to work with VMware ESXi, so
could you try with:

kern.timecounter.hardware=ACPI-fast

Please let us know if that works for you.

As a workaround you might consider setting a cronjob to call ntpdate every
5 minutes. If the clock drifts too much even ntpd can't fix it.

This didn't work though:

Code:

# sysctl
 kern.timecounter.hardware=ACPI-fast
 kern.timecounter.hardware:
 i8254
 sysctl: kern.timecounter.hardware: Invalid argument

Quote:

In that case I'm afraid I cannot help you. There doesn't seem to be a
FreeBSD fix for it. You might have better luck asking on a FreeBSD forum
or mailinglist. If you are able to fix it we would very much appreciate it
if you let us know how.

I should note that the only difference between your VPS and the ones of
customers who had the same issue (which was fixed when setting
kern.timecounter.hardware=i8254) is that you only have one CPU core, while
the others had 2 or more. It's quite possible an upgrade to 512Mb RAM will
fix the issue (because you'll then have 2 CPU cores instead of one), but I
cannot promise that.

Other possible workarounds:

- Set a cronjob for ntpdate every 5 minutes
- Downgrade to FreeBSD 8 (which doesn't have this issue as far as I'm
aware)
- Switch to Linux (which doesn't have the problem at all)

At this point, I am not sure what to do.

**SirDice** · #11 August 16th, 2012, 21:48

I don't run any NTP services, I actually expected the host to keep the clock correct. I haven't noticed any weird shifts in time.

Code:

root@XXXXXXX:~#uname -a
FreeBSD XXXXXXX.tilaa.nl 9.0-RELEASE-p3 FreeBSD 9.0-RELEASE-p3 #0: Wed Jun 27 19:40:57 CEST 2012     root@XXXXXXX.tilaa.nl:/usr/obj/usr/src/sys/VPS  amd64

For completeness sake, my kernel configuration: http://pastebin.com/41uUHpKj

cbrace · #12 August 16th, 2012, 22:19

You've compiled your own kernel for your VPS? May I ask why?

I've done it on a physical server to obtain ALTQ support, just wondering what the motivation would be for a VPS.

**SirDice** · #13 August 17th, 2012, 10:06

The GENERIC kernel contains some debugging stuff. I've also removed all things I don't need.

cbrace · #14 August 29th, 2012, 16:31

I just noticed that the date on my VPS at Tilaa is frozen at a single point of time:

Quote:

$ date
Tue Aug 28 10:35:34 CEST 2012

I can type date a hundred times and it still displays the same value. Bizarre!!

If I try running ntpdate, it just times out.

I haven't decided whether to try two cores as Tilaa support suggested or go back to v8 (or Linux).

cbrace · #15 September 21st, 2012, 07:31

Quote:

Originally Posted by cbrace

Just what I needed to know, thanks. Since I only have one VPS at Tilaa, I can safely leave em1 traffic blocked in my PF config.

FWIW, I have somewhat belatedly discovered that one must leave the internal interface open in PF. By default, FreeBSD is configured to use Tilaa's DNS servers; /etc/resolv.conf points to two addresses in the 172.*.*.* range. I didn't have em1 open in my PF config and it took me awhile to figure out why all sorts of things stopped working when PF was enabled

I haven't much experience with VPSs. Is this pretty standard?

Rame · #16 September 21st, 2012, 18:56

A VPS would act like a regular box so you should be blocked by default. So you should treat the VPS like you would real machine blocking by default anything that you don't need.

Regarding the Pf rules, if you don't know how to set it up you might wanna use one already made rule set like this one and change it to better suit your enviroment.

However i hardly suggest building your own, this way you'll learn a lot on how pf works saving you in the long run. Just take a look at Peter N. M. Hansteen's guide if you have time. You won't regret it.

Also not sure what kind of virtualisation Tilaa.nl uses but if it's KVM make sure you install the emulators/virtio-kmod. This guide might help you.

If you have time i hardly suggest you spend more time and learn some basic with pf, using Peter N. M. Hansteen's guide, will help you in long run.

I'm fairly new to the FreeBSD, and for me Hansteen's guide was one of the reason which make me switch many VPS's from Linux to FreeBSD as long as the visualization is KVM

cbrace · #17 September 21st, 2012, 19:58

Thanks for the advice. I bought the first edition of Hansteen's The Book of PF when it first appeared several years ago; it has been a huge help.

gkontos · #18 September 21st, 2012, 22:45

Quote:

Originally Posted by cbrace

Here is what Tilaa support said:

Quote:

- Set a cronjob for ntpdate every 5 minutes
- Downgrade to FreeBSD 8 (which doesn't have this issue as far as I'm
aware)
- Switch to Linux (which doesn't have the problem at all)

At this point, I am not sure what to do.

WTF!

Try adding this to your /boot/loader.conf, reboot and see if it helps:

Code:

kern.hz="100"

Make sure your time zone is set properly. Ask them if they use UTC or not.

Btw. Since they offer that service they have to support it.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Samba-problem 1:doesn't start, 2:host-related problem	naguz	Web & Network Services	7	September 29th, 2009 15:40
geometry problem maby? boot problem	gulanito	Installing & Upgrading	6	August 13th, 2009 16:19
Problem with configuration of 2’nd DSL link on FreeBSD 7.2 (gateway problem?)	fanz	Networking	1	August 13th, 2009 04:04