8MB Fibre Broadband to host live websites, the catch is?

zzatskl

zzatskl

My office has a BT Openreach (UK supplier) FTTC Fibre Broadband connection with at least 8MB Upload (the cabinet is outside my office). I have recently started to run some low volume production websites and I can't believe how well they are performing. Or rather there seems to be no noticeable drop in performance compared to when they were located in a hosted rack (I've never measured performance before as the sites have worked from a business perspective, I just need a few phone calls from a few people searching a few niche keywords to run my day to day business).

The upside is I can save money by not paying rent for hosting. Also, I find it really handy for my "mirco business" to have the web server on the LAN so I can easily update and amend the business website(s). I have the same FTTC line to my home and I'm thinking of installing a backup (CARP) server there. FYI, I'm this guy:

http://forums.freebsd.org/viewtopic.php?f=32&t=43592

I'm weighing up the downsides and would welcome any feedback. Obviously there is the physical security issue, but I now have the server in a locked cabinet in a locked room. I'm backing up the important data and it wouldn't be a big deal if the site(s) went down for a short time if say the line was cut in a storm.

I have started monitoring the website using these free services http://www.freesitestatus.com/en/index.php, http://www.monitor.us/en/website-monitoring and I have an app on my android phone checking for uptime too. It's been a week now and I have 100% uptime.

There must be a catch? I feel I should be looking behind my back for something obvious to go wrong. Please can anybody offer any advice or strategies for benchmarking, monitoring the server and any other tips. The web server is Apache24 and if I was to use Apache ab (Apache HTTP server benchmarking tool) or other software (?) what performance threshold should I be looking for to decide that I should go back to a rack hosting environment, ie., how can I determine the fibre lines limitations, or when will a site have too much traffic to be on the line?

Surely, there must be a serious downside? Or is Fibre Broadband the future for small (micro) business website hosting on mini PC platforms?

All comments and suggestions welcome.

Fibre Fan :stud
 
zzatskl said:
Surely, there must be a serious downside? Or is Fibre Broadband the future for small (micro) business website hosting on mini PC platforms?
Click to expand...
Not too long ago, before everybody had broadband at home, big companies would have had sites running on T1 (USA) or E1 (Europe) connections. Those were bundles of 32 ISDN lines giving a total of 2 MBit/s. Of course people have been making more and more "heavier" websites since then but I think your 8 Mbit/s connection will do fine ;)
 
zzatskl said:
Also, I find it really handy for my "mirco business" to have the web server on the LAN so I can easily update and amend the business website(s).
Click to expand...
There are security advantages to keeping the web server on a separate network from the desktops. If a desktop system is compromised it may be easier for an attacker to hit the server.
 
I've been hosting a few sites using Zen FTTC (which sits on the same BT fibre network) and everything is great tbh. I've had no downtime, excellent ping times and zero slowdown. I use pfsense to segment my network into LAN and DMZ and I've been very happy with it.

The only downside for me is that I'm on a consumer plan so if my line goes down there is no guarantee of getting a connection back quickly but it's very quick to move to another host if it should ever fail.
 
dazza said:
I use pfsense to segment my network into LAN and DMZ and I've been very happy with it.
Click to expand...

Thanks for the heads up on pfsense, I will investigate. What hardware are you using? Did you buy one from their recommended vendors http://www.pfsense.org/hardware/index.html#vendors ? I assume the hardware just connects to the LAN output at the rear of the Openreach box, as shown in the photo below:


and BTW what ping service do you use?

jrm said:
If a desktop system is compromised it may be easier for an attacker to hit the server.
Click to expand...
Thanks, I certainly appreciate the security risks v convenience. I put a credit card terminal on the LAN recently and now I have to do a full security audit with these guys https://www.trustwave.com/home/ for PCI Compliance. Phew business just gets more complicated. I'll have to devise a security strategy when I do the compliance audit. Reading their blurb they seem very concerned about criminals entering shops and hacking the wireless network to gain access to credit card terminals.

SirDice said:
ISDN lines giving a total of 2 MBit/s
Click to expand...
I had one of them, back in the day :) Life was simple then.

I'm still trying to understand the performance metrics from the free suppliers I use, I'm getting

  • 31ms from Maidenhead in the UK
  • 64ms from Seatle

Since my business is mostly in the UK, I think these are good speeds?

I also have 'top' running in a monitor on my PC desktop to watch for any spikes in usage on the server. I'm worried about DoS attacks bumping up my bandwidth and putting me on the naughty step with my ISP. Any suggestions for software I could install for monitoring the server that I could maybe push alerts to https://pushover.net/ on my android? Perhaps pfsense does this?

Thanks for now and Toodle pip.

PS pfSense website is a great responsive website design built on the http://getbootstrap.com/ framework.
 

Attachments

  • OpenReachFTTC-FibreBox.jpg
    OpenReachFTTC-FibreBox.jpg
    29 KB · Views: 749
zzatskl said:
dazza said:
I use pfsense to segment my network into LAN and DMZ and I've been very happy with it.
Click to expand...

Thanks for the heads up on pfsense, I will investigate. What hardware are you using? Did you buy one from their recommended vendors http://www.pfsense.org/hardware/index.html#vendors ? I assume the hardware just connects to the LAN output at the rear of the Openreach box, as shown in the photo below:


and BTW what ping service do you use?
Click to expand...

I've got an Alix2D3 http://www.pcengines.ch/alix2d3.htm in a standard aluminium case. It's important to get the LX800 cpu not the older ones as your bandwidth will be limited by the CPU. I had an old Alix board that ran at 200MHz (I think) and it throttled downloads by about 50%. In total I've run Alix boards first with M0n0wall and later with pfsense for over eight years without a single hardware or software fault. Very nice.

As you say, the OpenReach box just plugs in to the firewall (pfsense). I don't trust the OpenReach box at all so it's nice to firewall it off with something I trust. The OpenReach box just requires a PPoE connection from your firewall so it's pretty easy to set up. It's possible to hack the OpenReach box to get line data but I haven't done that yet. I was NOT impressed by the technicolor router they gave me. It looks very primitive to me.

I don't use a ping service, I monitor pings out of my network to popular sites so see how they change over time (google, bbc, yahoo). It's not that useful really but occasionally interesting. My average ping time to google.com (hosted in central London I presume) from my servers in north London is 18ms, it was at least 50ms from my ADSL connection (with Zen Internet as well).

I really should set up an external monitoring service but so far it's just on the todo list :)
 
dazza said:
standard aluminium case
Click to expand...
I'm trying to work out my UK buying channel, where did you buy the aluminium case (sounds cool) from, did it come with a power supply? I considered PC Engines when I was researching for a miniPC, but it looked a bit too scary for someone like me of average ability, maybe it was their simple (unappealing) website. I'll try their products out now though.
dazza said:
I was NOT impressed by the technicolor router they gave me. It looks very primitive to me.
Click to expand...
My thoughts entirely, it says "home router" on the packaging, say no more!

Thanks again for the heads up on pfsense. I was previously using pf when my server was in a rack which is a bit terse and unforgiving if you make a mistake in the pf.conf file - fortunately I concentrated hard and never had to make a visit to Telecity to get access to my terminal (my first hosting company had a bet on me having to make a visit to reset pf, they lost :) ).

Cheers

PS: If http://pcengines.ch/ have google alerts on, you should read http://www.copyblogger.com/ to improve your website sales copy :)
 
zzatskl said:
dazza said:
standard aluminium case
Click to expand...
I'm trying to work out my UK buying channel, where did you buy the aluminium case (sounds cool) from, did it come with a power supply? I considered PC Engines when I was researching for a miniPC, but it looked a bit too scary for someone like me of average ability, maybe it was their simple (unappealing) website. I'll try their products out now though.
Click to expand...

I bought it from LinITX http://linitx.com/. I seem to remember there were some sellers on ebay as well. At the time I bought the board, case and power supply as a bundle.

Believe me the aluminium case isn't that cool, it's very basic but it gets the job done. Once you install your firewall you'll never look at it again anyway :)
 
dazza said:
At the time I bought the board, case and power supply as a bundle.
Click to expand...
Thanks again, I've just ordered:

http://linitx.com/viewproduct.php?prodid=13242

with the Wistron wireless card. Price is good compared to something from a known brand like Netgear who will lack the flexibility of pfSense.

I found the enclosures (cases) are on the PC Engines website:

http://pcengines.ch/case1d1blku.htm

they must manufacture in bulk so Distributors like LinITX can rebrand them.

I think this post should be renamed "Micro Business FreeBSD Infrastructure" :h
 
I've hosted my own personal website and mail server etc. on the end of my ADSL for years. And that's with 1 meg upload. Generally it works perfectly fine and I've never had any issues. The latency to it is about 12ms and the speed of loading the pages is about the same as it would be hosted anywhere else. As it's only a personal website it's unlikely to get hit by multiple people at the same time and so this level of bandwidth works for me. It's worth doing what you can to keep things speedy though. Try not to use huge images or large files. Make sure all your content is cacheable by upstream proxies by including the appropriate headers. Use the SPDY protocol to multiplex the data down the same TCP connection and so on. And it goes without saying that you keep your O/S and webserver software etc fully up to date and protected by a firewall.

If anyone is interested I'm using a Zotac Zbox. It's small, virtually silent, and doesn't use much power. But it's powerful enough to run everything I need quite happily.
 
xtaz said:
Use the SPDY protocol to multiplex the data down the same TCP connection and so on.
Click to expand...
Thanks, some good advice there. I ran a http://httpd.apache.org/docs/2.2/programs/ab.html ab benchmarking test on Sunday afternoon from home to the office with these results:

Code: 
# ab -n 1000 -c 300 http://www.myworkfreebsdserver.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.myworkfreebsdserver.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.4.6
Server Hostname:        www.fttc.com
Server Port:            80

Document Path:          /
Document Length:        39078 bytes

Concurrency Level:      300
Time taken for tests:   45.022 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      39557200 bytes
HTML transferred:       39088264 bytes
Requests per second:    22.21 [#/sec] (mean)
Time per request:       13506.680 [ms] (mean)
Time per request:       45.022 [ms] (mean, across all concurrent requests)
Transfer rate:          858.02 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       48  589 1628.6     88    9292
Processing:   264 9253 6455.4   7446   40620
Waiting:       74  942 1497.6    381   14551
Total:        353 9843 7340.1   7738   45015

Percentage of the requests served within a certain time (ms)
  50%   7738
  66%   9801
  75%  11960
  80%  13387
  90%  18584
  95%  26170
  98%  34266
  99%  37055
 100%  45015 (longest request)

which seem quite impressive.

BTW, the upload speed on my BT home fibre line is a whopping 16MB. But I don't have a static IP, anybody know how I can translate the variable IP address using a *nix DynamicDNS software combo? I currently use http://www.noip.com/ from windows, but don't keep this machine on all the time and I'm thinking of putting another Zotac box at home as a backup server.
 
22 requests/seconds isn't actually all that impressive. Although it would be perfectly fine if that's all you need obviously. Another thing I did is that I spent a lot of time learning the tricks for optimising things like that. I eventually got my personal website up to around 450 requests/seconds using the same tool as you used. I switched from www/apache22 to www/nginx-devel and switched from using the mod_php apache module to using the php-fpm fastcgi server. This allowed me to serve static content at approx 15,000 rps and PHP content at around 100 rps. I also switched from databases/mysql55-server to databases/postgresql93-server which increased it to approx 200 rps. And finally I installed www/php55-opcache which caches the compiled PHP bytecode in memory which increased my rps to approx 450. It obviously isn't just as simple as installing these things. I also had to go through the settings learning how to tune the configuration to get the best out of it as well but these things are worth investigating if you want to get the best out of it.

For what it's worth though, since www/apache24 came along that has improved the situation although I still stick with nginx. With this version of apache you can use the event based MPM and the php-fpm engine and it performs equally with my own setup. The important thing really is to use a threaded webserver that doesn't use forking, and to offload PHP so that it's not invoked for static content like html, css, js, or images which on an older forking apache running mod_php is a huge resource hog.
 
xtaz said:
increased my rps to approx 450
Click to expand...

Thanks for some seriously good tips there to increase the speed of my fibre hosted websites. Like you say, a lot of time to invest. Very much appreciated.
 
I have BT 76M/16M Infinity at home with a static although I don't really have any intention of running any services from it. I'm a sysadmin for an ISP so have an entire data centre to play with.

The main issue with hosting at home is downtime. When my Infinity was installed it went down for >8 hours within a week. I don't know how long exactly, it stopped working about 8PM and was OK the next morning. BT really don't give any serious SLA's for their broadband products. They're not going to get into a van and come fix it within 4 hours like they will for a leased line. Of course you could be lucky and have a service that doesn't go down for years.

Power loss is slightly less common (at least for me) but again if your power goes off for longer than your UPS can hold (or you don't have one) you're in the same 'wait for the provider to get round to fixing it' boat.

A proper host will have redundant Internet links and UPS/generator backup. The really big ones will have multiple power feeds from the utility provider.
Generally speaking the price for a standard web hosting or an average VM is probably less than it would cost for you to power your own server these days. Co-locating a higher spec server can get a bit more expensive, but then if you need something that serious to run your website, you'll probably be willing to pay the price to have it somewhere with guaranteed uptime.

If you're running something small and can deal with possible outages, there isn't really any other major catch to running services from something like FTTC (vDSL). Some providers may not like adding reverse DNS which can cause problems if you want to run a proper mail server.

Regarding it being the future for small business hosting, unless you have a real interest in having the server (and/or data) local, I don't see any benefit over running the site on a traditional host or VM (or you're the sort of person I expect some here are - those that like to be in control and 'tinker' about with it). Anyone serious about their website isn't going to risk it going down for extended periods because of BT issues they have no control over or no real support on. Apart from the obvious that most businesses have no interest in running/maintaining their own servers.

Maybe I'm just biased due to working for an ISP but I don't really see the benefit unless you really do want the server on your local network and are the sort of person who's happy to look after it personally.

Regarding routers, if you don't have something to run pfSense on, the Mikrotik routers are pretty good. Extremely low power and incredibly versatile. I have an RB2011 which is fairly cheap and has Wireless + 10 ethernet ports. The great thing is that it's literally a 10 port router. You can configure it similar to a standard router - with a wan and 9 lan ports, or you can split them up however you want. I have my lan on ports 1-5 (+ wireless), BT wan on 6, and a completely separate lan with its own gateway address and DHCP pool on port 7 which connects to a FON wireless AP. I still have 3 ports left over I could use for other networks (or bridge onto my lan)
 
My pfSense router has arrived. I ordered these parts:

http://linitx.com/viewproduct.php?prodid=13242
http://linitx.com/viewproduct.php?prodid=10989
http://linitx.com/viewproduct.php?prodid=10987
http://linitx.com/viewproduct.php?prodid=10887

The middle two items are for a wireless (wifi) connection to the router. The guys at LinITX kindly inserted the wireless card and connected the internal wireless antenna (which with my eyesight might have been fiddly) to the http://pcengines.ch/index.htm alix board. It arrived in bubble wrap with no fancy packaging or branding, all quite anonymous. You could easily put your company branding on it. I like the sturdy aluminium case, see photos below. Now to configure pfSense :stud
 

Attachments

  • P1040934.JPG
    P1040934.JPG
    142 KB · Views: 642
  • P1040936.JPG
    P1040936.JPG
    157 KB · Views: 650
  • P1040937.JPG
    P1040937.JPG
    164.5 KB · Views: 651
You must log in or register to reply here.
Back
Top