Hi everybody!

This is my first posting on this forum.

I’m going to run a site (95% PHP + 5% Perl) that will request in and send out XML feed. Hopefully a lot!

For example, I’m expecting to get 500,000 requests per day. I request XML feed from 5-10 different sources, log and tweak it (lots of mySQL is involved) and send resultant to a user. Each feed (incoming and outgoing) is around 2Kb.

Now I’m hosting it on a large resellers shared plan: 150Gb per month with rollover! Server has 4x2.4 Xeon, 512 cash each, 2Gb RAM. Average CPU load 3%, memory used 30%, swap 1%. Pretty stable server, dissent support, VERY cheap, etc.

Questions:
1. Do you think I NEED dedicated?
2. Will it REALLY increase speed?
3. What hardware would you recommend?

Thanks a lot.

First off, how many mysql queries do you run per request?

How many users (or requests) do you currently have?

ANd thirdy, how much do you pay for your current plan?

FarCry, thanks a lot for a prompt response!

1. Currently, I have about 10,000 requests per day. But I "didn't go public" yet.
2. About 300 SELECT, UPDATE, INSERT SQL requests per XML request.
3. Now paying $30 per month.

Thanks again!

if you are really expecting 500,000 requests a day and want to ensure a reliable service I would def susggest dedicated.

gp

and each user makes a xml request?

or how many times does this 300 query request happen?

Actually, I expect 500,000 and hope for over a 1M. It will probably happen in a year from now, but I wouldn't like to move the site then.

Toooo many people will experience the down time.

FarCry,

Yes, all of them. I don't even count "simple" pages requests. It's nothing comparing to that.

gordonrp,

Which one?

ouch, thats alot of mysql queries... you should really think about some form of caching system.

500 000 requests per day is 5 requests per second, thats 1500 mysql queries.

I REALLY hope you know how to optimise mysql if your dont have some sort of caching or static update file that you could do. Maybe you could create a crontab that update a static file every 1-5 minutes. That way the file is not running mysql queries every time its requested which would really not do wonders for your server.

You would need a SuperXeon or Total Control for this sort of usage.


If you dont mind my asking, what sort of data is this? (Accounting, exchange rates, game server info, top pron sites, amazon books, ?)

I just purchased a "super resellerz" Dual xeon 2.4 for my soon to expect 1,000,000 request a day. $300/mo + management.

I may of purchased more power than I need, im not too sure, but I want room for expansion.

gp

oh, and cobweb, the more ram the better, you should look at at least 1GB extra. If your going to be using all those sql queries still. Another way you could drop your cpu usage is write it in C++ (or C), that will certainly improve performance over php/perl. If its a HTTP request you could also think about windows and .net as a future option, it has very efficient caching built into it, and its not very heavy on the CPU/RAM either.

I dont know what form of site you are expecting to dish up Gordon, but even a php forum wont use as much cpu as a 300 request/request script - thats pretty insane.

Like I said, maybe I purchased more than I need You only live once. Is that you in your avatar? A good looking lady in the computer scene, rare.

gp

Yes, I do have some programming experiene - 20 years. Yes, I will optimize it even more - that's why it's still in "pre-release" phase.

This is PPC SE feed.

So, you are saying that something like Super Celeron 2.4 wont make it? Not even close?

At what traffic rate do I REALLY NEED to go dedicated? I'd like to invest minimum upfront, because it's not profitable yet.

heh, im an 18 year old male.. its a render of a female from FFXI that i use as a wall paper, i never see my desktop but still. I dont play final fantasy, i just like their renders. But this is way off topic!

I need more ram in my server, less CPU at the moment, but im actualy finaly getting round to making the site which was the main reason that i originaly got this additional server (before i canned by Ezzi.net one for bad performance).

It will be interesting to see what sort of data cob web is serving up.

I knew it was too good to be true!

Yeah cobweb what sort of site are you looking to dish up?

gp

Sorry, i didnt see this post before i made another, it came in right when i was posting.

http://www.vise.ws/ppcse.htm is interesting explanation of PPCSE for those who dont do much with advertising

at 1500 queries a second you can forget the celeron, it will be a smouldering heap within minutes.

I'll just stress test a 2.4ghz celeron i have here and tell you the result (i have a 50,000 query database i can run)

What you could do to cut your costs is wait around on your existing provider, or even move to a smaller bandwidth provider (as by my calcs your not going to use 1gb/day with your 500,000k requests @ 2kbytes each) until you are ready to go live. Once you go live you can then get a new box to run everything off.

What you could even do is keep it up on a small provider until you get upto 50-100k requests a day, then order the new box, keep running the service simultaneously (sp?) on both accounts and switch you dns from your small time account. keep the small account for DNS that way you have minimal cost for the dns host ($5-20/mo) and you also have a dev environment. Could be feasable. If you keep your TTL's down VERY low you could even do a server move and migrate your dns over without any down time. I've only been doing this stuff 4 years (learnt perl/asp at 14 [got books as a birthday present i requested]), but i do know a thing or 2

Guys, we were typing at the same time.

This is PPC SE feed.

So, you are saying that something like Super Celeron 2.4 wont make it? Not even close?

At what traffic rate do I REALLY NEED to go dedicated? I'd like to invest minimum upfront, because it's not profitable yet.

One 23.7mb / 255,901 query sql script running

FarCry,

I started the same way at exactly the same age from 8-bit 300 KHz Intel 8080 procesoor. Computer had 32 Kb RAM, NO hard drive, etc. Internet didn't exist at that time...

Nice link! One of your sites?

I think I'll ask you to halp me with move I have lots of experience with Windows servers and MS SQL - still do it for paycheck But those guys ARE expensive...

i never mentioned MSSQL

You can mysq ByteFX.MySQLNet as a mysql library for connecting to your database, its a ADO.net compatable library, and its opensource. MySQL have hired the guy who wrote it to write their official library

Nope, its my google found site. PPCSE doesnt mean much to many people, sounds a bit like a drug more than an advertising strategy/system.

Windows on a total control or some of the SM servers is free, no outta pocket expense... .net is free too, iis is free, etc. Microsoft have started to listen to what people want/need.

My poor server is getting there, 95k queries so far. on a C2.4 you migh handle 1 request per second by the looks so far, but thats REALLY pushing it. Its got 383mb of ram (not 384, 383), 170mb used and rising, 144mb of page file used, cpu usage is sitting fairly high. Running the query over my 2.2gigabit network. My system is using 40% CPU sending the queries Upto 150k queries now (went back and added this after writing the post and checking it). 145 queries/second and climbing


I cahrge $35/hour for admin, not in my sig anymore because i've been too busy coding stuff for people, but i still will do it. There are also external companies that will do instant dns changes (they cost more per month though).

maxed at 252.103 queries/second. the box was totaly dedicated to this.

There is no way your going to be able to do this on a 2.4ghz P4. I can benchmark on my 2.8Ghz work pc here at home which the cpu would be the sole limiting factor. (very fast RAID-0 SATA array, DDR4000 ram in dual chan, 800mhz bus, etc)

Thanks a lot!

Does it mean, I better stay on Quad Xeon shared for now? I don't need much bandwidth, obviously...

Most likely your on a dual xeon with hyperthreading, not a try quad

You could always benchmark their server yourself, a simple php script which inserts 50,000 values into a database will test it pretty well. Dont know how pleased your provider will be, but who cares right?

I would suggest finding someone on these forums who will give you a good deal on hosting in the chosen OS environment and cpu specs you would likely purchase. You WILL need your own server, your webhost will kick you off for that much usage without a doubt. The reason i say someone within this network is that you will get a fair idea of the uptime/network speeds and quality of hardware. BTW: I'm not offering, and you would not want to be only any of my servers (1x at sm, and a few more at home) as they get fairly abused with software testing . My server is currently testing for memory leaks, hence they high/fluctuating ram usage hehe.

Out of interest, how much are you paying for your currrent hosting?

$30 - 150Gb with rollover. I should have a LOT rolled over from previous months. WHM shows 4 processors.

my WHM also shows 4 processors... its bescause its a dual xeon 2.4 with hyperthreading. acts like 4 processors.

gp

Thanks a lot for your help!

Any other opinions? Comments? Suggestions?

I'll be checking the topic.

Thanks again, FarCry!

no problems gordonrp helped too , check your pm's

252 queries/sec seems very slow to me. What exactly is your script doing?

CobWeb: What will your queries be like? All unique? What's your read to write ratio? I didn't bother to read that link FarCry posted. As many of you know, I'm lazy..

UD.

the script is just a dump of my database structure for an IP resolution system i built (finds a country from ip). They are all writes.

System has 1x 40gb IDE 7200RPM WD HDD w/8mb cache. 384mb of ram, C2.4ghz cpu, 2x gigabit intel cards hooked upto my gigabit switch (my computer has 2x as well)