On Mar 7, 2007, at 09:40, Greg Loriman wrote:
> I posted this in railsweenie, but it's not really a rails question,  
> and the
> post isn;t getting any answers.
>
> I'm not an experienced web developer. I'm still rather groping  
> around. I
> come from a client server background. I've plumped for Rails,  
> although this
> post's focus isn't really Rails.
>
> I'm putting together a relatively simple site which I want to  
> design from
> the ground up for horizontal scalability, partly for the challenge,  
> partly
> because I need to learn and get experience. To help me do this I am  
> going to
> run at least two virtual machines to enforce the correct environment.

The set of problems you'll run into by trying to plan for scaling by  
adding artificial constraints will not overlap the problems you'll  
really encounter if you want to scale.

Scaling is done by making the stuff that is objectively slow fast,  
not by planning for what you think might possibly end up being slow  
and guessing how you can make those things fast.

> Currently my idea is to federate the data so that the users are  
> divided
> between the 2 or more machines, perhaps splitting alphabetically by  
> user
> name (ie. A-G to machine 1, etc). Where there is interaction  
> between account
> holders I am thinking of Drb'ing. Obviously rails is not going to  
> be able to
> do the interaction side of things, but I am fine with that; I'm  
> prepared for
> a bit of manual labour.

Use a database.  You can run at least a tens of thousands of users on  
good hardware without breaking a sweat.  When that DB starts  
sweating, get better hardware.  If you're lucky enough to be really  
successful then you might need two machines.  In that case you can  
hire somebody who knows what they're doing.

> I would love comments/advice on my above ideas, and further  
> insights into
> horizontal scaling.
>
> But also....
>
> To facilitate the above I need some kind of proxy in front of the two
> machines directing incoming requests to the correct machine based  
> on the
> login name which will be part of the url. Here I come unstuck since  
> I have
> no idea how to do this.
>
> There must be proxies of this kind, but I'll be blowed if I know  
> what an
> appropriate one would be, or where to start in making it do what I  
> want.

Don't bother.  You don't need them until your hardware proves  
unreliable, or one machine can't handle the load.  Investigate then.

> Can anyone give me a few pointers? Is squid the thing? Mongrel (I  
> don't
> really know what mongrel is)? Can apache be made to do this, and if  
> so is it
> a bad idea? Obviously it needs to be pluggable since I'll be using  
> my own
> code (C or Pascal) to do the lookups for the redirection.

Stop trying to plan for the unknowable and write the software to be  
simple.  When you find something slow, optimize that.  Then repeat.