Thanks for the reply Wilson. Pardon my mangled reply as well.

-I believe our plan is to just use MySQL and Postgres side by side for
awhile. We're going to be forced to double write things but that should
not be too much extra work. We have some complex queries, for example
we need to return 1-7 depths of a web network in a query (think
friendster but a little more sophisticated. We've done the same thing
in other projects, but the problem becomes tuning it to the database we
select. I'm working on some scripts to start filling our tables with
millions of rows so we can test this thing under a big data load from
the start.

-We don't want to use Rails though we may take ideas/parts where
permission allows us to do so. I've looked a lot at Rails and would use
it for a smaller app, but I don't think it's well suited to what we
want to do. We're experienced developers and prefer our own mess, plus
we would like to show that there are alternative to Rails for using
Ruby via the web. Others are doing the same. We also don't like being
tied into a framework -- work with Spring, .NET, Struts, etc. has
taught us that frameworks are great, but often too generic or generate
too much work for specific tasks. Rails is no doubt a great framework,
but not for us and I doubt it would scale to meet our needs rendering
very complex templates and bringing back lots of DB data that an O/R
mapper might choke trying to do.

-We don't want to use an O/R mapper because of the complexity of a lot
of our queries. I am a fan of SQL, and I've been certified in other DBs
before so I know my way around the language. I like the idea of stored
procedures (segregrating code in one place, reusing accross systems,
assigning proper rights per procedure, caching, etc). Obviously some of
this is dependent on our choice of DB, but regardless I prefer not to
use stored procs. I've dealt a lot with O/R mappers in the past and
performance tuning in Oracle and SQL Server 2000 and I can tell you
that moving from stored procedures form an O/R mapper can make a world
of difference. You spend a lot more time and maintenance, but you gain
some more flexibility. Some people like the best of both worlds, but I
prefer to keep set based operations to SQL in the DB server. I don't
want an O/R mapper touching my precious schema :)


-Our reason for not using ERb is a difficult one to understand without
being involved in the project I admit. We need a system to allow users
to upload their own themes. We also have a complex widget system (think
portals) that may one day be nice to support on a thick client.
Standard templating would be a lot of work per widget. I would rather
design a templating system that works very fast and easy for what we
need but requires more time up front. ERb reminds me of ASP days, no
thanks. I've read David H.'s reasons for using ERb and I think they are
valid, but not valid for us. I am not a fan of code in the template.
That said, my developers and I are disciplined enough not to do
anything bad in a template. ERb would be my third choice behind
quicksilver.

-Thanks for the tip on Zimbra and the Zen profiler. Regarding the
threading code and some of my fastcgi/ajax concerns, I think I could
take a tip from Jetty and use Ruby continuations in conjunction with
the response/request. Our group is very experienced with threading so I
am not intimidated, but in the past I have dealt with the nightmares of
deadlocks in particular as well as race conditions, memory bullying,
etc. It seems to me like Ruby doesn't have a lot of the nice things in
Java and .NET to deal with some of the classic threading issues.

Thanks again for pointing me towards some projects I did not know
about. Any other thoughts you have would be great because your feedback
at the very least gets me questioning myself and new ideas flowing.
FYI, like I think I mentioned we've used Ruby, just not in the context
of a million user system.