Frederick, M. Edward, Ezra, J-Van, Eric First of all, thanks for ALL of the fantastic replies. I must say, the Ruby community is simply amazing. Vrtwo, Thought I'd address your question first since it might give some context to the rest of the replies. We anticipate the system should, at a minimum, support 1 million users with the ability to scale larger as required. We built a similar application about 6 years ago and it had around 250,000 users IIRC before we shut it down (unfortunately, the model was not sustainable back in 2000 though it is now). Given the growth in the Internet, I anticipate hitting 1 million users would not be difficult today. For a bit more background, my company currently has multiple website builders targeting small businesses. We also have a "website builder builder" or perhaps we could call it a "meta website builder." These automatically talk to DNS servers, email servers, other services and we have partner integrations (as well as marketing partnerships) with many external companies like Google, eBay, PayPal, Yahoo!, etc. In other words, the application is pretty complex. However, this business to business builder is built on standard web technology. We regularly review all of our competitors (in depth) and we believe it is currently the best such website builder overall (though it needs and we are currently working on a serious refresh of the home pages and a refresher in design tools which are written but not yet launched); however, it is not Web 2.0 as it was started in a different era. I believe we actually had the first online website builder on the Internet. Our new project will target personal websites as a free service to go into a MySpace like market but aim at a market a little older than that. I will say that many of the new ideas that we've coded have never been implemented on the web yet (that I've seen or heard) and are not trivial things to implement. We had to write custom graphic manipulation libraries to get the performance we wanted after having tested about a dozen available libraries on the Internet and being unsatisfied with them (it will blow Image Magick away in performance and power for example). Currently, I have written all the code for the new project and wrote all the base code for the business-to-business project though it has been taken over for a couple years now with a team of coders. Hi Ezra, Let me say wow. I find there are few people who really understand scaling issues well and allow me to say that it is fantastic information you've provided. This is exactly the type of info I was looking for and, quite frankly, I was worried there was still too little performance work being done out there. It seems there is some serious work being done however. Actually, the file upload issue is something that ColdFusion also has a problem with. In ColdFusion, the optimal number of maximum threads is a multiple of the number of CPUs but that doesn't work when 10 threads can be consumed by 10 concurrent uploads. Our solution would have been to have a dedicated upload server with a higher maximum thread count just to handle the uploads. In practice, the upload limit hasn't been problematic (yet). I love the idea of taking the upload handling and allowing that to scale better. It is, frankly, a brilliant idea. I also really like the fact that you are thread-conscious. I will be looking at Merb tonight. As an aside ----- If we decide to go the Ruby road (which I'm feeling a little more confident with now), I would consider contributing to merb and your project and am flattered that you asked. That said, allow me to be completely frank and say that I don't think I would be useful to the project immediately. I would say (again frankly) that I am a very good developer and that I think I could program in a way that is natural to the Ruby style, but still, I have a lack of experience in Ruby. I have, however, adopted many Ruby style programming constructs into ColdFusion because I like them so much (things like array iterators) and have created a pretty nice framework in ColdFusion. I was originally wooed into Ruby through Prototype which I am very good at coding with however. ----- I have to say that I'm not quite sure I understand why multiple Xen instances would provide better performance than a single environment running multiple Ruby instances. Not saying it isn't true. Just that I don't understand why this would be. Thank you for the info on ActionPack and MERB. I'm going to look into this more. As suggested, we do use a shared nothing approach in our applications using the database servers for persistence. Since our db has enough RAM, it basically acts like DRB. As you mention, we do a lot of caching but because of the dynamic nature of our application, we do most of it in the db. We basically do a lot of front-end processing to get data from a bunch of different places in the db, process it, then encode it into a single TEXT field, and drop it into a table we designate for caching. When we need it, we retrieve it then decode it for use. We currently encode/decode using WDDX but I now have a simple custom library do it about 100x faster. If we edit any of the underlying data, the edit action also deletes the related cache record. Upon next retrieval, if the data isn't found in the cache it is automatically recreated with the new data from the edits. Makes for a really simple interface to refresh data and also makes it simple since the DB is authoritative. It scales quite well. Thank you Eric, Edward and Joe (J-Van) and Frederick. The info on persistence and threading was useful. I do find it both cool and troubling that essentially the same application scope is used for every request though I understand there seems to be the ability to execute in Anonymous Modules. Seems a little dangerous but also seems like you can get some really cool performance tweeks by front loading processing at Class creation time. M. Edward, I would be excited to see JRuby becoming a good implementation of Ruby. With their team being adopted by Sun, this seems like a strong possibility. Plus a lot of the integration libraries are written in Java. Didn't mean to preach to the choir. Just thought the last comment suggested that I shouldn't worry so much about performance and just wanted to explain why our situation may not be the typical Ruby application and why I was concerned with performance so much. Thanks for referring me to Zed Shaw. I'm not sure if I'm ready for him just yet but he seems to be a good guy to talk to. J-Van Thanks for the ideas. By scaling with "processes" I'm assuming your meaning to have multiple complete application scopes instead of one application scope running concurrent threads? Just wanting to make sure I understood that correctly. Eric, Thanks for the insight into ERB vs eRuby and caching. Sunny Hirai CEO, MeZine Inc. -- Posted via http://www.ruby-forum.com/.