--0016364579903a8cbe0463c85f37
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Wed, Feb 25, 2009 at 12:58 PM, M. Edward (Ed) Borasky
<zznmeb / gmail.com>wrote:

> On Wed, Feb 25, 2009 at 11:53 AM, Dav Yaginuma <dav.coleman / gmail.com>
> wrote:
> > I've done some research on this and I'm fairly convinced it doesn't
> exist,
> > but I thought I would just ask here in case I missed something.
> >
> > I want to be able to full-stack load test a Ruby on Rails site (not that
> > rails matters in this context) running dynamic test plans across multiple
> > concurrent users in order to better replicate real world usage.
> >
> > In other words, I'd like to run say 50 concurrent user sessions where
> some
> > of them are signing up, some are just browsing around, some are logging
> in
> > and doing UserActivity1, some are doing UserActivity2, etc.
> >
> > I want to script the test plans in ruby, similar to a seleinum-rc test
> plan
> > where I can use the same DSL to simulate user input and identify parts of
> > the response html, and use that to drive further test activity. I'd like
> to
> > use some of the same DSL from my selenium or integration test helper
> modules
> > already developed in these tests as well in order to be DRY.
> >
> > I would like the test framework to mimc the browser behaviour: handling
> > cookies and even simulating caching to send the appropriate
> > If-Modified-Since headers.
> >
> > Since I would like to replicate actual request load on the server, I want
> to
> > make sure that all page assets, even static ones like
> images/css/js/flash,
> > are requested during the test, so it would be nice if an HTTP Proxy could
> be
> > used to create the initial test plans by recording actual browser
> sessions.
> >
> > Some things I've looked at so far:
> >
> > JMeter:
> > - seems difficult to make dynamic test plans here that will modify
> requests
> > based on earlier responses
> > - test plans are in XML, although I suppose I could use a ruby DSL to
> > generate the XML, but since the proxy records to XML I'd also have to
> > generate ruby from XML
> >
> > The Grinder:
> > - Looks awesome, and I may end up going with this, but it would be nicer
> to
> > use ruby instead of Jython in order to leverage my teams existing code
> and
> > skills.
> >
> > shell/curl/ruby script mish mash:
> > - A lot of control, and curl handles cookies, but no real framework
> support
> > and no proxy recording
> >
> > Ruby code:
> > - Doesn't really seem to be any test framework that deals with concurrent
> > requests like this
> > - I assume JRuby would be required (and would be sufficient) to get real
> > threading
> > - I feel like there are probably a lot of pieces that could be tied
> together
> > to make what I want, but unsure of what they are or which ones to
> > concentrate on. Hpricot? Mechanize?
> >
> > Anyone have any ideas? Is anyone doing load testing via ruby?
> >
> > --
> > Dav Yaginuma
> > http://AkuAku.org/
> >
>
> I've made similar searches and come up with the same emptiness. There
> are two major *commercial* load testing tools, LoadRunner and
> SilkPerformer. I have about seven years of experience with
> SilkPerformer and almost none with LoadRunner. There are a few open
> source recorders, but none I have found that are anywhere near as good
> as SilkPerformer.
>
> What you *could* do is get a low-end SilkPerformer or LoadRunner
> license ... just enough virtual users so you'd get a recording
> capability ... and then write a Ruby run-time and parser to translate
> the "native" language to Ruby. But check the EULA to see if this is
> legal. I never tried it because I consider it a waste of effort.
>
> Really, you need to make a business decision here. What is the cost to
> your business of the time you spend building a load testing framework
> when you could be developing and testing your product? I'd recommend
> you seriously consider buying a license for LoadRunner or
> SilkPerformer if performance of your application is mission-critical.
>
> --
> M. Edward (Ed) Borasky
> http://www.linkedin.com/in/edborasky
>
> I've never met a happy clam. In fact, most of them were pretty steamed.
>
>


Thanks Ed,

Good to know I wasn't missing something obvious.

I've contacted sales reps for SilkPerformer and LoadRunner. My fear (and
unfortunate experience) with this kind of software is that you spend a lot
to get a complicated but powerful system that then requires more time (and
money) to make actually work. They both seem to have a lot of features that
I don't need, so I was hoping to find something more streamlined and
ruby-oriented first. It's good to be aware of all the alternatives though,
so thanks for the pointer. I had heard of LoadRunner but not SilkPerformer.


-- 
Dav Yaginuma
http://AkuAku.org/

--0016364579903a8cbe0463c85f37--