--001636e0b9c9b665d50486e97ce8
Content-Type: text/plain; charset=ISO-8859-1

On 19 May 2010 11:56, Daniel DeLorme <dan-ml / dan42.com> wrote:

> Daniel N wrote:
>
>> On 18 May 2010 16:56, Daniel DeLorme <dan-ml / dan42.com> wrote:
>>
>>  Daniel N wrote:
>>>
>>>  On 13 May 2010 17:37, Daniel DeLorme <dan-ml / dan42.com> wrote:
>>>>
>>>>  Does anyone know how to do the following, but without threads, purely
>>>>
>>>>> with
>>>>> asynchronous IO?
>>>>>
>>>>> website  hread.new{ Net::HTTP.get(URI.parse(url)) }
>>>>> template  ompute_lots_of_stuff()
>>>>> puts template.sub("<content goes here>", website.value)
>>>>>
>>>>>  Event machine is perfect for this kind of stuff.  Weather it fits with
>>>> the
>>>> rest of your web framework is more likely the thing that makes it an
>>>> unlikely selection (if you're using anything rack based for example)
>>>>
>>>>  If you could show me how to use EventMachine in this case I'd be
>>> grateful.
>>> I couldn't figure out how to run compute_lots_of_stuff() while the http
>>> requests are executing.
>>>
>>>
>> What context are you trying to do this in?  Is it inside a rack request
>> (rails / merb / sinatra /pancake / other)?  or is this in a stand alone
>> script?
>>
>> Could you perhaps provide a bit of context for what you're trying to
>> achive?
>>
>
> Ok, here's the context. I didn't put this in my OP because I figured
> it would just bore everybody to tears.
>
> This is inside a rack request. The idea is that I'm assembling a web
> page by doing a bunch of sub-requests for the various parts of the
> page. So I'll have something like:
>
>  action "index" do
>    @news  ubreq("http://news.server")
>    @ad  ubreq("http://ad.server")
>    @blog  ubreq("http://blog.server")
>    @forum  ubreq("http://forum.server")
>  end
>
> All these sub-requests are launched asynchronously and, while they are
> executing, the app generates the layout within which the output of the
> subrequests will be embedded. So I'll have something like:
>
>  response  '<html><body>',
>    '<div>',@ad,'</div>',
>    '<div>',@news,'</div>',
>    '<div>',@blog,'</div>',
>    '<div>',@forum,'</div>',
>    '</body></html>']
>
> And when rack finally outputs the response to the client it will block
> on the various subrequests unless/until they have completed.
>
> What I can't figure out with EventMachine is how to have the "main
> thread" generate the layout while the subrequests are executing.
>
>
Ok now we're talking.  So with rack you can't do true async with a callback.
 Rack is callstack based, meaning that you have to return the value as the
response the the .call method on your application.  This means that any
callback based async actually needs to block in order for the rack
application you're in to return it's result.  You _could_ do it by returning
a custom object in the rack response that renders as much as possible while
it waits for the response, and then renders that when it can, but that
option may not be available depending on what framework you're using.

There are a couple of other things that could help you here that immediately
come to mind.

You can use http://github.com/pauldix/typhoeus which can fetch all the
resources in parallell, and then block until all the responses come in.
 This is probably going to be relatively easy to implement, and means that
the total request time for the resources is only very slightly higher than
the longest single response.

You can use ESI.  There's an esi for rack project on github by Joshua Hull
which could be useful to you.  http://github.com/joshbuddy/esi-for-rack You
can also use esi outside of the rack request in apache or nginx, by
responding first with a layout file containing esi tags pointing to the
content to use.  Ngins, Apache or the esi rack project can then assemble the
page for you using the resources specified.

Alternatively if you're not married to hard to rack, you can take a look at
something like cramp, or node.js for a true async server environment.

HTH
Daniel

--001636e0b9c9b665d50486e97ce8--