--Apple-Mail=_F23E1788-C11D-4601-B3EE-380E7991077C
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_51277E3B-2A33-42F6-8E71-C13CB69CC07A"


--Apple-Mail=_51277E3B-2A33-42F6-8E71-C13CB69CC07A
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252

Hello,

On Sep 12, 2014, at 22:24, Eric Wong <normalperson / yhbt.net> wrote:

> Panagiotis Atmatzidis <atma / convalesco.org> wrote:
>> On Sep 12, 2014, at 10:31, Eric Wong <normalperson / yhbt.net> wrote:
>>> Panagiotis Atmatzidis <atma / convalesco.org> wrote:
>>>> Generally speaking there is some latency in the server but doesn't
>>>> justify almost 60+ seconds of difference :-/
>>>=20
>>> I'd take a look at truss/ktruss/dtruss or whatever system call =
tracer
>>> which gives timings on FreeBSD and see what syscalls take long.
>>> Because based on the system+user times, it is waiting *something*
>>> external.
>>=20
>> Run a =91dtruss=92. I didn=92t look at it carefully yet. I=92m not =
familiar with this kind of things, it will probably take me some time to =
figure out what=92s happening. Please take a look and let me know if =
there=92s anything that jumps out directly. What I found odd are the =
path errors are every call. I don=92t see those on my MacOSX system. =
NOTE: that=92s a 'dtruss -a=92 output.
>>=20
>> https://gist.github.com/atmosx/93e7f202539d8d8a0d47
>=20
> As Ryan pointed out, select is taking a long time so that means it's
> waiting on I/O of some sort.  Unfortunately your dtruss output isn't
> outputting more detail about which FDs select is waiting on.
>=20
> I would look into more verbose tracing options of dtruss
> (probably equivalent to "strace -v -T" for Linux).

Thanks for the explanation.  I tried several different outputs by =
reading the man page. I think this might be a little more helpful:=20

$ sudo dtruss -ascf ./test.rb
[=85 huge file =85]
100364   95102/100683:   1863430   55365     10 select(0xE, 0x805D7D880, =
0x0)    =3D 1 0
100365=20
100366               0x80188606a
100367               libc.so.7`0x8009abe2d
100368               libc.so.7`0x8009ac2a2
100369               libc.so.7`getdiskbyname+0x3ad
100370               libc.so.7`0x8009ab486
100371               libc.so.7`0x8008a962f
100372               0x80402c1f2
100373               libc.so.7`0x8009a2250
100374               libc.so.7`0x8009a1e42
100375               libc.so.7`0x80098de58
100376               libc.so.7`0x80099a824
100377               libc.so.7`0x80099b125
100378               libc.so.7`fmtmsg+0x201
100379               libc.so.7`fmtmsg+0x12e
100380               env`malloc+0x9df
100381               env`malloc+0x8af
100382               ld-elf.so.1`0x80061d000
100383=20
[=85 huge file =85]

You can see the entire output here: =
https://gist.githubusercontent.com/atmosx/e32f494b17f5a9f4b1f4/raw/dtruss =
(but it=92s kinda huge, I used vim to find =91select=92 system calls)
I=92m not sure what the problem however, it=92s can be reproduced only =
on my VPS[1]. Other people with FreeBSD=92s are generally slower than =
Linux but by 1-2s not.

I don=92t this is something related to ruby. I think there something =
wrong with the VPS. The virtualisation engine maybe, who knows.

>=20
> One more thing: have you checked if the api.skroutz.gr endpoint is
> throttling your server IP for giving it too much traffic?

No I=92m 100% that=92s not an issue. I=92m respecting their limits + my =
calls are to slow to reach their limit anyway.

>=20
>>> I would also take a look at the Faraday gem + dependencies which it
>>> seems to use for HTTP access.  It could be misconfigured TCP knobs
>>> (TCP_NODELAY may be needed, or forgot to uncork the socket on
>>> TCP_NOPUSH).
>>>=20
>>> I suppose you can also try doing the equivalent with the excellent
>>> Net::HTTP::Persistent gem  .
>=20
>> I tried using =91open-uri=92 to do download a file of 1024kb side.
>> Net:Http made about 12s while OpenURI about 22s not that system.
>=20
> s/not that system/on that system/ ?

Yes :-) See [1] for details! The first system is the VPS (14s) while on =
other systems takes about 5-6s realtime.


[1] https://gist.github.com/atmosx/14efea27eb2c1e38af09

Panagiotis (atmosx) Atmatzidis

email:	atma / convalesco.org
URL:	http://www.convalesco.org
GnuPG ID: 0x1A7BFEC5
gpg --keyserver pgp.mit.edu --recv-keys 1A7BFEC5

"As you set out for Ithaca, hope the voyage is a long one, full of =
adventure, full of discovery [...]" - C. P. Cavafy





--Apple-Mail=_51277E3B-2A33-42F6-8E71-C13CB69CC07A
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;">Hello,<div><br><div><div>On Sep 12, 2014, at 22:24, =
Eric Wong &lt;<a =
href=3D"mailto:normalperson / yhbt.net">normalperson / yhbt.net</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite">Panagiotis Atmatzidis &lt;<a =
href=3D"mailto:atma / convalesco.org">atma / convalesco.org</a>&gt; =
wrote:<br><blockquote type=3D"cite">On Sep 12, 2014, at 10:31, Eric Wong =
&lt;<a href=3D"mailto:normalperson / yhbt.net">normalperson / yhbt.net</a>&gt;=
 wrote:<br><blockquote type=3D"cite">Panagiotis Atmatzidis &lt;<a =
href=3D"mailto:atma / convalesco.org">atma / convalesco.org</a>&gt; =
wrote:<br><blockquote type=3D"cite">Generally speaking there is some =
latency in the server but doesn't<br>justify almost 60+ seconds of =
difference :-/<br></blockquote><br>I'd take a look at =
truss/ktruss/dtruss or whatever system call tracer<br>which gives =
timings on FreeBSD and see what syscalls take long.<br>Because based on =
the system+user times, it is waiting =
*something*<br>external.<br></blockquote><br>Run a =91dtruss=92. I =
didn=92t look at it carefully yet. I=92m not familiar with this kind of =
things, it will probably take me some time to figure out what=92s =
happening. Please take a look and let me know if there=92s anything that =
jumps out directly. What I found odd are the path errors are every call. =
I don=92t see those on my MacOSX system. NOTE: that=92s a 'dtruss -a=92 =
output.<br><br><a =
href=3D"https://gist.github.com/atmosx/93e7f202539d8d8a0d47">https://gist.=
github.com/atmosx/93e7f202539d8d8a0d47</a><br></blockquote><br>As Ryan =
pointed out, select is taking a long time so that means it's<br>waiting =
on I/O of some sort. &nbsp;Unfortunately your dtruss output =
isn't<br>outputting more detail about which FDs select is waiting =
on.</blockquote><blockquote type=3D"cite"><br>I would look into more =
verbose tracing options of dtruss<br>(probably equivalent to "strace -v =
-T" for Linux).<br></blockquote><div><br></div><div>Thanks for the =
explanation. &nbsp;I tried several different outputs by reading the man =
page. I think this might be a little more =
helpful:&nbsp;</div><div><br></div><div>$ sudo dtruss -ascf =
./test.rb</div><div>[=85 huge file =85]</div><div>100364 &nbsp; =
95102/100683:&nbsp;&nbsp;&nbsp;1863430&nbsp;&nbsp;&nbsp;55365&nbsp;&nbsp; =
&nbsp;&nbsp;10 select(0xE, 0x805D7D880, 0x0)&nbsp; &nbsp;&nbsp;=3D 1 =
0<br>100365&nbsp;<br>100366&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp;&nbsp;0x80188606a<br>100367&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x8009abe2d<br>100368&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x8009ac2a2<br>100369&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`getdiskbyname+0x3ad<br>100370&nbsp;&nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x8009ab486<br>100371&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x8008a962f<br>100372&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp;&nbsp;0x80402c1f2<br>100373&nbsp;&nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x8009a2250<br>100374&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x8009a1e42<br>100375&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x80098de58<br>100376&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x80099a824<br>100377&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`0x80099b125<br>100378&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`fmtmsg+0x201<br>100379&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;libc.so.7`fmtmsg+0x12e<br>100380&nbsp;&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp;&nbsp;env`malloc+0x9df<br>100381&nbsp;&nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp;&nbsp;env`malloc+0x8af<br>100382&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; =
&nbsp;&nbsp;ld-elf.so.1`0x80061d000<br>100383&nbsp;<br></div><div>[=85 =
huge file =85]</div><div><br></div><div>You can see the entire output =
here:&nbsp;<a =
href=3D"https://gist.githubusercontent.com/atmosx/e32f494b17f5a9f4b1f4/raw=
/dtruss">https://gist.githubusercontent.com/atmosx/e32f494b17f5a9f4b1f4/ra=
w/dtruss</a>&nbsp;(but it=92s kinda huge, I used vim to find =91select=92 =
system calls)</div><div>I=92m not sure what the problem however, it=92s =
can be reproduced only on my VPS[1]. Other people with FreeBSD=92s are =
generally slower than Linux but by 1-2s not.</div><div><br></div><div>I =
don=92t this is something related to ruby. I think there something wrong =
with the VPS. The virtualisation engine maybe, who =
knows.</div><br><blockquote type=3D"cite"><br>One more thing: have you =
checked if the <a href=3D"http://api.skroutz.gr">api.skroutz.gr</a> =
endpoint is<br>throttling your server IP for giving it too much =
traffic?<br></blockquote><div><br></div><div>No I=92m 100% that=92s not =
an issue. I=92m respecting their limits + my calls are to slow to reach =
their limit anyway.</div><br><blockquote type=3D"cite"><br><blockquote =
type=3D"cite"><blockquote type=3D"cite">I would also take a look at the =
Faraday gem + dependencies which it<br>seems to use for HTTP access. =
&nbsp;It could be misconfigured TCP knobs<br>(TCP_NODELAY may be needed, =
or forgot to uncork the socket on<br>TCP_NOPUSH).<br><br>I suppose you =
can also try doing the equivalent with the =
excellent<br>Net::HTTP::Persistent gem =
&nbsp;.<br></blockquote></blockquote><br><blockquote type=3D"cite">I =
tried using =91open-uri=92 to do download a file of 1024kb =
side.<br>Net:Http made about 12s while OpenURI about 22s not that =
system.<br></blockquote><br>s/not that system/on that system/ =
?<br></blockquote><br></div><div>Yes :-) See [1] for details! The first =
system is the VPS (14s) while on other systems takes about 5-6s =
realtime.</div><br><div><br class=3D"webkit-block-placeholder"></div><div =
apple-content-edited=3D"true">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;"><div style=3D"color: rgb(0, 0, 0); letter-spacing: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;"><div style=3D"color: rgb(0, 0, 0); font-family: =
Helvetica; font-style: normal; font-variant: normal; font-weight: =
normal; letter-spacing: normal; line-height: normal; orphans: 2; =
text-align: -webkit-auto; text-indent: 0px; text-transform: none; =
white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div =
style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: 2; text-align: -webkit-auto; =
text-indent: 0px; text-transform: none; white-space: normal; widows: 2; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;"></div>[1]&nbsp;<a =
href=3D"https://gist.github.com/atmosx/14efea27eb2c1e38af09">https://gist.=
github.com/atmosx/14efea27eb2c1e38af09</a><br><br>Panagiotis (atmosx) =
Atmatzidis<br><br>email:<span class=3D"Apple-tab-span" =
style=3D"white-space: pre;">	</span><a =
href=3D"mailto:atma / convalesco.org">atma / convalesco.org</a><br>URL:<span =
class=3D"Apple-tab-span" style=3D"white-space: pre;">	=
</span>http://www.convalesco.org<br>GnuPG ID: 0x1A7BFEC5<br>gpg =
--keyserver pgp.mit.edu --recv-keys&nbsp;1A7BFEC5<br><br>"<span =
style=3D"orphans: auto; widows: auto;">As you set out for =
Ithaca,&nbsp;</span><span style=3D"orphans: auto; widows: auto;">hope =
the voyage is a long one,&nbsp;</span><span style=3D"text-align: =
-webkit-auto; orphans: auto; widows: auto;">full of adventure, full of =
discovery [...]</span><span style=3D"text-align: -webkit-auto;">" =
-&nbsp;C. P. Cavafy</span></div><div style=3D"color: rgb(0, 0, 0); =
font-family: Helvetica; font-style: normal; font-variant: normal; =
font-weight: normal; letter-spacing: normal; line-height: normal; =
orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: =
none; white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;"><br></div><div style=3D"color: rgb(0, 0, 0); =
font-family: Helvetica; font-style: normal; font-variant: normal; =
font-weight: normal; letter-spacing: normal; line-height: normal; =
orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: =
none; white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;"><br></div></div></div></div></div><br =
class=3D"Apple-interchange-newline">
</div>
<br></div></body></html>=

--Apple-Mail=_51277E3B-2A33-42F6-8E71-C13CB69CC07A--

--Apple-Mail=_F23E1788-C11D-4601-B3EE-380E7991077C
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: Public Key Encryption

iQIcBAEBAgAGBQJUE1k8AAoJEPy01a8ae/7FihwP/0oZ58GlqbcmNfiBgR/pw9k+
oULq6AAPrzqrGgr+ou33HJbZQM20PixilvNAAQHZAzTmOoALO15VBkwVjotSvtfp
dwnvEG60nTKjksccQoHhPTc1zT68bkRB1yCERPxQZ/FCG2JkN6dfWKzBAQPLWNlI
TtNMM7pVb7PmgC4KEWVOhGgk4gaJNwJCi7vMEGazeoKfOxEe05QIhmfCF7KhHJvg
dNbwQVO22G6BPdzvxXiXINWPN8HFKCu6BWv2g0WdzbGHxDQ4OAwixbjP+bbsUHOT
5IRYs6GvkGvl1+WNhdZTXRUbLUWs94aflgC+WAPPYCUJIjNmVPsD78cGpDDIhzdO
He4fMIdZKWBB5iiCzWz8TXGZ2iQ31BihY6qviLd4m6gW68+bkW6MHpVSNIu1Xpkb
WLQrKzH0lqgwWb06ztJlETByatSiArtMLYovmhf+WspqoMp6hoINs6VHVB7woHNY
PVDuZ/v5xwJ55Ur4ve5F/yQh8RunmPIhxD4ck5XUkaJk2NXzUXA5nd7fmhrH7IKS
NpTmPbLK9S0GuAH6Vu3ihjAz0ynWvPJL0222katiW3zOenP9he9sCYc3VGlC3U76
VpkwIMP9FmwgDiMAACfQSAGj/FrBzO21kMuuEoIFr+26sizayDLipZ1qxBOtmngB
TOaLy07C/lHTMlWOtNq5
=jBXY
-----END PGP SIGNATURE-----

--Apple-Mail=_F23E1788-C11D-4601-B3EE-380E7991077C--