--Apple-Mail=_D8F90B31-EAD7-4852-8763-1D66299ADFFF
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_8ED8F155-11AA-48FB-9978-8C6BBF070BB4"


--Apple-Mail=_8ED8F155-11AA-48FB-9978-8C6BBF070BB4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

Hi.
I am writing a client application, where I face consumer-producer =
problem.
Basically, the idea is that I have connection limit - because of that I =
need to limit amount of networking threads. I create multiple =
connections to the server and store them, as objects, in the queue. I =
don't want to reconnect every time - this would limit data transfer =
rate. But the data which I want to send to the server isn't available =
from the beginning. Because of that I implement the second queue, where =
I store this data.
So I have these two queues and I need to process the data in some =
reasonable manner. For this, I create thread pool(thread limit =3D =
connection limit) based on following class:
=
https://github.com/Burgestrand/burgestrand.github.com/blob/master/code/rub=
y-thread-pool/thread-pool.rb
And every scheduled job pops objects from the queues: data and =
connection. After data was transferred, connection is pushed to the =
queue where it came from and the next scheduled job, which was waiting, =
pops it and reuses.=20
This model is working, but not ideally. Data transfer rate is lower than =
it could be and speed is irregular. Network monitor shows diagram which =
looks like this:
=
/=E2=80=BE=E2=80=BE=E2=80=BE=E2=80=BE=E2=80=BE\/=E2=80=BE=E2=80=BE=E2=80=BE=
=E2=80=BE=E2=80=BE=E2=80=BE\/=E2=80=BE=E2=80=BE=E2=80=BE=E2=80=BE=E2=80=BE=
=E2=80=BE=E2=80=BE\/=E2=80=BE=E2=80=BE=E2=80=BE=E2=80=BE=E2=80=BE=E2=80=BE=
\
First I thought that I am encountering this problem, because I data =
isn't prepared as fast as it should, but this isn't the case. Now I =
think that I see this behaviour, because pushing/popping to queue isn't =
as good as I thought it would be, but I don't have any other ideas how I =
could solve this particular problem.
I would be thankful for any advice.

-----
Tad

--Apple-Mail=_8ED8F155-11AA-48FB-9978-8C6BBF070BB4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=utf-8

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;">Hi.<div>I am writing a client application, where I =
face consumer-producer problem.</div><div>Basically, the idea is that I =
have connection limit - because of that I need to limit amount of =
networking threads. I create multiple connections to the server and =
store them, as objects, in the queue. I don't want to reconnect every =
time - this would limit data transfer rate. But the data which I want to =
send to the server isn't available from the beginning. Because of that I =
implement the second queue, where I store this data.</div><div>So I have =
these two queues and I need to process the data in some reasonable =
manner. For this, I create thread pool(thread limit =3D connection =
limit) based on following class:</div><div><a =
href=3D"https://github.com/Burgestrand/burgestrand.github.com/blob/master/=
code/ruby-thread-pool/thread-pool.rb">https://github.com/Burgestrand/burge=
strand.github.com/blob/master/code/ruby-thread-pool/thread-pool.rb</a></di=
v><div>And every scheduled job pops objects from the queues: data and =
connection. After data was transferred, connection is pushed to the =
queue where it came from and the next scheduled job, which was waiting, =
pops it and reuses.&nbsp;</div><div>This model is working, but not =
ideally. Data transfer rate is lower than it could be and speed is =
irregular. Network monitor shows diagram which looks like =
this:</div><div><span style=3D"font-family: sans-serif; font-size: 13px; =
line-height: 19px; background-color: rgb(249, 249, =
249);">/=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE\/</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE\/</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE\/</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE</span><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">=E2=80=BE\</span></div><div><span style=3D"background-color: =
rgb(249, 249, 249); font-family: sans-serif; font-size: 13px; =
line-height: 19px;">First I thought that I am encountering this problem, =
because I data isn't prepared as fast as it should, but this isn't the =
case. Now I think that I see this behaviour, because pushing/popping to =
queue isn't as good as I thought it would be, but I don't have any other =
ideas how I could solve this particular problem.</span></div><div><span =
style=3D"background-color: rgb(249, 249, 249); font-family: sans-serif; =
font-size: 13px; line-height: 19px;">I would be thankful for any =
advice.</span></div><div><span style=3D"background-color: rgb(249, 249, =
249); font-family: sans-serif; font-size: 13px; line-height: =
19px;"><br></span></div><div><span style=3D"background-color: rgb(249, =
249, 249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">-----</span></div><div><span style=3D"background-color: rgb(249, =
249, 249); font-family: sans-serif; font-size: 13px; line-height: =
19px;">Tad</span></div></body></html>=

--Apple-Mail=_8ED8F155-11AA-48FB-9978-8C6BBF070BB4--

--Apple-Mail=_D8F90B31-EAD7-4852-8763-1D66299ADFFF
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iQIcBAEBAgAGBQJSj7UJAAoJEC0WlNH4dX4B6pwP/Rr1nRw/VnrBD170sTiVATEM
wlEaebMEh+ybIzPVmtFLJQW4JqMv+6houA8X1mrGQnz1qz1cky0+y7FuwhsSNy21
XTWVlz7/OIRVFiI7H6KObcZ9MGqOxU/z/LcXoNYe24rncmhowh3HcISXAJ0WZwYK
I4lWzxRCkxbOO5woNTxakQYECwDKvohRQgSi4mxSY3ocv85RP/0zactkWfRJHppP
AyKgIbskinromwS44FmU74H0Au5ZiQC/mmx5lWUoB28n6zj6wNlue+vx6CA/oKOR
CTCNj7SwtWeNcbs8KM8ViQYK6pIyZe9WUjROLw8E0rTbcKbIvMsUo/gcLiHLN+Mg
c4If8Q+Zgmu2o0CcKud64sKQN5gxN/OTrMIwN0S3HQluj1YYVCupgqb0oFEh3FZ2
F8Jy+aiLdgj9Ie9j19eMKDCTY7D90c/am7YcBFeaPewwKGf4CjdfIX56pci4WSt7
VljVNxM6kNGRsPk0XbHsRS0is8sU7UqUwWx/B1GOZfko460h5l420owZHxQFMgQX
VGsHOWwliG3aUlFIGcsRpbn9Rl9a+5j4jbYSeEnACF94DqOWvpJso88O8UAwh5QL
xbtb7sSvcrgYSvIzYrcbCfyZNHZ1GiN9bkwpCtlCgRnLkV7Y4kPca99+nKPgc+K+
avsdQc37vFautb3kTaK5
=d+c+
-----END PGP SIGNATURE-----

--Apple-Mail=_D8F90B31-EAD7-4852-8763-1D66299ADFFF--