Issue #11517 has been updated by Jonathan Cruz.


Yura Sokolov wrote:
> Why don't you use separate queue with separate worker? It will solve youruse case.
> 
> Even if it is hard to separate jobs before put into main queue,  workers could push such heavy tasks into separate queue after fetching it from mainqueue.

That is an excellent suggestion. It might complicate the implementation, but it's worth thinking about. We have a configurable number of worker threads running and it would be tricky to honor that while having a single workerthread dedicated to watching the 'heavy work' queue.

This really is just a suggestion for convenience. We already have a work around by reimplementing the Queue class in ruby and subclassing it with thisnew version of #pop (as well as #promote! and #promote_all! as discussed in https://bugs.ruby-lang.org/issues/11518). We just thought others might find it helpful. Since it is no longer implemented in ruby, there is no way to access the underlying array (even from a subclass) and there is no easy way to selectively pop data off the queue or otherwise inspect the queue.

----------------------------------------
Feature #11517: Queue enhancement - conditional pop
https://bugs.ruby-lang.org/issues/11517#change-54219

* Author: Jonathan Cruz
* Status: Open
* Priority: Normal
* Assignee: Yukihiro Matsumoto
----------------------------------------
Im submitting a patch to enhance Queue#pop. This allows the callerto provide a block that accepts data from the queue. Queue#pop will returnthe first element for which the block returns a truthy value, and remove it from the queue. Without a block, Queue#pop will behave the same way it currently does.

The motivation for this enhancement: On our project, we have a queue of work and several worker threads. Some work can incur a heavy load on the system and should not be processed while another worker is processing 'heavy load' work. We need a way for Queue#pop to skip over heavy load items while another thread is processing heavy load work.

---Files--------------------------------
ruby_queue_conditional_pop.patch (3.76 KB)


-- 
https://bugs.ruby-lang.org/