id,summary,reporter,owner,description,type,status,milestone,component,version,severity,resolution,keywords,cc
11895,Strand service scheduling is hurting ASIO scalability,Chris White <chriswhitemsu@…>,chris_kohlhoff,"This problem will be explained best by walking through a scenario:

I have an io_service being run with one thread per CPU core. The CPU has 4 cores. I also have a strand. I then post 10 one-second operations to the strand, and 30 one-second operations to the io_service. That's 40 cumulative seconds of work, and since there are 4 cores, that work should optimally take 10 seconds when performed in parallel.

However, that's not what happens. The work in the io_service is given precedence over the work in the strand. So first, the 30 seconds of cumulative work in the io_service is performed in parallel which takes roughly 7.5 seconds (30 seconds / 4 cores). Only after that work is complete does the 10 seconds of work on the strand get performed. So it ends up taking a total of 17.5 seconds to do all the work because CPU time wasn't distributed optimally.

If we think of the strand and the io_service as queues, what we have here is a queue within a queue. The outer queue's work can be performed in parallel, the inner queue's work is performed serially. It seems to make a whole lot of sense that when control reaches the inner queue, that queue should be serviced until it's empty. After all, there are other threads that can still service the outer queue while that's happening. The opposite is not true. By giving the outer queue priority, ASIO is crippling the concurrency potential of the work in the inner queue i.e. the strand.

This simplified scenario is a very real performance problem for my companies server application. As far as I can tell, the only options for me to address the problem are to implement my own strand that doesn't give control back to the io_service until it's work queue is empty -or- put all of my strands onto one io_service and post all non-strand work to a second, separate io_service.

I attached a sample application that demonstrates the scenario above, complete with timings and a visual print out of the order of operations. It also includes a stripped down strand implementation that demonstrates how the problem could be addressed. The application was built in Visual Studio 2013 with boost version 1.60.0.",Bugs,new,To Be Determined,asio,Boost 1.60.0,Optimization,,strand scheduling priority,