[job] exec job-graphs in a NUMA env (was a thread pool that allows recursive calls)
This library is related to the posting 'a thread pool that allows recursive calls' (http://lists.boost.org/Archives/boost/2015/06/223426.php), e.g. the library solves the many-dependent-jobs problem. boost.job is a derivative work of boost.task (which was published in 2008, the code is refactored and now split into boost.context, boost.coroutine2/boost.fiber and boost.job). boost.job provides a framework for asynchronous execution of job-graphs (a job is a small unit of code that can be executed independently and parallel, e.g. it's a fain-grained work item) of arbitrary dependency, e.g. solving the many-dependent-jobs problem (M: worker threads, N: jobs waiting for other jobs; M << N). Each (logical) processor of a dedicated processor set (configurable) gets a worker-thread assigned. Each worker-thread runs a fiber-pool (launch policy is customizable). A submitted job is always executed by a worker-fiber. While using boost.fiber jobs are scheduled cooperatively so that a job can yield to other jobs during its execution. Jobs can be synchronized via primitives like mutex's, conditions, barriers and channels (message exchange) from boost.fiber without blocking the worker-thread. Fibers provide a fast context switch. boost.job supports to inspect the NUMA topology (e.g. which logical processors are online, and which processors share L1/L2/L3 cache). This information can be used to select a appropriate processor for a job. For a fast access to job related data, boost.job provides functions and classes to allocate memory on NUMA nodes (processors of a NUMA node share cache and have a faster access to its associated memory bank than processors of other nodes.). source: https://github.com/olk/boost-job documentation: http://olk.github.io/libs/job/doc/html/
On 8/12/2015 1:27 PM, Oliver Kowalke wrote:
This library is related to the posting 'a thread pool that allows recursive calls' (http://lists.boost.org/Archives/boost/2015/06/223426.php), e.g. the library solves the many-dependent-jobs problem.
boost.job is a derivative work of boost.task (which was published in 2008, the code is refactored and now split into boost.context, boost.coroutine2/boost.fiber and boost.job).
boost.job provides a framework for asynchronous execution of job-graphs (a job is a small unit of code that can be executed independently and parallel, e.g. it's a fain-grained work item) of arbitrary dependency, e.g. solving the many-dependent-jobs problem (M: worker threads, N: jobs waiting for other jobs; M << N).
Each (logical) processor of a dedicated processor set (configurable) gets a worker-thread assigned. Each worker-thread runs a fiber-pool (launch policy is customizable). A submitted job is always executed by a worker-fiber. While using boost.fiber jobs are scheduled cooperatively so that a job can yield to other jobs during its execution. Jobs can be synchronized via primitives like mutex's, conditions, barriers and channels (message exchange) from boost.fiber without blocking the worker-thread. Fibers provide a fast context switch.
boost.job supports to inspect the NUMA topology (e.g. which logical processors are online, and which processors share L1/L2/L3 cache). This information can be used to select a appropriate processor for a job. For a fast access to job related data, boost.job provides functions and classes to allocate memory on NUMA nodes (processors of a NUMA node share cache and have a faster access to its associated memory bank than processors of other nodes.).
source: https://github.com/olk/boost-job documentation: http://olk.github.io/libs/job/doc/html/
This is a very exciting library. IIRC a lot of the feedback from the fiber review was related to differing performance expectations. As I'm reading through the documentation I just want to be sure my expectations are well based. This library is designed designed for high performance, correct?
2015-08-15 2:00 GMT+02:00 Michael Marcin
IIRC a lot of the feedback from the fiber review was related to differing performance expectations.
The opinions differed more in the use cases of the library - hight performance only or/and solving other programming problems. But boost.fiber will never compete with TBB because TBB never supports context switching (has to process the whole job-graph).
As I'm reading through the documentation I just want to be sure my expectations are well based. This library is designed designed for high performance, correct?
Boost.job? - yes, as much has possible, the main feature is the support for NUMA memory allocation.
participants (2)
-
Michael Marcin -
Oliver Kowalke