From: Vicente BOTET <vicente.botet(a)wanadoo.fr>
Date: Tue, Mar 29, 2011 at 7:00 PM
To: boost-users(a)lists.boost.org, boost(a)lists.boost.org
> As Vladimir said you can post the review to the dev ML and follow up the
> review using some archives as nabble boost dev for example.
As a non-member, I cannot post to the dev mailing list: my mail was rejected.
> If you are yet interested in making a review I will accept it on this ML
> exceptionally as the review is almost finished. I will forward to the dev ML.
>
> Waiting for your review,
> Vicente
Thank you! I appreciate that.
Eight days ago I sent my review to you and to Oliver personally.
Although Oliver did receive that and subsequent mail messages, the
fact that you are still awaiting my review means that you did not.
I must ask the Boost Review Wizards to reconsider the dev-list-only
review policy. Since a non-member can't post to the dev list and I
can't reach the Review Manager with personal mail, apparently I have
only two options:
- post to boost-users and hope for the best
- hop on and off the dev list every time I want to interject a
comment. Frankly that doesn't scale well.
I append my review and subsequent conversation with Oliver.
Disinterested parties can stop here.
Forwarded conversation
Subject: [Boost-users] [context review] Review starts today, Mars 21st 2011
------------------------
----------
From: Nat Linden <nat(a)lindenlab.com>
Date: Tue, Mar 22, 2011 at 10:38 PM
To: Vicente BOTET <vicente.botet(a)wanadoo.fr>
Cc: Oliver Kowalke <oliver.kowalke(a)gmx.de>
On Sun, Mar 20, 2011 at 8:18 PM, Vicente BOTET <vicente.botet(a)wanadoo.fr> wrote:
> The review of Oliver Kowalke Context library starts today Mars 21st 2011, and will end on Mars 30th.
>
> Boost.Context is a basic building block library that is used for more higher abstractions. It is used by Boost.Fiber, Boost.Tasklet and Boost.Task from the same author.
I see that Boost.Fiber is on the review queue, but Boost.Tasklet and
Boost.Task are not yet?
> About the library:
>
> Boost.Context provides a possibility to switch between different user-level context and is intended to be the basis for a higher abstraction like coroutine and fiber.
Thumbs up on introducing a separate library at the Context level of
abstraction. YES, I think the library should be accepted into Boost.
> Boost.Context allows to select the mechanism provided by the operating system (ucontext on UNIX, Windows Fibers on Windows) or a faster implementation provided by this library (fcontext using assembler).
Note on the alternative implementation: I am not enthused about the
idea of embedding the fcontext decision in source code. I would rather
make that decision externally in the build machinery.
> ---------------------------------------------------
>
> Please always state in your review, whether you think the library should be
> accepted as a Boost library!
>
> Additionally please consider giving feedback on the following general
> topics:
>
> - What is your evaluation of the design?
It seems adequate, a thin wrapper around the underlying
implementations. Nonetheless I'm going to suggest a few possible
enhancements.
void (fn)(void*) and void* vp are very classic-C. While I'm sure there
are those who prefer to work at that level, I'd also want overloads
supporting a templatized nullary callable, like Fiber, plus the
args-for-implicit-bind variant. Hopefully Fiber and other high-level
abstractions implemented on Context could simply leverage Context's
support rather than reimplementing it. Since every library built on
top of Context will want to support those constructs, if that support
can be centralized in Context, it should be.
One tiny quibble about the order of the parameters in the existing
context() constructor supporting 'context const& nxt': I think of the
function pointer fn and its data pointer vp as a logical pair. It
seems odd to me that the 'nxt' parameter is inserted here:
context(void (fn)(void*), context const& nxt, void* vp, stack_type && stack);
instead of here:
context(void (fn)(void*), void* vp, stack_type && stack, context const& nxt);
especially considering that 'nxt' controls what happens when 'fn' is done.
I'm not sure I understand the usefulness of a context object not
currently associated with a stack object, but I realize that
corresponds with a state designed into the underlying machinery, e.g.
ucontext.h. Oh: the initial current context doesn't manage an explicit
stack. Maybe a base class that represents a context with implicit
stack, and a subclass that represents a context with explicit stack?
(Is the static variable that underlies boost::context::current() thread-local?)
Fiber-local-storage on Windows: may I suggest a context-local<T>
abstraction for this library, implemented as Fiber-local for the
Windows Fibers implementation and thread-local elsewhere?
Exceptions: "must not throw exceptions" may be too limiting. It's hard
for a typical coder to guarantee that no underlying code will throw.
Of course you could require that every fn passed to context's
constructor must have an explicit outermost try/catch(...) -- but
that's the sort of boilerplate that's easy to forget. context could
provide a wrapper containing that try/catch(...). In fact, you could
even use Boost.Exception to propagate any such exception to "somewhere
else" -- perhaps a context user's policy decision? Then one such
policy choice could be "I guarantee no underlying code will throw,"
for those anxious to avoid the overhead.
Exception management may feel like a topic best left for a
higher-level abstraction -- but I don't see how that could work. If
the higher-level abstraction is implemented in terms of Context, and
if Context flatly forbids exceptions, how could the higher layer
support them?
On the other hand, if it *is* reasonable for Context to defer
exception support to a higher level, then the documentation should be
revised to state the requirements for that support.
But again -- since every library built on Context will want exception
support, if it can be centralized in Context, it should be.
Maybe there should actually be three layers: platform-dependent
RawContext with the currently-proposed level of support, Context with
platform-independent extensions as suggested above, then
Fiber/Coroutine/Task/whatever above Context?
What's the use of a stack object representing not-a-stack? Wouldn't
the abstraction be simpler and more robust if every stack object is
guaranteed to represent a valid stack? That would seem to simplify
some of the semantics of Context as well.
That said -- I'm a bit concerned that the documentation seems to imply
that stacks always grow downwards. Are we really never going to
encounter machines on which the opposite is true? Shouldn't the stack
concept include some support for directionality?
The stack concept is fairly dense. Would it make any sense to define
an actual stack abstract base class? Because Context is
template-based, an implementation can (and protected_stack should)
avoid that base class -- but it could provide user stack
implementations with a starting point.
Thumbs up on providing protected_stack rather than a plain memory area.
> - What is your evaluation of the implementation?
I briefly looked over impl/context_ucontext.hpp, and it's pretty
clear, more or less what I'd expect. I glanced into
impl/context_fiber.hpp. I didn't review the rest of the
implementation.
Again, the need to account for a context object that isn't currently
managing a stack feels like clutter.
The Boost.Coroutines documentation contains a discussion of the
possible unreliability of the GetCurrentFiber() == 0x1E00 trick, and
the value of calling IsThreadAFiber() on systems that support it. I
note some ambivalence in context_fiber.hpp about that. Why is the
IsThreadAFiber() implementation commented out?
> - What is your evaluation of the documentation?
At first I was confused about the usefulness of release_stack().
Here's my guess: although Context documentation doesn't say, Fiber
states: "When a fiber returns from its main function the fiber is said
to be finished and can not be further resumed." By implication, you
can't restart a context object that has finished, either. (Worth
stating in Context documentation.) However, you may want to create
another context object to perform the same work again, in which case
it would make sense to reuse the same stack object. Is that more or
less the rationale? Whether or not I've guessed right, I think we
could use a bit more explanation about your intended use of
release_stack().
The Boost.Coroutines documentation notes that you can save the cost of
ConvertThreadToFiber() and ConvertFiberToThread() per round trip by
explicitly calling ConvertThreadToFiber() beforehand.
context_fiber.hpp is implemented the same way; presumably the same
note could be useful. Or are you assuming that all cycle-conscious
users will select the fcontext implementation instead of the Fiber
implementation?
The page describing the stack concept is a bit confusing: it doesn't
use the term "concept," and it actually references class
protected_stack. Please clarify that "stack" is a concept, whereas
"protected_stack" is an implementation of that concept bundled with
the Context library.
There's a confusing phrase in the description of stack::address():
"Function void* address() must return the address where the stack
begins to grow downwards (to lower addresses)" -- so far so good, with
a nod to the note above about stack direction -- "and the stack must
be moveable."
This seems to say that the Context library might move the block of
memory in which the processor allocates stack frames. But that would
completely invalidate data residing in those stack frames! I *think*
the phrase is intended to mean that any class implementing the stack
concept (the object containing stack pointer and size) must be
moveable. Either way, the intent should be expressed more clearly.
What is that phrase doing attached to the description of
stack::address(), anyway?
> - What is your evaluation of the potential usefulness of the library?
C++ should have had standard library support for user-space
context-switching (coroutines, tasks, whatever) a decade or more ago.
As a stepping stone to getting there, Context is enormously valuable.
> - Did you try to use the library? With what compiler? Did you have any
> problems?
I hope to have time over the next few days to try using it. I thought
I'd better send these comments, based only on reading the
documentation and a bit of the code, in case I run out of time.
> - How much effort did you put into your evaluation? A glance? A quick
> reading? In-depth study?
I read through all the bundled documentation, plus a bit of the code.
> - Are you knowledgeable about the problem domain?
Yes. I've been using the (Vault) Boost.Coroutines library in
production code on Windows, Mac and Linux. Because Mac OS X 10.4
doesn't support ucontext.h, I hacked in an alternate underlying
context-switching implementation for 10.4 based on the GNU Pth
library. (Having the Boost.Context API around, even without a viable
implementation yet, would have been useful for that work.)
I intend to try replacing that Boost.Coroutines Pth implementation
with a Context implementation. (Those who still care about OS X 10.4
could then provide a Context implementation based on Pth. However, I
myself no longer care.) If Context is accepted into Boost, It strikes
me that a version of Boost.Coroutines based on Context would be a
useful evolutionary step in the Vault.
----------
From: Oliver Kowalke <oliver.kowalke(a)gmx.de>
Date: Wed, Mar 23, 2011 at 4:07 AM
To: Nat Linden <nat(a)lindenlab.com>, boost(a)lists.boost.org
> I see that Boost.Fiber is on the review queue, but Boost.Tasklet and
> Boost.Task are not yet?
boost.tasklet will be renamed to boost.strand (see my git repo).
because all of those lib require boost.context and hafe to be reqorked
(except boost.fiber) the review wizards requested to remove boost.task
from the review queue.
If boost.context is accepted and in a final state I'll request a
review for the other libs.
> > Boost.Context allows to select the mechanism provided by the operating
> system (ucontext on UNIX, Windows Fibers on Windows) or a faster
> implementation provided by this library (fcontext using assembler).
>
> Note on the alternative implementation: I am not enthused about the
> idea of embedding the fcontext decision in source code. I would rather
> make that decision externally in the build machinery.
I've pulished an alternative version of boost.context - the user can
decide in its code which implemention should be used.
> void (fn)(void*) and void* vp are very classic-C. While I'm sure there
> are those who prefer to work at that level, I'd also want overloads
> supporting a templatized nullary callable, like Fiber, plus the
> args-for-implicit-bind variant. Hopefully Fiber and other high-level
> abstractions implemented on Context could simply leverage Context's
> support rather than reimplementing it. Since every library built on
> top of Context will want to support those constructs, if that support
> can be centralized in Context, it should be.
Becuae most of the work is done ins assembler I'd like to keep the
things simple as possible. Supporting templatized ctors +
args-for-implicit-bind variant would make the assembler too much
complicated and thus error prone (different C++ calling conventions on
the platforms etc.).
So I want to stuck with the C-style callback and track boost.context
as a building block for other libs providing higher abstractions.
> I'm not sure I understand the usefulness of a context object not
> currently associated with a stack object, but I realize that
> corresponds with a state designed into the underlying machinery, e.g.
> ucontext.h. Oh: the initial current context doesn't manage an explicit
> stack. Maybe a base class that represents a context with implicit
> stack, and a subclass that represents a context with explicit stack?
You have to differentiate between a context created to execute a new
execution context (the ctor accepting funtion, stack etc.) which
manages its own stack (passed as argument) and the context instance
grapping the current execution context (default ctor) which collect
the CPU state but does not manage the current stack because it its
already managedd by the operating system (in the case of
main()-function for instance) or by another context create by its ctor
with protected_stack instance as argument.
void main()
{
...
// ctx1 grapps the current CPU state
// but does not manage a stack because the
// current stack belongs to the operating system/process
context ctx1;
// ctx2 represent a new execution path executing my_fn()
// it manages its own stack
context ctx2( my_fn, 0, protected_stack( 65536) );
// now we jump from the curretn execution context ctx1
// to the new execution context ctx2 which will execute
// function my_fn()
ctx1.jump_to( ctx2);
// we return from my_fn() == jumped out of execution context ctx2
// back to ctx1
...
}
ctx1 pin-points (you could have mutliple) a specific point in the
code/execution path.
> (Is the static variable that underlies boost::context::current()
> thread-local?)
context has no static variables and no boost::context::current()
(maybe you are using an old version?).
> Fiber-local-storage on Windows: may I suggest a context-local<T>
> abstraction for this library, implemented as Fiber-local for the
> Windows Fibers implementation and thread-local elsewhere?
hmm - this what I want to handle in libs with higher abstractions. So
boost.context can be thin as possible.
> Exceptions: "must not throw exceptions" may be too limiting. It's hard
> for a typical coder to guarantee that no underlying code will throw.
> Of course you could require that every fn passed to context's
> constructor must have an explicit outermost try/catch(...) -- but
> that's the sort of boilerplate that's easy to forget. context could
> provide a wrapper containing that try/catch(...). In fact, you could
> even use Boost.Exception to propagate any such exception to "somewhere
> else" -- perhaps a context user's policy decision? Then one such
> policy choice could be "I guarantee no underlying code will throw,"
> for those anxious to avoid the overhead.
The function which is pased as first argument to the ctor of context
must not throw. The reason is that it is passed to C-functions (==
functions declared as extern "C") and throwing an exception through
C-code results in undefined behaviour.
> Exception management may feel like a topic best left for a
> higher-level abstraction -- but I don't see how that could work. If
> the higher-level abstraction is implemented in terms of Context, and
> if Context flatly forbids exceptions, how could the higher layer
> support them?
Exception hamdling should be done by higher abstractions. For instance
using packaged_task/future (as shown in boost.tasklet).
> On the other hand, if it *is* reasonable for Context to defer
> exception support to a higher level, then the documentation should be
> revised to state the requirements for that support.
I believe yes.
> What's the use of a stack object representing not-a-stack? Wouldn't
> the abstraction be simpler and more robust if every stack object is
> guaranteed to represent a valid stack? That would seem to simplify
> some of the semantics of Context as well.
stack must support move-semantics -> a moved stack must be represented
== not-a-stack.
This is usefull for reducing memory allocation/deallocation for stacks.
boost.tasklet reuses (== moves) stacks from a context which is
finished and caches the stack. If a new context has to created in
moves the recycled stack into the new context. That's trhre reason why
stack must be move-able.
> That said -- I'm a bit concerned that the documentation seems to imply
> that stacks always grow downwards. Are we really never going to
> encounter machines on which the opposite is true? Shouldn't the stack
> concept include some support for directionality?
On all platforms I've been working on the stack grows downwards.
> The stack concept is fairly dense. Would it make any sense to define
> an actual stack abstract base class? Because Context is
> template-based, an implementation can (and protected_stack should)
> avoid that base class -- but it could provide user stack
> implementations with a starting point.
because the stack is a template argument of context it represents an
implicit interface - so no explicit interface like abstract_stack is
required. The current solution doesn't require stakc to derive from a
base class.
> Thumbs up on providing protected_stack rather than a plain memory area.
protected_stack appends a guard page at the end of the stack. This
guard page issues sefault/access violation if you acces the address of
that page.
This is usefull if the size of the stack is too small. Ypou could use
your own stack implementation without such a guard page. But if you
excced the stack range you overwrite in the worst case the memory of
your own application.
> Again, the need to account for a context object that isn't currently
> managing a stack feels like clutter.
see above - pin-point of current execution path/context
> The Boost.Coroutines documentation contains a discussion of the
> possible unreliability of the GetCurrentFiber() == 0x1E00 trick, and
> the value of calling IsThreadAFiber() on systems that support it. I
> note some ambivalence in context_fiber.hpp about that. Why is the
> IsThreadAFiber() implementation commented out?
That is because of the fact that Windows XP + Sevice Pack 2/3 defines
_WIN32_WINNT >= _WIN32_WINNT_VISTA but doesn't declare
::IsThreadAFiber().
I didn't found a way to detect XP + SP 2/3 beside of _WIN32_WINNT.
> > - What is your evaluation of the documentation?
>
> At first I was confused about the usefulness of release_stack().
moves the stack out of the context instance for inspection or recycling
> The Boost.Coroutines documentation notes that you can save the cost of
> ConvertThreadToFiber() and ConvertFiberToThread() per round trip by
> explicitly calling ConvertThreadToFiber() beforehand.
> context_fiber.hpp is implemented the same way; presumably the same
> note could be useful. Or are you assuming that all cycle-conscious
> users will select the fcontext implementation instead of the Fiber
> implementation?
Holger Grund has some concerns regarding exception handling in the
assembler implementation of fcontext on Windows. So I've dissabled it
- maybe if I could fix it I'll make it available too.
> There's a confusing phrase in the description of stack::address():
> "Function void* address() must return the address where the stack
> begins to grow downwards (to lower addresses)" -- so far so good, with
> a nod to the note above about stack direction -- "and the stack must
> be moveable."
see my comments above
> This seems to say that the Context library might move the block of
> memory in which the processor allocates stack frames. But that would
> completely invalidate data residing in those stack frames! I *think*
> the phrase is intended to mean that any class implementing the stack
> concept (the object containing stack pointer and size) must be
> moveable. Either way, the intent should be expressed more clearly.
> What is that phrase doing attached to the description of
> stack::address(), anyway?
do refere to moving the stack out of a context? then this don for
reusing that memory junk - otherwise the stack get deallocated.
> I intend to try replacing that Boost.Coroutines Pth implementation
> with a Context implementation. (Those who still care about OS X 10.4
> could then provide a Context implementation based on Pth. However, I
> myself no longer care.) If Context is accepted into Boost, It strikes
> me that a version of Boost.Coroutines based on Context would be a
> useful evolutionary step in the Vault.
I'm interessted in porting boost.context to MAC OS X (some first
stepps in Jamfiles -> allow ucontext) - unforutnately I've no box
running MAC OS X. I hope we can work together.
best regards,
Oliver
--
----------
From: Nat Linden <nat(a)lindenlab.com>
Date: Wed, Mar 23, 2011 at 10:22 PM
To: Oliver Kowalke <oliver.kowalke(a)gmx.de>
Cc: boost(a)lists.boost.org, Vicente BOTET <vicente.botet(a)wanadoo.fr>
On Wed, Mar 23, 2011 at 4:07 AM, Oliver Kowalke <oliver.kowalke(a)gmx.de> wrote:
> I've published an alternative version of boost.context - the user can decide in its code which implementation should be used.
I saw your mail about that, but I haven't actually compared its
documentation with the review version's.
Please give an example when it would be useful to have a mixture of
context-switching implementations, selected in source code? Since I
haven't thought of one myself, I'm not convinced it's a good idea.
>> void (fn)(void*) and void* vp are very classic-C. While I'm sure there
>> are those who prefer to work at that level, I'd also want overloads
>> supporting a templatized nullary callable, like Fiber, plus the
>> args-for-implicit-bind variant. Hopefully Fiber and other high-level
>> abstractions implemented on Context could simply leverage Context's
>> support rather than reimplementing it. Since every library built on
>> top of Context will want to support those constructs, if that support
>> can be centralized in Context, it should be.
>
> Because most of the work is done ins assembler I'd like to keep the things simple as possible. Supporting templatized ctors + args-for-implicit-bind variant would make the assembler too much complicated and thus error prone (different C++ calling conventions on the platforms etc.).
> So I want to stuck with the C-style callback and track boost.context as a building block for other libs providing higher abstractions.
I certainly wouldn't ask you to try to implement these higher-level
features in assembler. But I don't see why the library couldn't add
C++ header files providing such features wrapped around the (assembler
or native) underlying functionality.
We will want features such as templatized callables and exception
support in every higher-level library built on top of Context. Must
every such library provide its own redundant, inconsistent
implementation?
>> Maybe a base class that represents a context with implicit
>> stack, and a subclass that represents a context with explicit stack?
>
> You have to differentiate between a context created to execute a new execution context (the ctor accepting function, stack etc.) which manages its own stack (passed as argument) and the context instance wrapping the current execution context (default ctor) which collect the CPU state but does not manage the current stack because it its already managed by the operating system (in the case of main()-function for instance) or by another context create by its ctor with protected_stack instance as argument.
>
> void main()
> {
> ...
> // ctx1 grapps the current CPU state
> // but does not manage a stack because the
> // current stack belongs to the operating system/process
> context ctx1;
>
> // ctx2 represent a new execution path executing my_fn()
> // it manages its own stack
> context ctx2( my_fn, 0, protected_stack( 65536) );
> ...
> }
>
> ctx1 pin-points (you could have multiple) a specific point in the code/execution path.
Fair enough.
Here are two different objects initialized in different ways,
containing different state. Maybe they should be different classes?
They share some operations, but other operations are only valid on one
of these object types. Maybe these should be base class and subclass?
>> (Is the static variable that underlies boost::context::current()
>> thread-local?)
>
> context has no static variables and no boost::context::current() (maybe you are using an old version?).
I downloaded Context from the vault before the review began, so yes, I
guess that reference to boost::context::current() is obsolete.
My question about a static variable is based on a mistaken guess. I
assumed that there would be a global pointer referencing the context
object passed to the last successful jump_to() call: the context
object running now. You're saying there is not. I believe I
understand.
>> Fiber-local-storage on Windows: may I suggest a context-local<T>
>> abstraction for this library, implemented as Fiber-local for the
>> Windows Fibers implementation and thread-local elsewhere?
>
> hmm - this what I want to handle in libs with higher abstractions. So boost.context can be thin as possible.
Then I'd like to reiterate my suggestion that the library you're
proposing is the bottom level of a three-level library stack: this one
that implements "raw" functionality, a middle layer that implements
higher-level concepts useful to every top-level library, and whichever
top-level library suits the application.
It bothers me to assume that every top-level library must redundantly
implement these high-level features.
In terms of packaging, though, I actually think the library we're
calling Context should provide both lower layers. That's because of my
belief that every library implemented on top of Context will want the
high-level features. Put differently, I'm not sure how useful the
bottom layer is without the middle layer. Either way, I agree that we
want to be able to directly access the bottom layer.
>> On the other hand, if it *is* reasonable for Context to defer
>> exception support to a higher level, then the documentation should be
>> revised to state the requirements for that support.
>
> I believe yes.
Perhaps you're saying that the same wrapper machinery that handles a
templated callable could provide an outermost try/catch(...). That
wrapper, guaranteed not to throw a C++ exception, will be executed by
the top-level function passed to Context.
Again, every realistic consumer of Context will need this...
>> What's the use of a stack object representing not-a-stack?
>
> stack must support move-semantics -> a moved stack must be represented == not-a-stack.
> This is useful for reducing memory allocation/deallocation for stacks.
> boost.tasklet reuses (== moves) stacks from a context which is finished and caches the stack. If a new context has to created in moves the recycled stack into the new context. That's the reason why stack must be move-able.
Okay, there's a default-constructed stack object which manages no
memory, a stack object managing a memory area which is potentially
useful for a processor stack, and a stack object managing a memory
area actually in use by a processor stack. I'm still unclear on the
usefulness of the first kind, a default-constructed stack object.
>> Shouldn't the stack
>> concept include some support for directionality?
>
> On all platforms I've been working on the stack grows downwards.
I'll defer to the community. If any Boost users are targeting machines
with stacks that grow upwards, I'll let them request Context
protected_stack support.
>> Why is the
>> IsThreadAFiber() implementation commented out?
>
> That is because of the fact that Windows XP + Sevice Pack 2/3 defines _WIN32_WINNT >= _WIN32_WINNT_VISTA but doesn't declare ::IsThreadAFiber().
> I didn't found a way to detect XP + SP 2/3 beside of _WIN32_WINNT.
Understood, thanks. I'm rolling my eyes, but not at you.
>> > - What is your evaluation of the documentation?
>
>> The Boost.Coroutines documentation notes that you can save the cost of
>> ConvertThreadToFiber() and ConvertFiberToThread() per round trip by
>> explicitly calling ConvertThreadToFiber() beforehand.
>> context_fiber.hpp is implemented the same way; presumably the same
>> note could be useful. Or are you assuming that all cycle-conscious
>> users will select the fcontext implementation instead of the Fiber
>> implementation?
>
> Holger Grund has some concerns regarding exception handling in the assembler implementation of fcontext on Windows. So I've disabled it - maybe if I could fix it I'll make it available too.
Okay, so at present even cycle-conscious Windows users must use the
Fiber implementation. Therefore it does seem worth documenting the
fact that calling ConvertThreadToFiber() on the current thread before
first entry to Context can save a couple implicit
ConvertThreadToFiber() and ConvertFiberToThread() calls per round
trip.
>> I intend to try replacing that Boost.Coroutines Pth implementation
>> with a Context implementation.
>
> I'm interested in porting boost.context to MAC OS X (some first steps in Jamfiles -> allow ucontext) - unfortunately I've no box running MAC OS X. I hope we can work together.
Sure! As I said, unless one selects the 10.4u (or earlier) SDK, the
ucontext.h implementation should work. I haven't yet tried to build
the Context library; is the ucontext.h implementation header-only?
----------
From: Oliver Kowalke <oliver.kowalke(a)gmx.de>
Date: Thu, Mar 24, 2011 at 3:06 AM
To: Nat Linden <nat(a)lindenlab.com>, boost(a)lists.boost.org
> On Wed, Mar 23, 2011 at 4:07 AM, Oliver Kowalke <oliver.kowalke(a)gmx.de>
> wrote:
>
> > I've published an alternative version of boost.context - the user can
> decide in its code which implementation should be used.
I think I follow Vicentes suggestion - boost.context provides only one
class (boost::context<>) which preferes fcontext(asm) implementation
if available otherwise ucontext/fiber.
I'll implement a test-app in order to check how this design influences
UNIX signal handling
(hopefully the impact will be only to require not to change the UNIX
signal mask in context<>).
> I certainly wouldn't ask you to try to implement these higher-level
> features in assembler. But I don't see why the library couldn't add
> C++ header files providing such features wrapped around the (assembler
> or native) underlying functionality.
It would require to do an additional allocation and it might be that
the higher implementations can do it more efficient/better suitable
for there purposes/not required by some higher libs.
Otherwise those libs would pay for the additional memory allocaton
even if it would not be required.
> We will want features such as templatized callables and exception
> support in every higher-level library built on top of Context. Must
> every such library provide its own redundant, inconsistent
> implementation?
The higher libs can implement what they want and how they want.
> Here are two different objects initialized in different ways,
> containing different state. Maybe they should be different classes?
I think they should not be different - both do the same (jumping to
other context). The difference is only the one does and the other
doesn't manage a stack - but bot represent an execution context.
> Perhaps you're saying that the same wrapper machinery that handles a
> templated callable could provide an outermost try/catch(...). That
> wrapper, guaranteed not to throw a C++ exception, will be executed by
> the top-level function passed to Context.
>
> Again, every realistic consumer of Context will need this...
I can add an additional trampoline function which has a try/catch-all
block and issues an assertion if an exception was thrown. But this
adds an additional indirection!
> Okay, there's a default-constructed stack object which manages no
> memory, a stack object managing a memory area which is potentially
> useful for a processor stack, and a stack object managing a memory
> area actually in use by a processor stack. I'm still unclear on the
> usefulness of the first kind, a default-constructed stack object.
The first is required to be able to jump from any position in the code
to the second kind of context (which starts a new execution context).
> Sure! As I said, unless one selects the 10.4u (or earlier) SDK, the
> ucontext.h implementation should work. I haven't yet tried to build
> the Context library; is the ucontext.h implementation header-only?
yes
regards,
Oliver
--
----------
From: Nat Linden <nat(a)lindenlab.com>
Date: Thu, Mar 24, 2011 at 8:05 AM
To: Oliver Kowalke <oliver.kowalke(a)gmx.de>
Cc: Vicente BOTET <vicente.botet(a)wanadoo.fr>
On Thu, Mar 24, 2011 at 3:06 AM, Oliver Kowalke <oliver.kowalke(a)gmx.de> wrote:
>> I certainly wouldn't ask you to try to implement these higher-level
>> features in assembler. But I don't see why the library couldn't add
>> C++ header files providing such features wrapped around the (assembler
>> or native) underlying functionality.
>
> It would require to do an additional allocation and it might be that the higher implementations can do it more efficient/better suitable for there purposes/not required by some higher libs.
With a centralized implementation of high-level features, if someone
figures out how to make it more efficient, every consumer benefits.
> Otherwise those libs would pay for the additional memory allocation even if it would not be required.
I disagree. You need only continue to support the classic-C-style
(void (*function_ptr)(void*), void* data_ptr) pair as an alternative
to the templatized callable, for example by a different constructor
overload. If a consumer doesn't want to pay for extra
allocation/indirection, that caller can use the (function_ptr,
data_ptr) pair as you presently intend.
But a consumer library that does want the high-level features can
leverage your central implementation. Everybody wins.
>> there's a default-constructed stack object which manages no
>> memory, a stack object managing a memory area which is potentially
>> useful for a processor stack, and a stack object managing a memory
>> area actually in use by a processor stack. I'm still unclear on the
>> usefulness of the first kind, a default-constructed stack object.
>
> The first is required to be able to jump from any position in the code to the second kind of context (which starts a new execution context).
This confuses me.
I do understand that a default-constructed *context* object is useful
to be able to switch context from any place in the code.
But I'm now asking about the usefulness of a default-constructed
*stack* object. A default-constructed *context* object is not passed a
stack object.
If you're saying that a default-constructed context object must
nonetheless store a default-constructed stack object... this starts to
feel like an implementation detail leaking out into the API.
(This might be another good reason to break out a context object base
class that stores only processor state without also storing a stack
object.)
Let me suggest an alternative way to manage stacks.
Consider a raw stack memory area which never moves. It is allocated
and managed by a stack class which either successfully allocates the
requested size, or throws. The stack class object, containing only a
pointer and length and other small state, is moveable; but even when
the stack object moves, the actual memory area used by the processor
stack does not move. This would eliminate the need to track a
"not-a-stack" state: every validly-constructed stack object is a valid
stack.
Internally, a context object can store a pointer to a stack object,
possibly with std::auto_ptr for ownership transfer.
I do not believe that the overhead of following a pointer would exceed
the overhead of having to check for a valid stack object on every
access.
And as I've suggested, if you want to eliminate even that overhead,
break out a context class that always stores a valid stack, with a
separate context class that does not even have a stack data member.
>> [For Mac OS X,] unless one selects the 10.4u (or earlier) SDK, the
>> ucontext.h implementation should work. I haven't yet tried to build
>> the Context library; is the ucontext.h implementation header-only?
>
> yes
In that case, what further Context work is needed to support Mac OS X?
If you're saying that you want to extend the fcontext implementation
for OS X, I'm sorry, I don't know details of how the assembler
implementation should differ on OS X.
Perhaps you're confident in the existing assembly code, and you simply
want Boost.Build help in running the assembler on Mac. Unfortunately
(my guilty secret), I do not know Boost.Build. I tried using an
earlier version some years ago, but it failed in ways I found utterly
opaque; it was simpler to build Boost sources as part of my
application project.
----------
From: Oliver Kowalke <oliver.kowalke(a)gmx.de>
Date: Thu, Mar 24, 2011 at 8:36 AM
To: Nat Linden <nat(a)lindenlab.com>
Cc: boost(a)lists.boost.org
> This confuses me.
>
> I do understand that a default-constructed *context* object is useful
> to be able to switch context from any place in the code.
>
> But I'm now asking about the usefulness of a default-constructed
> *stack* object. A default-constructed *context* object is not passed a
> stack object.
because og the move-semantics supported by the stack class an default
constructed stack instance means not-a-stack. It doesn't allocates
memory.
> If you're saying that a default-constructed context object must
> nonetheless store a default-constructed stack object... this starts to
> feel like an implementation detail leaking out into the API.
It is not leaking - all classes providing move-semantics are used in
this ways. Please look into boost.thread - boost::thread has a default
ctor -> constructs a boost::thread instance that refers to
Not-a-Thread.
> In that case, what further Context work is needed to support Mac OS X?
assembler implementation/fcontext - the source code has already an
assembler file for MAC OS X (x86-64/i386). I wasn't able to test the
code because I haven't a MAC OS X installation.
Oliver
--
----------
From: Nat Linden <nat(a)lindenlab.com>
Date: Thu, Mar 24, 2011 at 11:31 AM
To: Oliver Kowalke <oliver.kowalke(a)gmx.de>
On Thu, Mar 24, 2011 at 8:36 AM, Oliver Kowalke <oliver.kowalke(a)gmx.de> wrote:
>> what further Context work is needed to support Mac OS X?
>
> assembler implementation/fcontext - the source code has already an assembler file for MAC OS X (x86-64/i386). I wasn't able to test the code because I haven't a MAC OS X installation.
Okay, I'll try building it on OS X 10.5 and running your tests. It may
take a little while for me to be able to pay attention to it.
