On 22 May 2008, at 20:03, Mathias Gaunard wrote:

Edd Dawson wrote:

...

Hello,

A while ago, I implemented the Boost.Thread API in terms of Windows Fibers and <uncontext.h> functionality. The result is the ability to run code written with boost threads in a single OS thread.

How is that different from boost.coroutine?

The API is different. The underlying mechanisms used very similar. Indeed, I looked at Boost.Coroutine as a basis for the library, but concluded that it would be very difficult to use it. I must admit that the reason escapes me currently and I am am happy to be proven wrong in this regard. I think it might have been something to do with not being able to represent the "main thread". Regardless of what's going on underneath, I wanted to be able to run my Boost.Thread-ed code without worrying about synchronization, to check the algorithmic correctness. The Boost.Coroutine interface is nothing like Boost.Thread's. Kind regards, Edd

On 26 May 2008, at 08:13, Ioannis Papadopoulos wrote:

Edd Dawson wrote:

...

Hello,

A while ago, I implemented the Boost.Thread API in terms of Windows Fibers and <uncontext.h> functionality. The result is the ability to run code written with boost threads in a single OS thread.

I find it useful for writing tests where determinism is important i.e. to test algorithmic correctness in isolation from synchronization- related concerns.

I haven't updated it to mirror the new 1.35 API yet, but I was wondering if any other boost users would be interested in this or if it might be thought suitable for inclusion in the boost library collection.

So what exactly is the status of the library?

It compiles and works on the systems listed on my webpage. I can replace all my code that uses boost threads very easily (i.e. by changing compiler/linker include and library search directories). The API still mirrors version 1.34 of Boost.Thread, however.

And how much different would it be from recompiling Boost.Thread with pth (http://www.gnu.org/software/pth/)?

No different. Except it would work on Windows, whereas a pth-based implementation of Boost.Thread wouldn't (AFAICT). Edd

On Thu, May 29, 2008 at 9:52 AM, vicente.botet <vicente.botet@wanadoo.fr> wrote:

----- Original Message ----- From: "Edd Dawson" <lists@mr-edd.co.uk> To: <boost@lists.boost.org> Sent: Tuesday, May 27, 2008 8:06 PM Subject: Re: [boost] Non pre-emptible Boost.Thread

...

On 26 May 2008, at 08:13, Ioannis Papadopoulos wrote:

...

Edd Dawson wrote:

...

Hello,

A while ago, I implemented the Boost.Thread API in terms of Windows Fibers and <uncontext.h> functionality. The result is the ability to run code written with boost threads in a single OS thread.

I find it useful for writing tests where determinism is important i.e. to test algorithmic correctness in isolation from synchronization- related concerns.

I haven't updated it to mirror the new 1.35 API yet, but I was wondering if any other boost users would be interested in this or if it might be thought suitable for inclusion in the boost library collection.

So what exactly is the status of the library?

It compiles and works on the systems listed on my webpage. I can replace all my code that uses boost threads very easily (i.e. by changing compiler/linker include and library search directories). The API still mirrors version 1.34 of Boost.Thread, however.

...

And how much different would it be from recompiling Boost.Thread with pth (http://www.gnu.org/software/pth/)?

No different. Except it would work on Windows, whereas a pth-based implementation of Boost.Thread wouldn't (AFAICT).

IIRC pth works on Windows too. I'll try to reply to some of your questions even if I'm not the author of the library, based on my experience with boost coroutines.

Hello,

The fact that your library mimics the Boost.Thread (1.34) interface is a good thing. Do you plan to move to the 1.35 interface? If yes, will you allow thread interruption?

I think however that limiting the utility of your library to making easier the writing of tests where determinism is important is not enough. Non-preemptible threads have its application domain on its own.

* non preemtible threads could increases the responsiveness and concurrency of an event-driven application because no time slicing.

But you also need asynchronous system call support. Fortunately Boost already has it, it is called Asio :)

* the critical sections needing some kind of synchronization are rare

In fact you need no critical sections. You just need not to voluntary give up the timeslice as long as an invariant does not hold.

Can an application needing only non-preemptible threads, perform better with your library than its preemptive Boost.Thread counterpart?

If it doesn't do any blocking system call *and* the machine as only one cpu, certainly yes. Of course this beg the question of why the application was multithreaded in the first place. An application specifically designed for cooperative multitasking is another thing of course.

Can an application needing only non-preemptible and preemptible threads, perform better if we use your library for the non-preemptible threads and Boost.Thread for the others?

Probably. Remember that in general, without compiler/system support is hard to support moving a non preemptible thread (i.e. a continuation), from an underlying kernel thread to another.

There are two functions that I would like to see in your library suspend() and resume(). Do you think that this represent a big development?

Do your library manages with errno?

It probably doesn't need to. If you need to read errno value, remember to do it before the next preemption point.

If I have understood, all the non-preemtible threads are attached to the same scheduler, there is only one static scheduler. I was wondering if it will be possible to make the scheduler a thread specific storage of a preemptible Boost.Thread and attach the construction of non-preemptible threads to this specific scheduler.

In general yes. for example for Boost.Coroutines you can use Asio io_service as the scheduler and you can have one io_service per kernel thread. Every kernel thread will schedule its own set of coroutines. IMHO, to be more flexible, the scheduler should be application specific and not be part of a general solution. For many applications Asio io_service will work well.

Do you think that we can have a common user space context which can be used by your library and the Boost/Coroutine library, and why not by other boost libraries or the users themself for implementing their own simple user-space context switch? Edd, Giovanni, what do you think?

I hope so. I've refactored coroutine's context as a (one shot) continuation, which should be usable as a generalized non local control flow abstraction. BTW, I was thinking of adding a task based interface on top of continuations in addition of a coroutine interface. Of course Edd's library would also work as well. -- gpd

On Fri, May 30, 2008 at 9:19 PM, Edd Dawson <lists@mr-edd.co.uk> wrote:

On 29 May 2008, at 12:49, Giovanni Piero Deretta wrote:

...

I hope so. I've refactored coroutine's context as a (one shot) continuation, which should be usable as a generalized non local control flow abstraction.

BTW, I was thinking of adding a task based interface on top of continuations in addition of a coroutine interface. Of course Edd's library would also work as well.

So can may I infer that you think I should be able to implement my library on top of the guts of Boost.Coroutine, now?

I certainly hope so. Look at the (one shot) continuation at http://tinyurl.com/55vzjm [boost trac]. Note that: - I might remove the movable interface and give it sort-of-copy semantics (but you better not activate more than one copy: i.e. all copies logically refer to the same state and activating one immediately invalidates all other copies). - I will add support for passing one void* argument the the activated continuation.

The thing that troubled me before was how to implement a default- constructed thread (or the stuff in the this_thread namespace in the 1.35 API).

I didn't allow representing the main 'thread' in boost.coroutine because I didn't find a way to do it without breaking typing in general. Continuations are untyped, so you can certainly do that. I'm willing to do change the continuation interface if it can help you implement the boost thread interface. -- gpd -- gpd

On 29 May 2008, at 09:52, vicente.botet wrote:

----- Original Message ----- From: "Edd Dawson" <lists@mr-edd.co.uk> To: <boost@lists.boost.org> Sent: Tuesday, May 27, 2008 8:06 PM Subject: Re: [boost] Non pre-emptible Boost.Thread

...

On 26 May 2008, at 08:13, Ioannis Papadopoulos wrote:

...

Edd Dawson wrote:

...

Hello,

A while ago, I implemented the Boost.Thread API in terms of Windows Fibers and <uncontext.h> functionality. The result is the ability to run code written with boost threads in a single OS thread.

I find it useful for writing tests where determinism is important i.e. to test algorithmic correctness in isolation from synchronization- related concerns.

I haven't updated it to mirror the new 1.35 API yet, but I was wondering if any other boost users would be interested in this or if it might be thought suitable for inclusion in the boost library collection.

So what exactly is the status of the library?

It compiles and works on the systems listed on my webpage. I can replace all my code that uses boost threads very easily (i.e. by changing compiler/linker include and library search directories). The API still mirrors version 1.34 of Boost.Thread, however.

...

And how much different would it be from recompiling Boost.Thread with pth (http://www.gnu.org/software/pth/)?

No different. Except it would work on Windows, whereas a pth-based implementation of Boost.Thread wouldn't (AFAICT).

Hello,

The fact that your library mimics the Boost.Thread (1.34) interface is a good thing. Do you plan to move to the 1.35 interface? If yes, will you allow thread interruption?

I do plan to move/fork to the 1.35 interface. I don't see why it shouldn't support interruption in the same way as the preemptive implementation, though I haven't looked closely at the details.

I think however that limiting the utility of your library to making easier the writing of tests where determinism is important is not enough. Non-preemptible threads have its application domain on its own.

* non preemtible threads could increases the responsiveness and concurrency of an event-driven application because no time slicing. * the critical sections needing some kind of synchronization are rare

I agree that non-preemptible threads have more uses than those outlined in my motivating case, but I for one might prefer to use something more akin to Boost.Coroutine. I can imagine that some might prefer a threads API, though.

Can an application needing only non-preemptible threads, perform better with your library than its preemptive Boost.Thread counterpart?

Can an application needing only non-preemptible and preemptible threads, perform better if we use your library for the non-preemptible threads and Boost.Thread for the others?

I think Giovanni answered these questions well.

There are two functions that I would like to see in your library suspend() and resume(). Do you think that this represent a big development?

They're theoretically possible to implement, but this is heading more towards what I would consider a coroutines-like interface. There's already a semi-hidden function to schedule a specific thread. I assume this would be what you have in mind for resume()? How would you like suspend() to work? It sounds a bit like what some might call a "yield"? Or perhaps it hides a thread from the scheduler?

Do your library manages with errno?

I don't do anything special with errno. It may get set by the underlying APIs that are used in the implementation. errno won't be "thread" specific, I suspect, so as Giovanni said, make sure to read from errno before a context switch.

If I have understood, all the non-preemtible threads are attached to the same scheduler, there is only one static scheduler.

That's correct.

I was wondering if it will be possible to make the scheduler a thread specific storage of a preemptible Boost.Thread and attach the construction of non- preemptible threads to this specific scheduler.

It's possible. I will need to think hard about how to do it without making things extremely messy :) If I can leverage an existing scheduling mechanism, that would be nice. I will look in to Asio's io_service.

Do you think that we can have a common user space context which can be used by your library and the Boost/Coroutine library, and why not by other boost libraries or the users themself for implementing their own simple user-space context switch?

This would be the ideal situation. Giovanni mentioned his context refactoring, which I will also look in to. Kind regards, Edd

Edd Dawson wrote:

No different. Except it would work on Windows, whereas a pth-based implementation of Boost.Thread wouldn't (AFAICT).

Pth means Portable threads. The whole point of pth is that it is fully platform independent, since it only uses setjmp/longjmp (of course it can use more efficient if available)