On Tue, 27 Jun 2006 20:18:33 -0500, Rene Rivera <grafikrobot@gmail.com> wrote:

Andy Little wrote:

...

[...] (and if you don't know what ICU is then you probably don't need it).

Well it's not really a Boost.Jam, or Boost.Build problem. It the individual libraries.

...

Thanks Bjam. You know what? I really don't care!

Others might care ;-)

Well, what I've rather found in my experience is that if you don't know what xyz is you probably need(ed) it and missed an important tool :) It's always true, instead, that if you ignore what something is then you shouldn't try to use it. --Gennaro.

"John Maddock" <john@johnmaddock.co.uk> writes:

Rene Rivera wrote:

...

Well it's not really a Boost.Jam, or Boost.Build problem. It the individual libraries.

I disagree: what on earth is bjam doing scanning the regex build instructions for a target that doesn't need regex?

Let's be clear about this: even if I build a project that doesn't use Boost at all with bjam v2 I'll see the messages.

Where did you get that idea? Sorry, John, that's just impossible. The messages are in the Jamfiles for the specific libraries, and BBv2 is usable without any of the rest of Boost. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Hi Volodya ! Am Maandag 03 Juli 2006, 08:29 schrieb Vladimir Prus:

If it's fine with everybody, I can make the warning appear only when the library is actually built.

Yes, I'm +10 for this change. Some of my co-workers are annoyed by these warnings, too.

Note that for folks building entire Boost, e.g. for installation, this won't change anything at all, and I got the impression that OP was specifically asking about build-for-installation.

Well, that's anmother matter. Yours, Jürgen (just back from vacation and slowly catching up...) -- * Dipl.-Math. Jürgen Hunold ! Ingenieurgesellschaft für * voice: ++49 511 262926 57 ! Verkehrs- und Eisenbahnwesen mbH * fax : ++49 511 262926 99 ! Lister Straße 15 * hunold@ivembh.de ! www.ivembh.de

Vladimir Prus wrote:

So, if you use Boost.Build and don't use Boost at all, you won't see any warning. If you use any Boost library, you'll see warning for Boost.Python and Boost.Regex.

Or if your Boost.Build is part of a complete Boost package, but you're not actually using any Boost lib's you'll see the message.

If it's fine with everybody, I can make the warning appear only when the library is actually built.

Yes please :-) John.

"Andy Little" <andy@servocomm.freeserve.co.uk> writes:

"David Abrahams" wrote

...

"Andy Little"writes:

...

Whenever I invoke bjam I get this first:

warning: Python location is not configured warning: the Boost.Python library won't be built Building Boost.Regex with the optional Unicode/ICU support disabled. Please refer to the Boost.Regex documentation for more information (and if you don't know what ICU is then you probably don't need it).

Thanks Bjam. You know what? I really don't care!

Got anything constructive to say about it?

Nope. Just more ranting I'm afraid...

well, please try to do better. Try to remember that we're all on the same team here.

Why not make disabling 'optional' Unicode regex support the default, seeing as its optional and I'm obviously too thick too know what ICU is, so I don't need it as the message rather rudely reminds me. After all if I was clever enough to know what ICU is, I would probably be clever enough to switch Unicode/ICU support on if required wouldnt I, which seesms to be the implication.

Same for Python library. I havent configured it, I obviously don't want it unless some dependency crops up in my source code which I am trying to build. At that point tell me about it and then I'll figure out how to configure Python.

Well, they're a bit crude, but these warnings were put in for the benefit of the person doing bjam install who makes no explicit request to build Python or Regex at all.

The fact is that all my tests and examples seem to build fine without me having done anything about configuring Python or Regex, so these messages are completely bogus AFAICS. What purpose do they serve?

To let you know that although you may get some usable subset of Boost functionality, you won't get all of it. -- Dave Abrahams Boost Consulting www.boost-consulting.com

On 6/28/06 1:14 AM, "Andy Little" <andy@servocomm.freeserve.co.uk> wrote:

"David Abrahams" wrote

...

"Andy Little"writes:

...

Whenever I invoke bjam I get this first:

warning: Python location is not configured warning: the Boost.Python library won't be built Building Boost.Regex with the optional Unicode/ICU support disabled. Please refer to the Boost.Regex documentation for more information (and if you don't know what ICU is then you probably don't need it).

Thanks Bjam. You know what? I really don't care!

Got anything constructive to say about it?

Nope. Just more ranting I'm afraid...

Why not make disabling 'optional' Unicode regex support the default, seeing as its optional and I'm obviously too thick too know what ICU is, so I don't need it as the message rather rudely reminds me. After all if I was clever enough to know what ICU is, I would probably be clever enough to switch Unicode/ICU support on if required wouldnt I, which seesms to be the implication.

Same for Python library. I havent configured it, I obviously don't want it unless some dependency crops up in my source code which I am trying to build. At that point tell me about it and then I'll figure out how to configure Python.

I'm tired of seeing these messages too, but I want the problem to be solved in the other direction. My operating system (Mac OS 10.4) already comes with Python and ICU (and GZip and BZip2). Why do I have to keep getting "can't be found" messages for these optional libraries when Bjam could just use the system's copies? (We would still give an option to skip inclusion or to use a user's private version, since a user can download updates faster than Apple includes them in official builds.)

The fact is that all my tests and examples seem to build fine without me having done anything about configuring Python or Regex, so these messages are completely bogus AFAICS. What purpose do they serve?

-- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com

Jody Hagins <jody-boost-011304@atdesk.com> writes:

Compilers allow you to disable a warning if you don't want it. The build should do the same. How about a command line, or config file, where I can stick in -Wno-python-not-configured -Wno-regex-unicode-disabled or something, so I can make it be quiet...

In BBv1 there was "pass --without-python to suppress this message in the future" in the message. If that's been dropped for v2 it should probably be reinstated. -- Dave Abrahams Boost Consulting www.boost-consulting.com

"John Maddock" <john@johnmaddock.co.uk> wrote:

David Abrahams wrote:

...

Jody Hagins <jody-boost-011304@atdesk.com> writes: In BBv1 there was

"pass --without-python to suppress this message in the future"

in the message. If that's been dropped for v2 it should probably be reinstated.

OK, but none of this addresses the basic issue: why is BBv2 scanning these Jamfiles in the first place, when it manifestly is not building these lib's or anything dependent upon them?

I feel like a borken record, but this would be solved if there was always an explicit configuration step before the actual build. That way, it is only during the one configuration step that you find out that python and ICU will not be available, rather than every single time you build. Many other libraries with lots of optional features operate this way. Cheers, Walter Landry wlandry@ucsd.edu

Hi Robert,

...

1/ First, we are using multiple files, which can be created at different moments. The main reason is that we want to be able to share resources contained in one file for multiple projects. An alarm rung in my head telling me that serialize won't be able to handle this case,

why not?

Because I saw nowhere in the docs how serialize would restore pointers when loading from multiple files. Suppose I load file1 that contains a resource named "r1". After file1 is fully loaded, and say 5 minutes later, I load file2. Inside that file, there is a resource "r2" that needs a link to "r1", how will this case be handled? I suppose I'll have to manually iterate over the freshly loaded resources and check if they need to be "post-loading associated". As I can't know if these resources need or not this last step, I'll have to check each time I load a file, and thus, I could handle the internal linking as well in this step, though it will be easier (and more efficient) if it's done by the loading lib. So, is there something I misunderstood, or some feature I overlooked?

...

2/ And secondly, we want to be able to load files progressively, say 100KB by 100KB. Once the file is completely loaded, the pointer restoration can happen. Does serialize support such feature or plan to? If this is not a huge work, is there a way to provide help to get this feature quickly?

I'm not sure what this means. Taking a wild guess, I would seem that want

you might want to do is run serialization on a separate thread. If you want to "interrupt or suspend that thread, this would require support from a custom archive. Such a custom archive would be derived from one of the existing ones.

Not necessarily from a thread, but the aim is to suspend the loading of a file, and then resume it some time later. What would be even better would be to specify how long or how many bytes should be read before the loading function suspends and returns. I read about your custom archives (some time ago I have to admit), and it didn't seem an evidence to me that I could implement this feature. Could you provide some more hints?

...

Today, we have a manual phase of association that is called after all files are loaded. Each time we load a new file, we "post-load" all the resources contained in it, and pointer restoration is achieved this way.

In the serialization library, pointers are restored "on the fly" (depth first).

Yes, that's why I'm considering porting to it :) What does "depth first" means?

Robert Ramey

Cheers, SeskaPeel.

Hi Robert, sorry for my bad understanding, but your previous answer left me in the haze. Could you please elaborate? Just so that we stay on the same track, I'm using shared_ptr all through my application. I have, as instance, a resource that points to another one, both resources being stored in a manager (even if resources are of different types, they all derive from the same base class). All the way, we have shared_ptr. I use a lot of these links, to express dependency, and hierarchy. I don't even see how I could use another approach, as I need to be able to dynamically change the hierarchy or the dependency. So what I really need, is to be able to have a resource of type USED being loaded, used by some other resources of type USING (as there is a lot of sharing), and then later load an additional resource of type USING, and have that last one linked to the resource of type USED. Does it make sense? Cheers -----Message d'origine----- De : boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] De la part de Robert Ramey Envoyé : mardi 4 juillet 2006 17:16 À : boost@lists.boost.org Objet : Re: [boost] [serialize] Handling multiple files,deferred marshalling, and progressive loading SeskaPeel wrote:

Hi Robert,

...

...

1/ First, we are using multiple files, which can be created at different moments. The main reason is that we want to be able to share resources contained in one file for multiple projects. An alarm rung in my head telling me that serialize won't be able to handle this case,

why not?

Because I saw nowhere in the docs how serialize would restore pointers when loading from multiple files. Suppose I load file1 that contains a resource named "r1". After file1 is fully loaded, and say 5 minutes later, I load file2. Inside that file, there is a resource "r2" that needs a link to "r1", how will this case be handled?

Its not a case I ever considered so off hand I'm not sure how to answer. But I can speculate a little. Take two cases class a { ... x *ptr; template<class Archive> void serialize(Archive, const unsigned int version){ ar >> ptr; } }; class b { x & ref; template<class Archive> void serialize(Archive, const unsigned int version){ ar >> ref; } }; Both cases are quite similar - one object refers to some external object. In the first case a new object of type x is created each time an instance of a is loaded. In the second case the object is presumed to exist and the data is just loaded into it. SO the serialization library sort of presumes that one is using pointers and references in this common way. This is just a decision I made. Most people don't see this as it turns out this is the way the people use pointers vs references so things "just work" like one expects. By making the serialization "fancier" one could replace the behavior for pointers to work like the the current behavior for references which sounds like something you want to do to address your situation. Or you could just change the pointers to references in your own code and get everything to "just work" Its up to you.

I suppose I'll have to manually iterate over the freshly loaded resources and check if they need to be "post-loading associated". As I can't know if these resources need or not this last step, I'll have to check each time I load a file, and thus, I could handle the internal linking as well in this step, though it will be easier (and more efficient) if it's done by the loading lib.

Of course you could do it that way as well.

So, is there something I misunderstood, or some feature I overlooked?

Hmmm - I suspect that since serialization works so painlessly ALMOST all the time, one doesn't have much occasion to look under the hood until things get more complex and specialized. I think that at that point its valuable to take another look at what one is trying to serialize and ask himself - hmmmm - why is this hard? going through and considering each variable as to wheter it should be a pointer, reference, pointer to a "const" object, reference to a "const" object, a "const" pointer to a non-const object, or just a normal member variable (const or not) etc. will often lead to a re-characterization of some variables and then things will again "just work". I also believe that it will improve the rest of one's code as it forces him to think about why each variable is used the way it is. Also, the excercise will end up giving the compiler more information about how each variable is used and can permit the compiler to better optimize generated code (at least theoretically). FWIW I believe that "const" is generally under-appreciated and under-used.

...

...

2/ And secondly, we want to be able to load files progressively, say 100KB by 100KB. Once the file is completely loaded, the pointer restoration can happen. Does serialize support such feature or plan to? If this is not a huge work, is there a way to provide help to get this feature quickly?

If you're in a hurry - consider the prescription above. It will require changing your own classes - for the better in my view - but you'll be done with it.

Not necessarily from a thread, but the aim is to suspend the loading of a file, and then resume it some time later. What would be even better would be to specify how long or how many bytes should be read before the loading function suspends and returns. I read about your custom archives (some time ago I have to admit), and it didn't seem an evidence to me that I could implement this feature. Could you provide some more hints?

The deserialization process uses the stack to store its state. Hence, the only simple and practical way to do this in a practial way is to invoke loading on a separate fiber, coroutine or thread. If you want to hack your own code some you could do something like having a top level array of serializable objects and serialize them each independently so you could to the process piece by piece. The serialization library does permit the same streambuf to be used and passed around so that serialization can be "embedded" inside of some other streambuf operations - etc. But by far the easiest way would be to use the co-routine approach above.

...

...

Today, we have a manual phase of association that is called after all files are loaded. Each time we load a new file, we "post-load" all the resources contained in it, and pointer restoration is achieved this way.

Using my suggestion above - this wouldn't be necessary. The commonly referenced objects are "pre-created and registered" by the top level object constructors and everything is keep in sync automatically throughout the serialization process.

...

In the serialization library, pointers are restored "on the fly" (depth first).

Yes, that's why I'm considering porting to it :) What does "depth first" means?

a contains pointer to b which contains a pointer to c, etc so the sequence of operations is a, b, c, .... . That is, a is complete when only when all its components have been loaded. Robert Ramey _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Hi Robert, You are suggesting to leave the object creation outside the serialize lib, and that's a good point as I have some very specific constructors (though they need some data read from the file), but I can't see where doing this will remove my post loading association phase (and that's the point, I want to remove the post loading phase).

This is how I understand your situation - I'm not sure my understanding is correct - but there it is.

You perfectly understood.

So when asked how to handle the "matching" - my thought is to say - wait a minute - why are we in this situation in the first place? Then re-think the design. Currently after loading and matching to the other pointers in the program an getting things all working is going to be an error prone process. I'm sure its doable, and maybe there is no other options. But when I see a situation where I have to insert some very elaborate code to address the fact the my other components don't quite fit together, I'm thinking that maybe its a good time to step back an review how I got to this situation in the first place.

Whatever happens, I'll always need to link elements coming from separate files, loaded at separate times. This linking I was hoping could be done by the serialize lib, or at least let me do that specific "external" linking (link a resource to another coming from a separate file), and handle for me all the "internal" linking (link resources coming from the same file). But, to be able to do this, the lib would have to let me know which resources weren't successfully linked internally, and thus, need to attempt an external link. Or, I could simply post process all freshly loaded resources, but doing this, as I said earlier, is what I have today and what I'm trying to remove from the code.

One big idea would be not use the serialization library to create new objects but just make sure they created a head of time. This is the essence of my suggestion about using references rather than pointers.

Sorry, I still don't understand how this would help me... [...] And for the progressive loading part, I would like to avoid threads, as I can't be sure that the platform offers multi-threading facilities. I really would like the serialize lib to return after having finished loading, and then be able to resume some time later. What kind of derivation from Archive should I do? SeskaPeel. _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost