On Thu, Nov 4, 2010 at 10:24 PM, Christian Henning wrote:

...

It's really hard to tell, but it just looks like cpp-netlib is having a hard time fetching data from the URL you provided.

My program option is "--source=www.boost.org"

It would be good if you tried to put the "http://" part in the source. Try "--source=http://www.boost.org/" or "-S http://www.boost.org/". I just tried it locally and it does require you put a fully qualified HTTP URI -- meaning there should be an 'http://' in the URI. HTH -- Dean Michael Berris deanberris.com

On Sun, Nov 7, 2010 at 10:56 AM, Christian Henning wrote:

Hi there,

...

It would be good if you tried to put the "http://" part in the source. Try "--source=http://www.boost.org/" or "-S http://www.boost.org/".

I just tried it locally and it does require you put a fully qualified HTTP URI -- meaning there should be an 'http://' in the URI.

Thanks, that worked. Cool!

Nice, you're welcome. :)

Can you tell me why the app creates 8 threads? Is there some documentation that explains how threads are being used in netlib?

That's odd, 8 threads? It only creates 2 AFAIK on Linux, one for the reactor thread and one for the thread that handles the completion handlers. Although I do use Boost.Thread shared futures, which may be creating threads of their own -- I haven't checked yet.

BTW, I know I raised the pointed before. But compiling my code takes up to 1.2GB of memory when compiling boost/network/protocol/http.hpp.

Yeah, happens to me to. Imagine how I get any work done while developing it. :D

It also takes almost 3min to crunch. Why is that? Are you trying to create a compiler benchmark? ;-)

Well, to explain a few reasons why, I'd have to point you to the paper I wrote for BoostCon 2010 which I also gave a talk about. [0] In a nutshell: 1. I use Boost.Spirit internally for parsing -- that already exercises the compiler alone. 2. The tag dispatch mechanism used to determine the behavior of the implementation at compile time relies on template metaprogramming. Traits, factories, and selections are done using tag dispatch which relies heavily on Boost.MPL. 3. The amount of code that's already in the library lends itself to using a lot of memory, and especially since everything is a template, that's more time the compiler spends in processing the templates. The code generated then is as efficient as the compiler can make it -- I hide nothing from the optimizer, and all the opportunities for optimization are there. Functions are inlined, loops unrolled, and in some cases operations are vectorized by the compiler's optimizer. I'm trying (hard) to get the compile times down but adding new features to the HTTP server part and later on supporting more overloads to member and free (template) functions will only make the compile time pretty heavy (or heavier, depending on how you look at it). ;) HTH [0] http://goo.gl/3sU8Q -- Dean Michael Berris deanberris.com

On Fri, Nov 19, 2010 at 3:22 AM, Christian Henning wrote: [snip]

Thanks for your insight. It's very interesting!

You're welcome, thanks! :)

I'm sure this idea already came up but why not let the user have the option of making parts of netlib compile into a lib?

Yes, it's already come up. ;) The real reason is because I don't see which parts of cpp-netlib can be moved into a compiled lib. Most -- I should say "all" -- of the algorithms I implement is generic. Maybe later on I can do that, but only once I see which parts can be moved to a compiled lib. :D Also, I aimed to do it header-only, which I think I've largely succeeded in doing now. Going the other way looks really hard from where I'm sitting. :D Thanks for taking a look and I hope this makes sense. :) -- Dean Michael Berris deanberris.com

On Fri, Nov 19, 2010 at 3:11 PM, Joel de Guzman wrote:

On 11/19/2010 12:10 PM, Dean Michael Berris wrote:

...

On Fri, Nov 19, 2010 at 3:22 AM, Christian Henning

...

I'm sure this idea already came up but why not let the user have the option of making parts of netlib compile into a lib?

Yes, it's already come up. ;) The real reason is because I don't see which parts of cpp-netlib can be moved into a compiled lib. Most -- I should say "all" -- of the algorithms I implement is generic. Maybe later on I can do that, but only once I see which parts can be moved to a compiled lib. :D

Also, I aimed to do it header-only, which I think I've largely succeeded in doing now. Going the other way looks really hard from where I'm sitting. :D

A good way to start is by separating the implementation files from interface files such that the user can include the interface files from headers and the implementation files from cpp plus template instantiations with the client's actual template types.

Interesting. This sound like a lot of work though. My problem is that a lot of the types are determined through template metafunctions. I don't have many opaque types lying around that I can move the implementations of to compilable files. I also have a ton of template functions that are marked 'inline' to avoid ORD violations when compiling/linking. Also, providing the template specializations myself sound like a really bad nightmare. If I didn't aim to support for example a means of supporting std::string, std::wstring, std::vector<char>, CString, QString, and <insert-string-implementation-here> in the messages, I might be able to move implementations over and just support std::string all throughout the library.

I'm sure this will be asked time and time again. It's better to deal with this sooner rather than later.

Yeah, I'm going to have to talk this over with those with more experience with this sort of thing to see how I can even begin to unravel this moving from header-only to compiled-lib thing. :D

Regards,

Thanks Joel for the hints, I'll look again once 0.8 is out to see if this can be made an option! :) -- Dean Michael Berris deanberris.com

On Fri, Nov 19, 2010 at 4:14 PM, Joel de Guzman wrote:

On 11/19/2010 3:25 PM, Dean Michael Berris wrote:

...

On Fri, Nov 19, 2010 at 3:11 PM, Joel de Guzman

...

A good way to start is by separating the implementation files from interface files such that the user can include the interface files from headers and the implementation files from cpp plus template instantiations with the client's actual template types.

Interesting. This sound like a lot of work though.

My problem is that a lot of the types are determined through template metafunctions. I don't have many opaque types lying around that I can move the implementations of to compilable files. I also have a ton of template functions that are marked 'inline' to avoid ORD violations when compiling/linking.

Also, providing the template specializations myself sound like a really bad nightmare. If I didn't aim to support for example a means of supporting std::string, std::wstring, std::vector<char>, CString, QString, and<insert-string-implementation-here>  in the messages, I might be able to move implementations over and just support std::string all throughout the library.

I'm sure you can find ways around these (I'd say) trivial problems with some bits of imagination ;-) I can offer a detailed solution, but I don't want to spoil the fun :-P

LOL :) Yeah, I'll try to figure something out. I'll get some help from the developers list too, some people there have already been asking for a way to sidestep the long compile-time requirements and have a built-library non-header-only option. :D

Hesitation is your main obstacle, I would say.

Yep. I'll give it a go one of these days. Thanks again! :) -- Dean Michael Berris deanberris.com