June 2017 - Boost - lists.preview.boost.org

[program_options] behaviour change for implicit options
by Vladimir Prus 25 Jun '17

25 Jun '17

Hi, I have committed, and merged to master, a change making program_option consider subsequent tokens for values of implicit options. For example, if "foo" is implicit option and "bar" is some other option, then: 1. "--foo --bar" will be parsed as option "foo" followed by "bar", and implicit option for "foo" will be used. 2. "--foo bar" will be parsed as option "foo" with the value of "bar" This is the way things worked prior to 1.59, was changed in 1.59, continued to cause confusion, and is now reverted back. For background, see: - http://permalink.gmane.org/gmane.comp.lib.boost.devel/269832 - https://github.com/boostorg/program_options/issues/25 Just in case client code might need to check this behaviour, I've added a feature macro, BOOST_PROGRAM_OPTIONS_IMPLICIT_VALUE_NEXT_TOKEN, which will be set to 1 starting with 1.65. Thanks, Volodya

1 0

Boost MSM attribute sharing problem
by Denis Kotov 25 Jun '17

25 Jun '17

Hello everyone, I have an issue with* Boost MSM*. Using *Boost MSM eUML* I have created a state machine. Actually I created *HSM* (Hierarchical State Machine) or Composite State Machine as second name. And I faced with issue: How to share attributes between Parent State Machine and Sub-State Machines (or Child State Machine) ?? I have not found an answer on this question ... Could anyone possibly help me ? *Thanks,* *Best RegardsDenis Kotov*

1 0

Noexcept improvements
by Emil Dotchevski 25 Jun '17

25 Jun '17

I've completed a second iteration on Noexcept, incorporating valuable feedback from the previous discussions here, and I'd like to thank everyone who participated. https://zajo.github.io/boost-noexcept/ The new result<T> type has a hard value-or-error invariant and can be used to transport errors between threads or to temporarily postpone the consumption of function results, e.g. by storing result<T> objects into a queue (of course, as before, users do not need result<T> just to report a failure from a function.) There is also a new macro, BOOST_NOEXCEPT_CHECK, which makes writing error-neutral functions easier and improves error checking in general. The documentation is also much more complete, including better examples and a new section on programming techniques.

1 0

Fw: [stacktrace] [winapi] header out of place
by Peter Dimov 24 Jun '17

24 Jun '17

The header stacktrace: boost/detail/winapi/capture_stack.hpp should probably either be moved to Winapi, as the name suggests, or renamed to reside in boost/stacktrace/detail. And, as a general rule, libraries should strive to keep their headers under boost/<libname>, with the only exception being boost/<libname>.hpp. We have many legacy libraries not following this requirement, but any new ones and any new headers added to existing ones should. The reason for this requirement is that otherwise it's impossible to perform dependency analysis unless all of Boost is checked out so that the tool can tell in what module boost/random-header-name.hpp resides.

3 2

[review] Boost.Nowide is Accepted into Boost
by Frédéric Bron 24 Jun '17

24 Jun '17

Dear all, I would like to thank all the people who took part in the discussions during the review. All contributors admitted that Boost.Nowide addresses a real issue and solves it, at least partially. Moreover we received 8 official reviews with 6 “accept”, 1 “do not accept as is” and 1 “I am not against it”. So Boost.Nowide is accepted for inclusion in Boost. Congratulation and many thanks to Artyom Beilis for this contribution. However, 2 major issues arose in the discussions and they will have to be fixed before inclusion: 1. handling of ill-formed Unicode input should be improved/clarified 2. the documentation needs improvements In particular, this needs to be fixed: * Design: There was a lot of discussions on what to do with ill-formed UTF-16 strings on Windows, in particular coming from the file system while converting them to narrow UTF-8 strings. Basically, 3 options: 1. allow the roundtrip ill formed UTF-16 > UTF-8 > ill formed UTF-16, this is not possible with UTF-8 encoding, WTF-8 (not a standard) may be an option with some issues 2. issue an error (today’s nowide option, even if the author would like to move to 3.) 3. convert as much as possible to fully conformant UTF-8 with character replacement U+FFFD for invalid input. This is what the NT kernel does with functions like RtlUTF8ToUnicodeN. This would allow cout to continue to work after invalid output instead of failing. After reviewing what was chosen for std::filesystem in C++17, I think 3. is the right think to do: - std::filesystem::path::u8string does not convert strings on Posix. If a conversion is done (on Windows) the output is fully conformant UTF-8. - the roundtrip is not guaranteed in std::filesystem::path conversions (implementation defined) but the roundtrip of a converted paths is guaranteed to work which is the case with Boost.Nowide: once in fully conformant UTF-8, narrow > wide > narrow works fine. - the use of the replacement character U+FFFD is what is done by the NT kernel (question: do you have to reimplement RtlUTF8ToUnicodeN and RtlUnicodeToUTF8N, why not using them)? It should been made clear in the doc that this choice does not guarantees the roundtrip on Windows so that ill-formed filename may not be opened. * Documentation: - the main part of the doc (the non-reference part) should be more detailed - it should be clear to the reader that Windows and Posix platforms are not handled the same; on Windows, we get UTF-8 strings while on Posix platforms, the user should not expect UTF-8 encoding, just narrow strings, even if UTF-8 is today the most common encoding (addressed in Q/A but is probably too far away). It must be clear from the beginning that on Posix platforms, the library does nothing (including no checking of UTF-8 conformance), just forward to the standard library. - review the use of UTF-8 so that it do not let think that narrow strings are always UTF-8 as this is not necessary the case on Posix platform; maybe just use narrow string? - clarify what is converted to what and when: for each function, we need to know precisely what is the output for all possible input; in particular, better explain for each function what the library does with ill-formed input (either UTF-16 input or narrow input), in particular for args and getenv; say clearly what happens (failbit, error, exception, invalid character…) - clarify what is done for Windows and what is (not) done for Posix - explain better what is behind nowide::cout, cin, cerr, what is done? - clarify what does the filesystem integration on Windows vs Posix - check if basic_stackstring::swap needs to swap buffer_ when both strings are on the stack - The doc/ directory is missing a Jamfile, and there's also no meta/libraries.json. * Implementation: - The global BOOST_USE_WINDOWS_H macro should probably be respected. When windows.h is not used, the WinAPI prototypes are declared in the global namespace. This can cause issues when windows.h is also included before/after the Nowide headers, and the prototypes don't match exactly (e.g. short vs. wchar_t). Also it pollutes the global namespace. Nowide should do what other Boost libraries do, and declare the prototypes in a private detail namespace instead. Example in <boost/thread/win32/thread_primitives.hpp>. Need to add a namespace around your definitions. - MinGW gcc: fix warnings in c++03 mode and errors in c++11 and above - Cygwin gcc/clang (errors because of missing ::getenv and friends) - when this is fixed, an error remains: in file included from test\test_system.cpp:11:0: ..\../boost/nowide/cenv.hpp: In function 'int boost::nowide::putenv(char*)': ..\../boost/nowide/cenv.hpp:104:41: warning: comparison between pointer and zero character constant [-Wpointer-compare] while(*key_end!='=' && key_end!='\0') - Under MSYS, one test failed Fail: Error boost::nowide::cin.putback(c) in test_iostream.cpp:17 main - filesystem integration should return somehow the previous locale for potential undoing * Misc: - There are no .travis.yml and appveyor.yml files in the root. - a number of copy/paste references to Boost.System (test/Jamfile) and Boost.Locale (index.html), should be corrected This are just suggestions and are not mandatory for acceptance: - implement stat/opendir/readir - integration with boost::interprocess::file_lock - integration with boost::program_options (in particular for correct line breaks when showing options) Frédéric

3 4

Re: [boost] [review] Review of Nowide (Unicode)
by Groke, Paul 23 Jun '17

23 Jun '17

Hi, this is my official review of the Nowide library. I haven't had much time to look at Nowide, but I have noticed some things that might be of interest, so here we go... > - What is your evaluation of the design? Overall the design looks good. Some things I'd suggest to reconsider: * I'm a big fan of "try" names, so I'd suggest to rename the array versions of widen/narrow to try_widen/try_narrow. Makes it more clear that they can fail without throwing. Also I'm not sure about the benefit of returning the output buffer pointer or NULL. I'd simply return bool instead. * nowide_filesystem... to be honest I'm not sure what this does. If it does something related to UTF-8 on POSIX systems, then I smell a potential problem. If it's only for wide->narrow on Windows, it should be OK. * Also not sure what boost::nowide::cin/cout/cerr do. Do they call SetConsoleCP/SetConsoleOutputCP? Will they work with re-directed input/output streams? What will happen if the user uses them AND std::cin/cout/cerr or stdin/stdout/stderr in the same application? * I'm not a big fan of the basic_stackstring classes. I'd suggest to provide template function overloads for narrow/widen instead. Like: users_favorite::wstring_class wstr; if (boost::nowide::try_widen(wstr, str)) use(wstr); else too_bad(); They could write to the target-string using the usual duck-typing - e.g. via clear/push_back or clear/back_inserter. That way the user can use whatever SSO string class he likes and Nowide wouldn't have to provide string classes -- which IMO isn't its job. As a bonus, the user also gets an exception-free (except bad_alloc) function that doesn't require him to provide a raw array output buffer. * The Windows & invalid UTF-16 thing... I still think invalid UTF-16 should round-trip. > - What is your evaluation of the implementation? Haven't had much time to check the implementation. What I did notice is that when windows.h is not used, the WinAPI prototypes are declared in the global namespace. This can cause issues when windows.h is also included before/after the Nowide headers, and the prototypes don't match exactly (e.g. short vs. wchar_t). Also it pollutes the global namespace. I think Nowide should do what other Boost libraries do, and declare the prototypes in a private detail namespace instead. > - What is your evaluation of the documentation? Incomplete. Too few examples and the reference documentation leaves many things unclear. IMO the reference documentation should unequivocally define the contract of each API. Also I think the documentation should make it very clear what Nowide does and doesn't do on different platforms. Especially that it doesn't do anything Unicode-related on POSIX platforms and that using it does NOT mean the application will only have to deal with UTF-8 strings on POSIX platforms. Also parts of the documentation that suggest or even _state_ that systems like Linux internally use UTF-8 should be fixed. > - What is your evaluation of the potential usefulness of the library? I'd say it's useful. > - Did you try to use the library? With what compiler? Did you have any > problems? No. > - How much effort did you put into your evaluation? A glance? A quick > reading? In-depth study? Quick reading of the documentation + some very quick glances at parts of the implementation. > - Are you knowledgeable about the problem domain? More or less. I have never used or written a "nowide" layer myself, but I have spent plenty of time "widening" Windows-only applications. > And most importantly: > - Do you think the library should be accepted as a Boost library? Be sure to > say this explicitly so that your other comments don't obscure your overall > opinion. I certainly wouldn't vote against it. I didn't spend enough time to understand every aspect of it though, so I don't feel qualified to vote for it either. I would however suggest to require a major update of the documentation before accepting it. Regards, Paul Groke

4 12

Re: [boost] [cmake] Minimum viable cmakeification for Boost
by Thomas Heller 23 Jun '17

23 Jun '17

Am 20.06.2017 5:08 nachm. schrieb "Niall Douglas via Boost" < boost(a)lists.boost.org>: >> Niall says that using the global BUILD_SHARED_LIBS is a cmake2ism, and >> that explicit targets are preferred nowadays. > > Niall, can you point me to a reference stating this? Preferably some cmake > documentation telling me what the right(tm) thing to do is. Any cmake construct which requires you to write an if() in a non-root-level CMakeLists is a cmake2-ism. There are a very few places in cmake3 remaining where no good alternative to global variables exist. BUILD_SHARED_LIBS is definitely not one of those. I'll ask again: do you have any reference to those statements? The official cmake docs seem to disagree about this specific case. I would even call it non sensical alltogether. Why should the root cmake be responsible for configuring a specific sub module. This is asking for cluttering everything into one single entity in a (almost) totally unrelated place. Why not keep it local to where it should be used. One concrete example: I have networking layer which supports different transports, say tcp, libfabric, MPI and infiniband verbs. The user of my library doesn't really care about which one is used, so ideally, I would give him a compiled version of my library with the internal settings I believe are best for him. In that case, my network layer module would be configured with one of those options. Naturally, I would think, those options are defined within this very sub project. The root project really doesn't care, nor do I want that this implementation detail leaks into the users CMakeLists.txt. Would you consider that bad practice to begin with? How would you solve this? There are tons of other use cases very similar to that. Having everything clobbering up in the root Cmakelists.txt doesn't sound appealing to me. Shared libraries usually have different settings to static libs and again to header only libs. They have different relationships to their dependencies, different usage requirements for consumers. You can use generator expressions to encode those differences, but then you've hard coded them so external cmake no longer can easily override them. You have just made your non-root CMakeLists hard to reuse by cmake you didn't write nor design. Generator expressions are also hard to write and debug, are overwhelmingly confusing to read, and randomly don't work in some places depending on which build system generator you are using. So best avoid all of that complexity - place no complexity at all outside the rootlevel CMakeLists. Declaration only. Place all custom logic solely in rootlevel CMakeLists only. > Any definite source on what a cmake2ism is or is not would be highly > appreciated. > > > As a general observation, I see a lot of statements along the lines of "I > state that XYZ is preferable over UVW", it would be nice to have to have > background information (pros and cons anyone? what do the cmake > authors/docs have to say about this?) on those statements so that everyone > can form their own opinion instead of having to choose whom to trust about > what's "standard" cmake. I believe Stephen Kelly is negotiating a book deal on this. It'll be some time before that book lands though. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/ _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

13 42

[build] Use address-model in library filenames
by Vinnie Falco 23 Jun '17

23 Jun '17

Please add the address-model (32 or 64) into the encoded name which describes a library, so that 32-bit and 64-bit libraries may reside in the same dir. I'd like to avoid a lengthy debate, and if no one wants to do it just provide me with working code for accomplishing it and I'll submit a pull request. -- Follow me on Github: https://github.com/vinniefalco

5 4

Outcome/expected/etc/etc/etc
by Gottlob Frege 22 Jun '17

22 Jun '17

I've now read ALL the emails. Here's some thoughts, in no particular order. - I wonder, Niall, if your uses of the empty state could/should be compared to how people sometimes use -1 or 0, etc - ie "special values as errors". I typically don't like that pattern (yet I've done it anyhow - performance/laziness/etc - but typically only in localized cases, not in "APIs"). So I'm wary about this empty state thing. If we do have an empty state, I like it as a separate type - code needs to deal with it separately. I like the idea of taking a possibly-empty type, checking for empty, and creating a never-empty type, and passing that along so future code need not worry. - I think the narrow and wide contracts should be in the same type, not separate types. This is C++. I suggest "access" if you want long names. ie T & val = oc.access_value(); // possible UB (I almost also suggested "deref" but "deref_value" is wrong - you are deref'ing the outcome, not the value) - I agree with Vicente's concern that functions that behave differently need to be named differently. I haven't looked at the new (or old) API carefully enough, but as a (possibly hypothetical) example: wide_type::value() // returns value, or throw if no value narrow_type::value() // returns value, or UB if no value These functions are not the same, and should not be named the same. Since you thus need different names anyhow (ie value and access_value), why not put them both on the same API? I'm not yet sure if you end up with similar examples with possibly_empty_type::something() and never_empty_type::something(). - I see value in the idea that the narrow contract takes more typing, but I actually *like* using * and -> for the narrow contract because I have 30+ years of training in looking at * and -> and recognizing it as "this may be unsafe, examine the code to see how it is guaranteed". Boost.Optional specifically chose * for that very reason. (on the other hand, I have 0 training on spotting access_value(), but on the third hand, it is easy to grep for) - exception() returns error, error() doesn't return exception, etc: would it be better/easier/clearer if we introduced another name, such as "failure" or whatever? ie exception() only returns the stored exception, error() only returns the stored error(), and then has_failure() would return true in either case, failure_exception() returns exception, generates an exception from error if necessary, etc? - I dislike "exceptions are for exceptional circumstances". Yes it is a common thing, but it is usually wrong. Using Outcome to enable the pattern of "handle errors, but totally cancel the operation on exceptions" might actually turn out to be a useful pattern, but I think this is actually slightly different from the "exceptions are for exceptional cases" camp, as they typically use that chant to choose whether their API will use exceptions or errors, not both in a single API. So I agree with Emil - for most APIs, error-codes vs exceptions" is basically trying to handle the same thing - failure (or "disappointment"). Using both error-codes and exceptions in a single API is somewhat new. And interesting. (There is lots of code that almost-unknowingly uses error-codes and exceptions in the same API, but that is because they don't realize push_back throws or they pretend it doesn't, etc - it isn't really an API design choice) - this ring buffer thing sounds interesting. I haven't looked at it at all, so maybe this makes no sense: should it use thread-local? - I think std::expected should always throw something deriving from std::exception. So if E derives from std::exception, we can throw it. If E is exception_ptr we can (re)throw it. If E is error_code/system_error/etc we can figure out what to throw. If E is user-type, then we wrap it in bad_expected or whatever. (And/or allow user E-to-exception customization somehow). This sounds complicated, but it is easily summarized/taught as "the standard always throws std::exceptions" - I HATE the dual interface of std::filesystem. Worse, there is a chance it is setting a precedent for future std APIs (dual APIs have been proposed elsewhere). Let's please kill it before a dual API strikes again. However, the other problem with trying to "single-fy" the std::filesystem (or any other) API is that users who want error codes sometimes don't want the cost of the extra data (paths, etc). The only way to handle that may be with customizable callback functions that capture the error-info however you like, or something similar to callbacks. - I really look forward to Peter's never-empty variant - and have been hoping for one ever since std::variant was finalized. Not because I dislike std::variant, but just because I think the option should be there for those who need/want it. Regardless, std::variant, in my world, is never empty, since it is only empty when a move operator throws, which should be never and/or when I can't allocate 64 bytes, in which case I'm screwed anyhow. Some days I'm like "man just accept empty, it would be a simple API", but then I think "it is stupid for variant<int, double> to be empty". Other days I think "just double buffer when necessary" (I assume that's your direction Peter?), but then I think "I don't want double-buffering variant<list, vector> to go on/off based on whether I use MS std vs libc++ etc" and also "I don't want double buffering for cases that only happen in theory, not in practice". I like the trade-offs of std::variant. I do agree that Boost should have a variant2 with different choices than std::variant - I think that actually helps the standard - we can choose a direction and instead of telling people "tough luck" when they disagree, we can say "use boost variant2 when you want those properties". - Andrzej: > Actually, now I am thinking not about Boost.Outcome but specifying these > preconditions in the Standard. If you want to std::terminate upon some > condition rather than let the program do random things, you could say. > > *Requires:* `!r` with rescue action `std::terminate()`. This sounds a bit like the contracts proposal - where you get to define what happens when contracts are broken. I'm cautiously hopeful that contracts could help with LEWG's constant struggles with wide vs narrow. - I agree with those who were saying (or thinking) that Boost was - and should again be - the place where APIs go before going into the standard. APIs (mostly) shouldn't bypass boost and go to the standard directly. Or at least an API needs a similar user-base (both in scale and diversity - ie an internal google API has scale, but not necessarily diversity). Since I know a number of LEWG members agree with me on this, don't be surprised to see some push-back from the committee for proposals without that kind of backing, and also don't be surprised if Boost gets flooded with "the committee told me to come here first". Of course I don't speak for LEWG, so maybe none of that happens, we haven't had that discussion yet. I actually probably had more to say - I should have made notes as I was reading all... those... emails. Tony

10 75

[review] Review of Nowide (Unicode) starts today
by Frédéric Bron 22 Jun '17

22 Jun '17

Hi Everyone, The formal review of Artyom Beilis' Nowide library starts today and will last until Wed. 21st of June. Your participation is encouraged, as the proposed library is uncoupled, focused and rather small. Nowadays, everyone needs to handle Unicode but this is very difficult using only the standard library in a platform independant way. Nowide offers a very simple way to handle Unicode the same way on Windows/MacOS/Linux. Key features: * work with UTF-8 in your code, Nowide converts to OS encoding * Easy to use functions for converting UTF-8 to/from UTF-16 * A class to fixing argc, argc and env main parameters to use UTF-8 * UTF-8 aware functions: - stdio.h functions (fopen, freopen, remove, rename) - stdlib.h functions (system, getenv, setenv, unsetenv, putenv) - fstream (filebuf, fstream/ofstream/ifstream) - iostream (cout, cerr, clog, cin) Documentation: http://cppcms.com/files/nowide/html GitHub: https://github.com/artyom-beilis/nowide git clone https://github.com/artyom-beilis/nowide.git Latest tarballs: - to be unzipped in boost source: https://github.com/artyom-beilis/nowide/archive/master.zip - as a standalone library: http://cppcms.com/files/nowide/nowide_standalone.zip Nowide has standard boost layout. So, just put it into the boost source tree under libs/nowide directory for building and running tests. Or alternatively - for header only part of the library add the nowide/include path to the include path of your projects. Please post your comments and review to the boost mailing list (preferably), or privately to the Review Manager (to me ;-). Here are some questions you might want to answer in your review: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With what compiler? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? And most importantly: - Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. For more information about Boost Formal Review Process, see: http://www.boost.org/community/reviews.html Thank you very much for your time and efforts. Frédéric

11 70