[optional] Feedback on the range interface for Boost.Optional

Hi Everyone,
I have just recorded the following message in the Mailing List.
sob., 14 wrz 2024 o 02:44 Alan de Freitas via Boost
We are in the process of implementing the range interface for Boost.Optional, as proposed in P3168R2 https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p3168r2.html and to the extent possible in C++11. I would like to hear opinions from the people in this list, on if and why this would be a bad idea. Regards, &rzej;

Every feature should be well motivated by a real use case or a set of use cases. I don't find any of the examples in this proposal motivating. In fact, this feature seems to encourage bad code, judging by the examples: - using transform(f) | join with an optional-returning function f, instead of using a simple filter(predicate) - materializing optionals when not really necessary (incurs extra moves/copies) - abusing the range-for loop for unwrapping optionals Best regards, Thomas

The core motivation is not having yet another optional-like in the standard
library. Ranges needs a 0-or-1 range as a primitive for where the filter
algorithm isn't quite the right answer, in particular for range
comprehension desugaring which comes up fairly often by hand.
Having a range type like empty or single, without a bunch of the implicit
conversion operations from the underlying type, was my preference, however
adding the range ops to optional<T> was a reasonable fallback.
And optional not being a container was always a suspect position,
especially after the addition of other fixed size or limited size
containers. Making it a full Cpp17 Container is probably over the top, and
might have confused existing code, but adding the range interface is
minimal.
On the other hand, once we have std::optional

Hi Steve,
Thanks for weighing in.
niedz., 15 wrz 2024 o 16:42 Steve Downey via Boost
The core motivation is not having yet another optional-like in the standard library.
Understood. However, there is a trade-off here: additional optional-like type versus additional member functions in std::optional.
I am not well familiar with ranges, so I must admit that upon reading the phrase "range comprehension desugaring" I bailed out. Is there a place that describes the term? However, I understood that in order to effectively compose ranges one needs the additional component, which can either be implemented as a stand alone ranges component or through adding member functions to `optional`.
Indeed, I would also think that if this is only needed to compose ranges, it should be added as a <ranges>-component: view, adapter. And optional not being a container was always a suspect position,
In retrospect, I think I agree with Tony Van Eerd's observation that it was a design mistake to combine two models in `boost::optional`, and by extension in `std::optional`: 1. That optional<T> is a T with an additional state. hence the conversions and comparisons with `T` and `none`/`nullopt`. 2. That optional<T> is a container holding zero or one element of `T` hence the interface for querying for being empty and for accessing the element. But now that we have this confusion, I am not sure if extending one of these interfaces further is the right way to go. I can think of a couple of reasons why adding a range interface to optional is not desired. 1. A general design principle that a class should have a minimal interface required to manage its invariant. Anything else, if needed, should be free functions operating on the class interface. The Standard Library components do not necessarily follow it (std::string being the best (worst) example). 2. Given that we have standard concepts that detect the range interface, `std::ranges::range` it is reasonable to assume that programmers use it in their code, also for controlling the overload resolution. Suddenly adding the range interface to optional is likely to break their code. (While any change whatsoever could theoretically break code, the current problem is more likely, because we are talking about the standardized concept.) 3. Increasing (thus complicating) the class interface without motivating usages is a bad trade-off. The usages for the interop with ranges were given, but it looks like there are no other usages not involving ranges. The usage with the for-loop indeed seems like a hack rather than a technique that I would comfortably endorse. Regards, &rzej;

On 9/15/24 20:55, Andrzej Krzemienski via Boost wrote:
The usages for the interop with ranges were given, but it looks like there are no other usages not involving ranges.
I'm not familiar with ranges, but is there an integration layer that would allow std::/boost::optional plug into the ranges infrastructure without modification? Some sort of traits that could be specialized or ADL-found free functions? For example, we could provide begin()/end() overloads for boost::optional.

Making ADL findable begin/end is sufficient to make something a range for std::ranges::range purposes, but I'm not sure what that saves you over adding them to the member interface? The implementation is slightly cleaner then? Note that you do not have to add a base class or mixin to optional, just the begin/end const and non-const functions. On Sun, Sep 15, 2024 at 3:25 PM Andrey Semashev via Boost < boost@lists.boost.org> wrote:

вс, 15 сент. 2024 г. в 20:55, Andrzej Krzemienski via Boost
I am now realizing that this is in fact what will happen with Boost.JSON's support for optionals. Luckily, the fix is trivial. Although it will lead to a behaviour difference observable by users, so may in turn lead to breakages (though, that's not very likely). I guess, I should make that change before standard libraries start shipping Range interface for std::optional.

For optional, it should already have a hash implementation, and that ought
to be chosen first? How is boost.JSON modelling optional types, other than
range-like now, and is this boost optional specific, or will std:: optional
have issues?
I haven't looked really closely at boost JSON in a while. It looked much
better than rapidJson and Nlohmann JSON, but very similar to one being
maintained and supported internally, so I have not used it anger.
New evidence can be a reason to revisit a proposal that had consensus.
On Mon, Sep 16, 2024, 07:57 Peter Dimov via Boost

Steve Downey wrote:
What Boost.JSON does now is documented in the table here https://www.boost.org/doc/libs/1_86_0/libs/json/doc/html/json/conversion.htm... and it currently checks is_sequence_like before is_optional_like, so a type that matches both would be treated as a sequence.

On Mon, 16 Sept 2024 at 12:57, Peter Dimov via Boost
This is an extremely important point. This change will break real-world code. I have many cases where I have an overload that is differentiated by using the std::xxx_range concept. If I understand correctly, having optional model range will potentially alter overload resolution. This most likely will lead to compilation errors, but it seems entirely possible to have silent runtime errors, making the change unforgivable, in my opinion. The evidence, already, is that library maintainers are having to check to see if this will break their code. This to me, demonstrates that there is acceptance by many others that this has the potential to break existing code. The design alternative to have a non-member function adapter seems safer, more extensible and more re-usable. I very rarely want an optional to behave like a range. I find it hard to imagine this is the typical semantic most use-cases have in mind for optional. Making "optional" model any of the xxx_range concepts directly is an extremely bad idea because it will break code for very little upside compared to having an explicit call to adapt. Regards, Neil Groves

On Sun, Sep 15, 2024 at 1:55 PM Andrzej Krzemienski
Comprehensions are the expressions in other languages like [ (x,y) | x <- [1, 2, 3], y <- [4, 5, 6], is_even(x+y)] based on set comprehension notation. Python has it as [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y] The boolean constraint becomes an expression that is like if(cond) return [val] else return [], either an empty list or a list of one element that is the values under examination. We don't have that in C++, but it's an important technique in writing range pipelines. That's how you turn [(x, y, z) | z <- [1...], x <- [1...z+1], y <- [x ... z+1], (x*x + y*y) == z*z] into something like auto triples = and_then(iota(1), [](int z) { return and_then(iota(1, z + 1), [=](int x) { return and_then(iota(x, z + 1), [=](int y) { return yield_if(x * x + y * y == z * z, std::make_tuple(x, y, z)); }); }); }); Filter can be expressed as joining over yield_if, but not all expressions using the idiom go the other way, at least not trivially.
wg21.link/P1255R10 The criticism was "do we really need another optional?" The main difference being that it did not allow direct assignment from the underlying, because that's what introduces ambiguity, particularly for references. And optional not being a container was always a suspect position,
And making optional model a fancy pointer just made additional issues. . But now that we have this confusion, I am not sure if extending one of

On Mon, Sep 16, 2024 at 8:32 AM Steve Downey via Boost < boost@lists.boost.org> wrote:
...
For what it is worth, I rather like p3168r2 over the alternative. The design of ranges seems such that it anticipates an open set of library and user-defined adaptors, and long pipelines containing many small transformations to achieve a desired result. The `views::maybe` and `views::nullable` adaptors look very much like in the spirit of the ranges design: small, focused algorithms to achieve a specific well-defined purpose. p3168r2 claims that this: // Compute eye colors of 'people'. vector<string> eye_colors = people | views::transform(&Person::eye_color) | views::transform(views::nullable) | views::join | ranges::to<set>() | ranges::to<vector>(); is worse than this: // Compute eye colors of 'people'. vector<string> eye_colors = people | views::transform(&Person::eye_color) // no extra wrapping necessary | views::join | ranges::to<set>() | ranges::to<vector>(); I prefer the first one, since the transformation is explicit. p3168r2 suggests that the new syntax is "dramatically more straightforward" which seems exaggerated to me. It is in fact less straightforward, as now the reader is expected to know about std::optional's expanded role as a range. I'm all for simplicity, but when you choose C++ you are opting in to a certain minimum level of complexity. views::maybe and views::nullable seem like just the right level of complexity for C++. A rangeful std::optional does not do much for me. Thanks

czw., 3 paź 2024 o 00:51 Vinnie Falco
Ok, I now have a paper to revert the range interface from Optional: https://isocpp.org/files/papers/D3415R0.html Regards, &rzej;

On Sat, Oct 5, 2024, 10:42 Andrzej Krzemienski
I think when you say
Whether [P2988R7]
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2988r7.pdf should
be added instead, we have no strong opinion.
You mean P1255, view maybe? P2988 is optional

On Sat, Sep 14, 2024 at 4:36 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p3168r2.html
This perfectly demonstrates why everything added to the standard creates a perpetual additional recurring cost. Thanks
participants (9)
-
Andrey Semashev
-
Andrzej Krzemienski
-
Neil Groves
-
Peter Dimov
-
Ruben Perez
-
Steve Downey
-
Thomas Fowlery
-
Vinnie Falco
-
Дмитрий Архипов