On Mon, Dec 16, 2024 at 12:02 PM Andrzej Krzemienski wrote:

pon., 16 gru 2024 o 11:36 Ivan Matek via Boost napisał(a):

...

Hi everybody,

I presume somebody thought of this before but for some reason it never happened, but just in case it was never discussed I wonder if people here believe adding an nicer version of find(with different name obviously) to associative containers in boost would be beneficial?

Since boost::optional allows references this would work.

It does, but conceptually optional as a return type is not much different than T* return type.

Well I do not think so, as optional avoids using raw pointers(I understand some do not care about this fact, but I think many people do), and as mentioned before people like to chain https://devblogs.microsoft.com/cppblog/cpp23s-optional-and-expected/ operations on optional. I could type here few more sentences, but I think Barry did it better than I ever could. https://brevzin.github.io/c++/2021/12/13/optional-ref-ptr/

*All the operations in the orange circle are highly irrelevant to this problem and would be completely wrong to use. They are bugs waiting to happen.*

...

boost::optional did not get all the updates C++23 std::optional got, e.g. transform that make this usage compose nicely, but those could be added to boost::optional.

Note that boost::optional has had the "monadic interface" for a while. It just uses a different name (map) as at the time we were adding it, a monadic interface in std::optional was not a thing.

my bad, sorry, I just googled and ended up here, did not find it: https://www.boost.org/doc/libs/1_87_0/libs/optional/doc/html/optional/refere...

On Mon, Dec 16, 2024 at 12:41 PM Andrey Semashev via Boost < boost@lists.boost.org> wrote:

Another downside is that as soon as you choose boost::optional as a return value, proponents of std types everywhere will complain that the new interface doesn't compose well in their code bases. You couldn't choose std::optional as it doesn't support references (yet?), but if it did at some point, it would mean requiring a very recent C++ version, which will be a regression in compatibility for the existing Boost libraries. It would also exacerbate the boost::optional vs. std::optional argument.

What is the point of vocabulary types like optional, variant and span if you cannot use them in library interfaces? Boost is "not std" and std is "too new", so is the solution to just give up? IMO Boost libraries should just use Boost vocabulary types, and those should have some convenience functions to interoperate with std types.

пн, 16 дек. 2024 г. в 14:57, Janko Dedic via Boost :

What is the point of vocabulary types like optional, variant and span if you cannot use them in library interfaces? Boost is "not std" and std is "too new", so is the solution to just give up? IMO Boost libraries should just use Boost vocabulary types, and those should have some convenience functions to interoperate with std types.

Boost library maintainers have to consider the cost of adding dependencies, even if those dependencies are other Boost libraries. That being said, system::error_code is interoperable with std::error_code, so I got that benefit too.

On Mon, Dec 16, 2024 at 12:41 PM Andrey Semashev via Boost < boost@lists.boost.org> wrote:

On 12/16/24 13:35, Ivan Matek via Boost wrote:

...

Hi everybody,

I presume somebody thought of this before but for some reason it never happened, but just in case it was never discussed I wonder if people here believe adding an nicer version of find(with different name obviously) to associative containers in boost would be beneficial?

Since boost::optional allows references this would work.

I presume naming is what most people will have strong feeling about. I personally prefer ofind/oat since I love short names, Barry suggested different names in his blog https://brevzin.github.io/c++/2023/05/23/map-api/, e.g. try_at.

As a general comment on the article, I do use the iterator returned from find(), lower_bound() etc. quite often (e.g. to later pass it to erase() or as a hint to insert()), so I disagree with the suggestion that an iterator return is somehow suboptimal.

Barry never said that iterator find is useless, just that for most usecases it is an overkill since it returns info most users do not need most of the time. So I do not think that doing find and then using it for erase or insert hint is very common. In my code for sure more than 60% of usecases would work fine if I never knew of the iterator.

Adding find() overloads to containers means adding a dependency on boost::optional, which makes the container a more heavy dependency for the users. Since containers are often used as data members in classes, this increase in source code footprint arguably will be fairly visible across user's code base.

This is true, on my machine it is 60k LOC, unclear how many of those are common headers users include on their own very often. Also it may be possible to cut it down with some tricks that some libraries use, but that is a per library decision since some boost OWNERS hate solutions like that and prefer to include everything as specified by the standard while others do prefer using implementation headers https://github.com/boostorg/move/commit/b7a04d320104dff1f0ed335addadbb65e9a0... to speed up compile.

Another downside is that as soon as you choose boost::optional as a return value, proponents of std types everywhere will complain that the new interface doesn't compose well in their code bases. You couldn't choose std::optional as it doesn't support references (yet?), but if it did at some point, it would mean requiring a very recent C++ version, which will be a regression in compatibility for the existing Boost libraries. It would also exacerbate the boost::optional vs. std::optional argument.

Partially true. It will not compose well if they do pass the returned boost::optional outside of the function and entire API of their codebase uses std::optional. But as mentioned often people just want to use returned optional to perform some logic on the result, not necessarily return it. As for std::optional: IDK how to read the progress here https://github.com/cplusplus/papers/issues/1661 but it may or may not be in C++26.

An alternative would be to return a pointer instead of an optional, but eliminates the main benefit of the addition in the first place.

Which is the addition is a more succinct syntax for testing and transforming the found value. Personally, I don't see myself using this syntax very often, if ever, because in my experience the mapped value is rarely as simple as an int, and I almost never want to construct a default value when one is not found in the container. My mental model is that constructing an object has a non-insignificant cost to it, and I should avoid it, if possible. However, I understand that other people may have different experiences, and there the proposed syntax could be more useful.

If you think value_or is too expensive it is fine to not use it. You still can benefit from lack of compare to .end()

I don't think the benefits outweigh the costs in this case. It may be useful to have an external adapter of the same effect though:

template< typename Range, typename... Args > optional< typename Range::const_reference > ofind(Range const& r, Args&&... args) { using result_type = optional< typename Range::const_reference >; auto it = r.find(std::forward< Args >(args)...); return it == r.end() ? result_type{} : result_type(*it); }

Note that I'm intentionally using a variadic form to allow for passing an ordering function to polymorphic find() e.g. in Boost.Intrusive containers.

I have proposed ofind long time ago on this mailing list, but as linear search O(n) algorithm :). Here I would personally not allow Args to be variadic since variadics usually mean a big drop in quality of error messages, but that is personal preference. In general I agree that ofind is best possible solution without containers providing member functions.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Mon, Dec 16, 2024 at 10:46 PM Ivan Matek wrote:

On Mon, Dec 16, 2024 at 6:24 PM Vinnie Falco wrote:

...

Can this be implemented as a free function over the public API of unordered containers? In other words, this signature:

Yes, but it is not like Barry or other people https://www.youtube.com/watch?feature=shared&t=150&v=kye4aD-KvTU wanting member functions do not know this.

It seems there are two choices here: 1. Add a member function to uncountably many existing and future unordered containers by adding a member function 2. Write a single, separate free function template which works for all existing and future unordered containers The member function requires selecting an optional type and including its header, while the free function approach can scale to different optional types (one per free function). Please explain why we should prefer 1 instead of 2. Thanks

On Tue, Dec 17, 2024 at 7:21 AM Andrey Semashev via Boost < boost@lists.boost.org> wrote:

I don't see why it should be limited to unordered containers.

I meant associative, not just unordered. The choices are: 1. Add a member function to uncountably many existing and future associative containers by adding a member function 2. Write a single, separate free function template which works for all existing and future associative containers If we include all associative containers instead of just the unordered ones, the case for a free function is even stronger as the free function will work with more types. And the member function would require considerably more work (since there are even more containers which have to change). Thanks

On Tue, Dec 17, 2024 at 8:00 AM Neil Groves wrote:

It doesn't even need to be limited to containers. I think it generalizes to ranges. Isn't the free function along the lines of auto find_or(Rng&& rng, Key&& key, Default&& default) ? This works for all containers that allow the default to be something other than nullopt, which is very useful in many contexts. Where there are associative containers it can delegate to the optimal container implementation, but it can fall back to the general logic of std::ranges::find. A production version would, I assume, have the projection and other usual range idioms.

Working with ranges is an interesting idea, and comparisons to std::ranges::find might encounter obstacles. I presume that the OP was talking about returning an optional value given the key. This presumes an associative container. I am not too familiar with ranges, is there an "associative range" concept? I do like the default value idea though, thanks for that :) template< typename AssociativeContainer, typename Key, typename DefaultValue = std::nullopt > auto try_find( AssociativeContainer&& c, Key&& k, DefaultValue&& alternative = {} ) -> std::optional< typename AssociativeContainer::mapped_type >; Thanks

On Tue, 17 Dec 2024 at 16:20, Alexander Grund via Boost < boost@lists.boost.org> wrote:

...

Working with ranges is an interesting idea, and comparisons to std::ranges::find might encounter obstacles. I presume that the OP was talking about returning an optional value given the key. This presumes an associative container. Why limit this to associative containers? A generic version can search a std::vector as well as a std::set or boost::unordered_map You just need to dispatch to either container.find or fallback to std::find

I'm just prototyping something to explain my mental model clearly. There is a little more to it than just container.find or std::find because we have std::map where we probably want to return the an appropriate reference to the mapped_type when found (V& or const V&), and the mapped_type of a set (also V, of course) but requires different implementation. In other words, for std::map we would need to return it->second rather than *it upon a successful lookup.

...

I am not too familiar with ranges, is there an "associative range" concept? I do like the default value idea though, thanks for that :)

We can get what we want by using "if constexpr" and "requires" or by defining new concepts. We need to obtain the return value differently for map and set, since the expectation, I assume, is that we return the appropriate reference to the mapped_type, rather than the value_type.

Having the optional return type you get the default for free: find(...).value_or() I'm not too sure about supporting ranges: The hypothetical find function would return an optional reference.

I see no reason to force it to always return an optional reference, or anything optional. That seems to prematurely restrict the implementation. Consider sentinel values for the return, especially containers with non-null pointers, often we can return nullptr to indicate the item is not found. The proposed solution works for this common case, and many others besides, and still supports optional should one want to represent the default value in this manner.

So we have the footgun of getting dangling references already when someones does `find(build_map(), key)` We can avoid that by deleting the overload for rvalues but i guess for ranges we might not be able to easily detect an underlying temporary.

Is there a new lifetime issue given that std::ranges::find already exists and works. Our additional layer can take the default by universal reference and forward as the result? I shall post some code a little later, possibly tomorrow. Then we can debate the alternatives a little more concretely. Neil Groves

On Tue, 17 Dec 2024 at 16:46, Christian Mazakas via Boost < boost@lists.boost.org> wrote:

On Tue, Dec 17, 2024 at 8:00 AM Neil Groves via Boost < boost@lists.boost.org> wrote:

...

The free function is clearly the objectively superior solution. This is fact and is preferred over opinions no matter the authority.

The member function approach is objectively inferior, and has no compensatory advantage.

I really don't think this is true at all.

For one, unordered containers have transparent lookup which I'd imagine might actually complicate things.

My opinion on it is that people really only want optional interfaces in the associative containers and nothing else.

While this is probably true for your specific use-cases, this is false for "people" assuming I count as a person! I have frequently come across production code that benefits from looking up configuration in a map and using a default value if the key is not present. I've needed to do this frequently and I see this implemented in code frequently.

You can feel free to objectively totally prove me wrong here but I think if we just updated the base set of containers, that'd make all of our users happy. Especially because member functions work better with code completion and Intellisense.

When I referred to objectively being superior earlier this was not aimed at making a person right or wrong. I was referring to the objectively superior re-usability, the objectively looser coupling, and the broader applicability. These are objective quality factor advantages. I was implicitly inviting contrary opinions and ideas that would open my mind to the merits of the alternatives. The non-member solution is open and would work non-intrusively with other containers that users may have implemented.

A good rule of thumb in software engineering is to start with the concrete and abstract later.

I have many years of experience where the default value non-member version adds benefits. I've implemented the non-member function and found it useful in large scale production code. It happens to generalize to supporting optional too. While it is wise to avoid excessive abstraction. Sometimes we have prior knowledge about the pattern of problem and established criteria from which to make an informed decision. This is the case with member vs non-member functions, as discussed in Effective C++ Item 23 and elsewhere. The non-member function is looser coupled, and has the well-known, published advantages. We famously prefer non-member functions for well substantiated reasons. Therefore if we have the choice and there isn't a compelling advantage, we would run with the established superior default choice. The burden of proof is on the argument for preferring a member function.

- Christian

I shall complete my prototype, but obviously do not demand that this is the approach that is taken. I simply thought it may help make the one approach I am proposing more concrete and easier to modify and we consider the design alternatives. If the member-function is the better solution it should make this clearer by allowing easy demonstration of the disadvantages. Neil Groves

On Tue, Dec 17, 2024 at 5:00 PM Neil Groves via Boost wrote:

The free function is clearly the objectively superior solution. This is fact and is preferred over opinions no matter the authority.

The member function approach is objectively inferior, and has no compensatory advantage.

There is no "objectively superior" solution here. The "compensatory advantage" of a member function is the caller syntax. Something like `map.get(key)` is easier to write and read, especially in more complex expressions. Compare `map.get(key).value_or(other)` to `value_or(get(map, key), other)`. For some libraries (like Ranges), the free function syntax went from a minor nuisance to a deal breaker, so Ranges overload operator| to compensate. The only "objectively superior" solution here is a language feature (akin to extension functions in C#, Swift, Kotlin etc.), which allows defining non-member functions that are callable with member-like syntax. Since we don't have that ideal solution right now, we have to recognize that there are two options here, both with their pros and cons, and we have to weigh them to make a decision.

On Tue, Dec 17, 2024 at 3:08 PM Vinnie Falco wrote:

On Mon, Dec 16, 2024 at 10:46 PM Ivan Matek wrote:

...

On Mon, Dec 16, 2024 at 6:24 PM Vinnie Falco wrote:

...

Can this be implemented as a free function over the public API of unordered containers? In other words, this signature:

Yes, but it is not like Barry or other people https://www.youtube.com/watch?feature=shared&t=150&v=kye4aD-KvTU wanting member functions do not know this.

It seems there are two choices here:

1. Add a member function to uncountably many existing and future unordered containers by adding a member function

2. Write a single, separate free function template which works for all existing and future unordered containers

The member function requires selecting an optional type and including its header, while the free function approach can scale to different optional types (one per free function).

Please explain why we should prefer 1 instead of 2.

There are two related points here 1. member syntax is better for users 2. existing practice Member Syntax is Better for Users Herb's UFCS P3021 https://open-std.org/JTC1/SC22/WG21/docs/papers/2023/p3021r0.pdf has a list of reasons explaining this in detail. Since some people mentioned UFCS here before just to make clear: I am linking this paper *not* because of UFCS, but because it explains why member syntax is better for users(sections 3.1.1, 3.1.2, 3.1.3). Existing Practice We have std::basic_string::starts_with std::basic_string_view::starts_with although there is(or more precisely there will be) std::ranges::starts_with You may say that having member function 2 times is fine since it duplicated just 2 times, but we also have std::set::contains std::map::contains std::unordered_set::contains std::unordered_map::contains std::multiset::contains std::multimap::contains std::unordered_multiset::contains std::unordered_multimap::contains If you say this is just some modern C++ nonsense: Since C++98 containers had empty member function although it is trivially implementable with free function empty. Here not even talking about "complicated" C++17 std::empty that understands C arrays/std::array/..., talking about simple function that could just take container with size member function. template<typename C> bool empty(const C& c) { return c.size() == 0; } But for decades we have this duplication in member functions since empty is commonly called function. To recap situation is that in C++ we have tension between ease of use of common operations on container vs blowing up it's API size. And as Barry wrote 2 most common operations on map are insertion and lookup. So I believe lookup is definitely important enough to get nicer syntax. few notes regarding suggested free function discussed: 1. Dispatching to member may be confusing. std::ranges::find / std::ranges::contains does not https://stackoverflow.com/a/75687947/ dispatch to members if available 2. try_find seems like a wrong name. try_at makes sense since at has precondition(unless you use exceptions for control flow :) ), while find does not, i.e. it can fail. So it is kind of weird to have a prefix that says try in algorithm that has same "success rate" as find.

On Wed, Dec 18, 2024 at 10:16 AM Vinnie Falco wrote:

On Wed, Dec 18, 2024 at 1:12 AM Ivan Matek wrote:

...

starts_with returns a bool, while your function returns an optional. Not

comparable.

I disagree.

Thanks

On Wed, Dec 18, 2024 at 10:29 AM Alexander Grund via Boost < boost@lists.boost.org> wrote:

C++98 used a member function that was generalized in C++17 to a free function -> Seems like time showed that free functions are better

That is not my reading of history. :) I believe std::empty was added for use in generic functions as "poor man's UFCS". Again this is *not* about UFCS, but please note this part of Herb paper(underscore by me): (new) It continues to lead to workarounds, including in the standard

library, such as overloading operator| (e.g., ranges) or *providing nonmember versions of a limited set of common functions that cannot be made members on built-in types such as arrays (e.g., std::begin).*

I do not believe users writing non template functions are encouraged to replace all their uses of .empty() with std::empty. If you are aware of those suggestion in teaching materials or talks from well known C++ experts please let me know.

...

few notes regarding suggested free function discussed:

1. Dispatching to member may be confusing. std::ranges::find / std::ranges::contains does not https://stackoverflow.com/a/75687947/ dispatch to members if available Why does it matter to a user? The result is the same, although I'm not 100% sure for multi-map/set

Because it affects complexity of operation. And even if we agree this design is better(I changed my mind on this during years and I no longer consider it better) it is often better to have consistent inferior design than inconsistent design where you have inferior and better design "randomly" dispersed among functionality. This may sound trivial to you to remember(e.g. boost::ofind dispatches to member, std::ranges::find does not) , but you must remember that you are not even close to average knowledge of C++.

...

2. try_find seems like a wrong name. try_at makes sense since at has precondition(unless you use exceptions for control flow :) ), while find does not, i.e. it can fail. So it is kind of weird to have a prefix that

says

...

try in algorithm that has same "success rate" as find.

I'd definitely want to have "find" or "lookup" in the name. `map.try_at(key)` makes we wonder what is tried at the key. Maybe `map.get(key)` as it always "gets something" if you want. But that is bike-shedding for after a decision.

I agree it is premature, but I still believe it is important to be consistent, again not for experts, but for average user. try_at already exists in Boost.Json, and it is name suggested by Barry in his blog, for find version afaik Barry did not provide suggestion, and there is no that functionality in Boost at the moment afaik.

On Wed, Dec 18, 2024 at 10:33 AM Andrey Semashev via Boost < boost@lists.boost.org> wrote:

On 12/18/24 12:12, Ivan Matek via Boost wrote: std::list::size() had linear complexity in C++03 and std::list::empty() was constant. It was definitely a reason to have it separate from size().

I am not really convinced that this was motivation. I am not aware of C++98 design discussions that are available online so hard to check. For std::list sure, but why for every other container? There could have easily been std::empty that does std::true_type/std::false_type equivalent of if constexpr using member empty if available, else compares size() to 0. But maybe during C++98 design time there was no use of SFINAE in STL?

contains() is a specialized algorithm that is not equivalent to `std::find() != end()` in each container's case. Historically, such specialized algorithms were implemented as members, while the generic algorithms were provided as free functions. Another such example is swap().

I am aware contains() can be implemented faster, e.g. you could have SIMD logic for integer set where for example 8 integers are grouped together so you can compare them all with lookup value. But I do not believe this is motivation. contains() replaces count()!= 0. P0458 https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0458r2.html is pretty clear about contains() motivation: [...]involves doing a lookup and checking the returned iterator:if

(some_set.find(element) != some_set.end()) { // ...}This idiom suffers from excessive boilerplate code and is inferior to if (some_set.contains(element)) in terms of expressing intent in code.

On Wed, Dec 18, 2024 at 10:12 AM Ivan Matek via Boost wrote:

Member Syntax is Better for Users Herb's UFCS P3021 https://open-std.org/JTC1/SC22/WG21/docs/papers/2023/p3021r0.pdf has a list of reasons explaining this in detail. Since some people mentioned UFCS here before just to make clear: I am linking this paper *not* because of UFCS, but because it explains why member syntax is better for users(sections 3.1.1, 3.1.2, 3.1.3).

Unfortunately, the UFCS proposals have been inherently misguided, because they are proposing something that is harder to implement, and something that nobody actually wants. We don't really need the ability to call every function in two different ways (other than for ADL customization points, but those are also a way to cope with the lack of a language feature anyway). Nobody wants to write `x.max(y)` and `starts_with(string, prefix)`. Each function should use a spelling that makes more sense for it. Many languages are perfectly happy with extension functions without any "universal" syntax. Just my two cents on the topic, to hopefully frame the discussion in the proper way.

On Wed, Dec 18, 2024 at 2:20 PM Janko Dedic wrote:

Just my two cents on the topic, to hopefully frame the discussion in the proper way.

To repeat what I said: I am *not *saying UFCS should be adopted to C++. I just use Herb's proposal for summary of why it is nicer for users to use member functions than free functions. Example you provided of "symmetric" operation like std::max certainly look more natural in free function style. btw Herb's proposal does not make starts_with(string, prefix) work, it is only 1 direction, you can check section 3.2.3 Speaking of Herb's proposal and original discussion of free function vs member function 3.2.2 section also seems relevant(not for your comment, but in general). Q: Wait, isn’t std::begin/end/size a counterexample, evidence fa-

voring non-member function call syntax being UFCS? A: No.

On Thu, Dec 19, 2024 at 7:30 AM Christian Mazakas via Boost < boost@lists.boost.org> wrote:

users.find(user_id).map([](auto& user) { user.status = deactivated; return user; });

try_find( users, user_id ).map( [](auto& user) { user.status = deactivated; return user; }); Thanks

On Thu, Dec 19, 2024 at 8:19 AM Christian Mazakas via Boost < boost@lists.boost.org> wrote:

My point was that users don't want this compared to the method chaining version and I think the code here kind of speaks for itself.

users.try_find().map() is the preferred API over try_find(user).map().

In C++, free functions which operate on a concrete type or a type which matches named requirements are considered to be part of the public API for that type. std::swap is a good example, this is why cppreference.com lists associated free functions under a section titled "Non-member functions:" https://en.cppreference.com/w/cpp/container/vector Good library design means designing for C++. Therefore: Free functions should be considered equally ergonomic to member functions. Good encapsulation means giving an algorithm access to as few implementation details as possible. Therefore: Prefer free functions when an implementation can use only the publicly visible members of a type. Scott Meyers elaborates on this: https://www.drdobbs.com/cpp/how-non-member-functions-improve-encapsu/1844011... When designing a library you have to assume that the user knows C++. If they don't understand the use of free functions, the solution is that they should educate themselves, not to always use member functions. The free function is better than the member function for several reasons: 1. The implementation only needs to be written once 2. Authors of new containers have less to implement 3. The header file for the containers does not change; no need to include <optional> 4. Multiple free functions can exist, each returning their own optional type, permitting std::optional and boost::optional to co-exist harmoniously. 5. Better encapsulation: the free function does not have access to private members 6. A common objection to the use of free functions is that the member function is more discoverable, such as in an IDE. This problem is better solved in the IDE, of which there are few, rather than solving it in the container of which there are uncountably many. That is, if someone was to type "u." in the IDE, then the autocomplete feature could show associated free functions in addition to member functions. And when an associated free function is selected, the IDE will change the member function call to become the free function call syntax. For all the reasons listed above, the free function is preferable to the member function. As this allows the functionality of containers to be extended without imposing additional work on all existing and future container authors. And I do believe that we need better educational resources, so the myth that member functions are somehow more ergonomic can be decisively dismissed. Thanks

Sure, that's just another opinion.

I'm just relaying my experiences having talked with other developers across multiple languages and programming chats.

Many of the developers on this list have also spoken to other developers and have equally valid experiences to draw from.

In reality, I don't think anyone wants `try_find(my_vector)` because you don't normally do try_find on a container like vector.

It's important to follow the principle of avoiding over-engineering and under-delivering.

Many developers on this list have designed one or two software projects. The experienced developers are not learning from the memes that you are repeating. It isn't over-engineering to try and design the solution to avoid unnecessarily requiring the modification of every associative container. If the solution genuinely required it, then fine, but so far this does not appear to be necessary. Thought, rational debate, and consideration of alternatives early in a project are not symptoms of over-engineering. They are a signal of experienced developers that are trying to ensure that the subsequent time invested in the SDLC for the feature has a good return on investment. Since I maintain Boost.Range by exploring the possibilities for non-intrusively adding the feature-set it would make it possible for me to provide the entire solution. If we require intrusive modification we have our own associative containers to change, the std ones would need to change, and then user-supplied ones would also need to be modified. We would need a very considerable advantage to go this route since it inevitably would mean divergence while libraries adopted the new approach at different speeds even if we guaranteed eventual convergence. A non-intrusive solution, if it indeed works as hoped, allows me to provide and ship something that works for existing associative containers, the new ones in C++26, and conforming containers obtained from elsewhere. This is a huge, objective, design advantage. If the non-intrusive approach is limited to containers then I would make a pull request for Boost.Container but the same broad advantage would remain.

The value proposition here is dubious in the first place because it relies on using Boost.Optional in an API and most people aren't huge on using Boost versions of STL components anymore in 2024.

I find at places of work that like money the decision about what to use is based upon what works and provides a good return on investment. The original idea, you posted, can be made immune to being tied to Boost.Optional easily. Each and every premature / unnecessary coupling / intrusive decision brings with it the chance to have a show stopping objection for a user. It's one of the reasons we are trying to keep the essence of the original request / proposal while trying to help shape it in a way that: 1. can be implemented without requiring changes to an unnecessarily large number of libraries, and 2. explore the other similar cases where the current interface is deficient to see if this can stimulate even more improvement.

- Christian

Neil

On Thu, Dec 19, 2024 at 10:33 AM Christian Mazakas via Boost < boost@lists.boost.org> wrote:

...we don't actually know if people are even going to use this kind of construct in the first place and it's just way easier to add try_find to e.g. `boost::unordered_flat_map` and call it a day.

If users buy into the construct and they want the free function version, they'll let us know.

I think this is exactly backwards. Adding a member function to decide if "users buy into the construct" means that if we later observe that users aren't buying in, then we are either stuck with a useless member function or worse we have to break existing code. If we want field experience, it is better to implement it as a separate free function to figure out buy-in and then later consider adding it as a member function. Thanks

czw., 19 gru 2024 o 18:04 Vinnie Falco via Boost napisał(a):

On Thu, Dec 19, 2024 at 8:19 AM Christian Mazakas via Boost < boost@lists.boost.org> wrote:

...

My point was that users don't want this compared to the method chaining version and I think the code here kind of speaks for itself.

users.try_find().map() is the preferred API over try_find(user).map().

In C++, free functions which operate on a concrete type or a type which matches named requirements are considered to be part of the public API for that type. std::swap is a good example, this is why cppreference.com lists associated free functions under a section titled "Non-member functions:"

https://en.cppreference.com/w/cpp/container/vector

Good library design means designing for C++. Therefore:

Free functions should be considered equally ergonomic to member functions.

Good encapsulation means giving an algorithm access to as few implementation details as possible. Therefore:

Prefer free functions when an implementation can use only the publicly visible members of a type.

Scott Meyers elaborates on this:

https://www.drdobbs.com/cpp/how-non-member-functions-improve-encapsu/1844011...

When designing a library you have to assume that the user knows C++. If they don't understand the use of free functions, the solution is that they should educate themselves, not to always use member functions. The free function is better than the member function for several reasons:

1. The implementation only needs to be written once 2. Authors of new containers have less to implement 3. The header file for the containers does not change; no need to include <optional> 4. Multiple free functions can exist, each returning their own optional type, permitting std::optional and boost::optional to co-exist harmoniously. 5. Better encapsulation: the free function does not have access to private members 6.

A common objection to the use of free functions is that the member function is more discoverable, such as in an IDE. This problem is better solved in the IDE, of which there are few, rather than solving it in the container of which there are uncountably many. That is, if someone was to type "u." in the IDE, then the autocomplete feature could show associated free functions in addition to member functions. And when an associated free function is selected, the IDE will change the member function call to become the free function call syntax.

For all the reasons listed above, the free function is preferable to the member function. As this allows the functionality of containers to be extended without imposing additional work on all existing and future container authors. And I do believe that we need better educational resources, so the myth that member functions are somehow more ergonomic can be decisively dismissed.

I would like to offer some philosophical remarks here, which are no longer tied to optional or map, so they deserve a separate thread. In C++ we have what we have, but technically we could think about "encapsulation" and "member function notation" as two orthogonal things. This already works one way in C++: you can have friend functions, which have access to private state but force their users to use the free function notation. This doesn't work the other way though. I am personally not sold by the "discoverability" argument, because I mostly code in vim where I do not get any hints from the IDE anyway. I do not care that much about the member function notation, but I do care about the left-to-right notation. This is because I come from the latin culture, where we are taught to read, write, and think left-to-right. So reading "find value, transform the value" comes naturally, and the notation where "find" is left of "transform" becomes more intuitive. This doesn't mean that I like member functions (because they involve encapsulation), or that I like the pipe notation (because of the error messages nightmare). But if there was a left-to-right solution devoid of the above problems, I would take it. Regards, &rzej;

On Thu, Dec 19, 2024 at 11:20 PM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

czw., 19 gru 2024 o 18:04 Vinnie Falco via Boost napisał(a):

In C++ we have what we have, but technically we could think about "encapsulation" and "member function notation" as two orthogonal things.

Can you elaborate? For me without UFCS this is *not* orthogonal. For example my reading of P3021R0 https://open-std.org/JTC1/SC22/WG21/docs/papers/2023/p3021r0.pdf is that std::begin is poor man's UFCS. Maybe I misunderstood you.

I am personally not sold by the "discoverability" argument, because I mostly code in vim where I do not get any hints from the IDE anyway.

I personally do not use exceptions(except one I got from std::/libraries), that does not mean I would oppose improvements for people who use exceptions. Not every improvement needs to benefit 70+% of users. On Fri, Dec 20, 2024 at 12:57 AM Vinnie Falco via Boost < boost@lists.boost.org> wrote:

std::unordered_map< int, char > m;

upgrade(m).try_find( 1 ).map( []( auto&& v ) { std::cout << "not empty"; return v; } );

I do not think people will be happy to use this pattern because it beside it being verbose it requires wrap on every use, would be better if it was a type wrapper that inherits public interface of container e.g. in class you have member boost::fancy> user_id_to_user_; // has find, size, ... + try_at so every use of user_id_to_user_ does not need upgrade wrap call. Issue here is that you can not inherit from containers(or you can, but you shouldn't). This is a well known issue in C++. To go back to adding member function and concern about code duplication. To reduce code duplication you could add CRTP helper base class that provides try_at for every container that implements find and inherits from it. In my opinion using CRTP to avoid 5 lines of duplicated code is not worth it. But I could be wrong. As existing example unordered_flat_set, unordered_flat_map, unordered_node_map, unordered_node_set all have same implementation of count /contains, there may be some value in extracting "nice/utility" members that can be implemented using "core" functionality(in this case find + end ). i.e. maybe this same code in those classes: BOOST_FORCEINLINE bool contains(key_type const& key) const { return this->find(key) != this->end(); } BOOST_FORCEINLINE size_type count(key_type const& key) const { auto pos = table_.find(key); return pos != table_.end() ? 1 : 0; } could be moved to some base helper? In that case try_at can also live there. P.S. regarding godbolt code: I know it is not production code, but I think you do not need std::forward in try_find.

On Fri, Dec 20, 2024 at 9:20 AM Andrzej Krzemienski wrote:

Sorry, I meant to say something like, "in an alternative language, we could have ..". I agree: in C++ they are not orthogonal.

This is why I liked the idea of the pipeline operator. https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2011r1.html

I liked that proposal, but I also knew it has downsides, I presume C++ would be only mainstream language to use it, and it would be another thing almost every developer would need to learn. So not sure if it was mistake or not to reject it. My guess is that C++ would be a better language if it was accepted, unless we get UFCS.

Agreed. I only put it so that you can weigh my opinion accordingly.

I see, thank you for clarifying.

On Sat, Dec 21, 2024 at 2:15 PM Andrzej Krzemienski wrote:

...

Issue here is that you can not inherit from containers(or you can, but you shouldn't). This is a well known issue in C++.

Could you explain what the issue is? I do not believe I have ever heard of it.

https://github.com/isocpp/CppCoreGuidelines/issues/1232

Relevant containers do not have virtual destructor. And AFAIK there is no way to prevent people from putting your type in unique_ptr. IIRC some time ago I read that even if objects are of same size(i.e. derived object has no extra members) according to standard it is UB to delete derived using pointer to base, but I did not check in the standard.

On Thu, Dec 19, 2024 at 9:26 AM Neil Groves via Boost wrote:

I am not arguing that your view is incorrect.

I am in fact arguing that preferring the member function over the free function is incorrect. Free functions are objectively better than member functions. It doesn't matter how users feel, we have to design for the current rules of the language. I also think UFCS is a poor solution to this non-problem. Thanks

On Thu, Dec 19, 2024 at 12:13 PM Ivan Matek wrote:

I am not sure what you mean by current rules of the language. There is no such thing in this case since language allows both.

The language allows for free functions so we have to give them equal weight.

If you are talking about what experts prefer in terms of design... ... Herb and Bjarne

...

Core guidelines ... Barry Revzin ...

The needs and incentives of people who do standardization work are different from people who publish and maintain third party libraries. Changes proposed to the standard library have the luxury of not worrying about third party libraries. That is, when a wg21 apparatchik adds a member function to all associative containers, the amount of work is bounded as the number of containers in the standard library is a small constant. Of course, that approach is as short-sighted as is the assumption that highly active committee participants are experts. I think the impact of a design choice should be measured not only by what it does to the standard but how it affects the entire C++ ecosystem. Thus I restate to my original point. Given that free functions are as equally ergonomic as member functions, the better of: 1. Modify the source code for every current and future associative container to include a new member function, or 2. Write a single free function which works with every current and future associative container should be obvious (it is 2). If you would like to list the specific reasons why you believe the member function better, I could respond to each point. Thanks

On Thu, Dec 19, 2024 at 6:26 PM Neil Groves via Boost wrote:

One aspect that strikes me about all of our alternatives is that none of them work particularly well when we are using multi_ associative containers.

If I understand your comment correctly: "correct" return type is a std::ranges::subrange, equal_range returns std::pair so you need some helper to "rangify" that pair. (this example uses biset so you can ignore .get<1>).

EmployeeSet ems{{1, "Alice"}, {2, "Bob"}, {3, "Chad"}, {4, "Alice"}}; for (const Employee& employee : ems.get<1>().equal_range("Alice") | rangify) { std::print("{} {}\n", employee.id, employee.name); }

Vinnie Falco wrote:

On Thu, Dec 19, 2024 at 7:30 AM Christian Mazakas via Boost < boost@lists.boost.org> wrote:

...

users.find(user_id).map([](auto& user) { user.status = deactivated; return user; });

try_find( users, user_id ).map( [](auto& user) { user.status = deactivated; return user; });

In what namespace and in what header is try_find defined?

On Thu, 19 Dec 2024 at 15:30, Christian Mazakas via Boost < boost@lists.boost.org> wrote:

The case against a free function gets a lot stronger once you remember that optional in C++ still supports unchecked accesses with trivial syntax and that the most ergonomic and safe way to actually use it is via something like this:

users.find(user_id).map([](auto& user) { user.status = deactivated; return user; });

As you know I've tried seeing how far I could get with some work in Boost.Range. I appreciate this isn't the solution you are dreaming of. I have working on a private branch (without new member functions in any associative container): users | mapped_values(user_id) | invoke([](user_type& user) { user.status = deactivated; }); To me, this looks remarkably similar to your example. This works for std::map, std::multimap, std::unordered_map and std::unordered_multimap. I find this more readable than the original. Of course, this is for one very specific example and I lack support for many of the functions that boost::optional has that one may likely wish for when using a container that has at most 1 mapped_value per key. mapped_values(key) is a Boost.Range adaptor similar to the existing "map_values" except that it takes a key parameter and applies equal_range to the source range before extracting the mapped_type. I did also find the map adaptors in Boost.Range to be unnecessarily fussy about const-ness propagation. I had to fix this on my branch to make any substantial progress. I wondered if reading this example line above if it didn't look quite as horrific as perhaps you were imagining? One unfortunate aspect of the original snippet is the boost::optional choice of the word "map" where we are also working with a concrete "map" container. There is plenty of work to do if this is of interest to anyone. I would want to ensure there was no performance overhead, and currently I'm not entirely convinced that my initial implementation that uses the member equal_range is optimal. This is an implementation detail, which I would work through. The value_or ( or front_or ) functionality for non-multi containers can be added in a similar manner, and be broadly usable for range algorithms. Were this a free function, a user wouldn't be able to create this nice

fluent interface that users, by and large, seem to actually like.

For the left-to-right syntax I noticed that there has been a neat looking proposal https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2011r0.html I'm hoping the left-to-right solution will appear by one or more of the active proposals while we make progress coping with the ways we have.

- Christian

If this is of any interest to you I'll look at putting some of this into the develop branch of Boost.Range. If it is not interesting I shall continue on my branch and build a more compelling iteration that I shall land later. I hope this helps, Neil Groves

On Fri, 20 Dec 2024 at 20:04, Peter Dimov via Boost wrote:

Neil Groves wrote:

...

I have working on a private branch (without new member functions in any associative container):

users | mapped_values(user_id) | invoke([](user_type& user) { user.status = deactivated; });

To me, this looks remarkably similar to your example.

That's the best I can do using the standard ranges: https://godbolt.org/z/WMrzszMGP

But it's not the same as the original example, because it doesn't return the found user.

It is true that my example didn't the same type as the original but incorrect to state that it didn't return the found user. It isn't clear that it does but it does by returning the equal_range(key) | map_value. It returns the range it was given by the invoke adapter, and so it does return the user in the range of 1. I started with this because I'm working on a layer of the most general (that work with multi- containers and well as the single value containers) and working toward the specific which I believe I can achieve without problems layering on top of the general solutions and optimizing in places.

There might be a way to say something like `| views::front` that will make the pipeline return .front(), but I don't know what it is.

I proposed something very similar to this in my earlier, far too lengthy, waffle. What I was thinking was that we could have front_or and when empty provide another value. This with some syntactic sugar can produce the value_or / ref_or functionality. I think layering this on the range solution should work nicely, and I believe is able to be done with optimizations that make the 0..1 ranges zero overhead. I have not demonstrated this yet. I intuit that performance ought to be superior using 0..1 ranges vs optional, but until I have benchmarks I can only say it is my intuition. I would never enforce the absence of optional, but I intend to be able to avoid optional for many of the use-cases for correctness and performance. You can say `| views::take(1)` but that's still a range.

`| views::to>` would work if it worked, but I don't think it does.

.. .but it could. I'm happy to make chagnes were necessary to get what we want. I'm not totally sold on the optional idea. I prefer the range to fn idiom for it's elimination of the dereferencing of a nullopt. The return of a default value (which may or may not be the same type as the mapped_type) seems to offer an option-free approach to the common use-case where one wishes to select a default alternative value when it is missing. It also leaves the door open to returning an optional for when that makes sense. I believe I can do all of these things. If people want it. I shall build this. Of course, Boost.Range is not state-of-the-art and there could potentially be work to do if we want simliar things in the standard ranges. I don't see any reason the considerably superior standard ranges cannot do this as well or better. However Peter I know how clever and experienced you are and so I wonder if you have spotted trouble ahead that I am missing? Regards, Neil Groves

On Fri, 20 Dec 2024, 22:58 Christian Mazakas via Boost, < boost@lists.boost.org> wrote:

On Fri, Dec 20, 2024 at 11:37 AM Neil Groves wrote:

...

As you know I've tried seeing how far I could get with some work in Boost.Range. I appreciate this isn't the solution you are dreaming of.

...

...

I wondered if reading this example line above if it didn't look quite as

...

horrific as perhaps you were imagining?

One unfortunate aspect of the original snippet is the boost::optional choice of the word "map" where we are also working with a concrete "map" container.

There is plenty of work to do if this is of interest to anyone. I would want to ensure there was no performance overhead, and currently I'm not entirely convinced that my initial implementation that uses the member equal_range is optimal. This is an implementation detail, which I would work through.

The value_or ( or front_or ) functionality for non-multi containers can be added in a similar manner, and be broadly usable for range algorithms. ... If this is of any interest to you I'll look at putting some of this into the develop branch of Boost.Range. If it is not interesting I shall continue on my branch and build a more compelling iteration that I shall land later.

I hope this helps

Fwiw, I don't mean to be a hater and I'm happy you're not discouraged.

I am discouraged by how long it took me to recognising the signs that no outcome would have been satisfactory other than total unconditional agreement with the original proposition. The find operation often wants the power of the iterator, but then other times a query like "at" that doesn't throw would be useful. I thought the find, tolerating absence, was your motivating use-case.

It's like I said, I don't actually have a strong opinion here either way.

I'm with Andrey, I think the iterator API *is* the useful API. Typically, when I find something in a map, I'll either erase or do something else where reusing the iterator is a key part of everything.

I would not take away the iterator idiom. It has proven to be extremely powerful.

My whole thing was about what users actually wanted vs all the costs of everything else.

In C++, we have basically unlimited power so we can honestly do basically whatever we want.

I think most users just want `try_find() -> optional`, with `std::optional` and not Boost. Boost is okay as a polyfill, but users have told us ad nauseum that in 2024, they prefer STL versions.

This is a Boost mailing list. I assumed you wanted to see what we thought and could achieve and use what was learned. If the users now only want std ( these users seem to change their requirements very frequently ), what was it you wanted from the irrelevant boost developers? By showing changes in Boost.Range I hoped to make Boost.Range more useful and assumed similar changes could be made to std ranges. The header includes for boost range are not huge and so far do not require optional of either flavour. I have shown that by changes to range it can be achieved. The std ranges can do all that I have done in Boost.Range by way of example. I accept there may be issues with the std range header overhead, but consider that something fixable.

Second, I think the burden on users to opt into the perfectly generic version is too much for the little we're actually doing.

The header overhead is hundreds of lines less than your proposal with optional. My bias is to reducing the duplications, fixing the abstractions to be small and orthogonal and fixing the headers to have more and smaller files. The excessive header include overhead is partly due to a failure to decompose the problems. The include of optional adds more overhead than the include of the range map adapter. My implementation shows it can be implemented in just a few lines of code. Less overhead than including optional. And my solution only requires including the range adapter if you use it. Your solution requires every map header to include optional even if the feature is not used. My solution is more general, less intrusive, and has less header cost than the original proposal. Even the target syntax was largely approximated. My self assessment is that this works well for my understanding of the expressed requirements. I was surprised this is not an attractive option but fully accept it isn't what you want.

For a user to now get the behavior they want, they'll have to depend on Boost.Range, include the Range headers and then setup their functional pipeline just to do something that could've been this:

for (auto& user : users_map.try_find(user_id)) { user.status = deactivated; }

Yes for Boost.Range they need to include the relvant Boost.Range adaptor header. A similar adapter could be added to the std. To use optional you would have to include optional. The adapter header is considerably smaller than the optional header but for some reason including a smaller file is unacceptable overhead.

To this end, the user doesn't have to include anything more than ``, which they would already have.

The cool thing about Boost is that we're supposed to innovate and that means trying new things. I'm not saying Range-ifying everything is bad and

It was always a range, the most relevant function is even called equal_range. The map is already a range and the selected subset via a key is naturally a range for single and multi associative containers. What was done was not rangifying everything. It was continuing with ranges avoiding option-ifying the solution. The going in and out of optional is pointless additional overhead most of the time introducing opportunity for error. You were right to choose the optional::map member function in your final example as the safest solution. This good choice made it clear that the optional was neither necessary nor desirable for this specific use-case. I'm happy you're giving serious efforts into making it happen.

I'm just trying to give my opinion about the potential reception of such efforts. It's fine on paper but I'm not sure we have users clamoring for the feature to manifest in this way.

Having produced something that works, builds in less time, almost matching your syntactic ideal, and provides solutions for other contexts I expected users would be more happy with this solution. You clearly have unique access and understanding of the users requesting the original change. If std ranges header is too bloated it could be split in a similar manner to Boost.Range. The include demonstrably need not be as high overhead as optional, when implemented appropriately.

I'd be happy to be proven wrong.

- Christian

I shall park the branch, assess the changing landscape including Range V3 and see if anything useful can be contributed. I regret the noise I have made on the list. I genuinely have little understanding of what you want, fully accepting my part in my lack of comprehension. I will bow out from trying to help with this request. Perhaps another Boost developer will succeed in understanding the issues and context better, yielding a more agreeable result. I hope you get what you want and one day I will understand this all better. The best thing I can do for you at the moment is get out of the way. Regards, Neil Groves

On Sat, 21 Dec 2024 at 12:11, Peter Dimov via Boost wrote:

Christian Mazakas wrote:

...

I'm with Andrey, I think the iterator API *is* the useful API. Typically, when I find something in a map, I'll either erase or do something else where reusing the iterator is a key part of everything.

My whole thing was about what users actually wanted vs all the costs of everything else.

What this user has always wanted is to not have to write this:

auto it = map.find( key ); if( it != map.end() ) { // do something with it->second }

That's why I'm always defining a helper function `lookup` that returns a pointer, which allows me to write this instead:

if( auto p = lookup( map, key ) ) { // do something with *p }

If there's a built-in way to obtain an optional instead of an iterator, I can use that instead of `lookup`, without changing the syntax.

What I tried to do was provide that with map.equal_range(key) | mapped_value | invoke(fn). And then tidy the syntax, but I accept that wasn't an outstanding success! I collapsed the map.equal_range and mapped_value into one step and then we had map | mapped_values(key) | invoke([](...) {}) In the end what I had for you paraphrasing you snippet above would be: map | mapped_values(key) | invoke([&](auto&& p) { // do something with p }; Obviously here I've pulled the "if" into the boilerplate code and eliminated the need for p to be optional. I like eliminating the possibility of a dereference of nullopt and I like not having to include either implementation of optional. If we need to have work in the "else" path then this is less awesome and projection to optional is one approach to tackle this. Where we would like to pick a replacement default value when missing that could probably also use syntactic sugar to replace the invoke. The composition possibilities looked interesting to me, and I thought that the common cases could appear approximately as the expressed desired syntax of other approaches with a little syntactic sugar. I was always open to name changing, so mapped_values could be "lookup" if you wanted. Of course I could go a step further and collapse those two adapters. We also discussed what would be returned, and of course it would be possible to return an optional single element where appropriate. I like the generality of passing the range of values back, especially since that works for multi containers. The optional, to me, looks to be an inferior alternative for an empty or not range, for many of the cases. Perhaps more real world examples would change my mind. Nonetheless I could give the projection to optional no trouble at all. Hence, in the end, I thought I'd pretty much got every issue I'd heard addressed. I realize though that small details to me may be much more important to others. I don't typically confuse my opinion with fact. I'm definitely not young enough to know everything. I liked not having the explicit if, but totally accept that before it becomes familiar it may look odd. It also takes away the possibility of incorrectly dereferencing a nullopt, while apparently providing a very similar solution to the expressed problem. One of the posted suggestions used the optional map function to get the same advantage. Thus I went on this journey to see if we could just keep the ranges we already had and get the syntax close to the original expressions. It looked pretty close to me, but it didn't appeal, which is totally fine. It would be boring if we all agreed all the time. I thus don't have anything concrete to merge that addresses the original poster's request, which is disappointing but totally okay from my perspective.

Although it strikes me that it should probably be named `try_at` instead

Totally with you on "try_at" versus "try_find", but if we use 0..1 ranges it is just equal_range, that's mapped_value adapted.

of `try_find`. `find` is already a try-ing member function, because it doesn't fail, whereas `at` both returns the correct reference type, and can fail.

Of course `try_at` by convention ought to return boost::system::result or std::expected instead of an optional, but that's another story.

Interestingly, and maybe relevant for this discussion, `at` was added to associative containers even though it could have been made a free function instead.

I've looked at the recent standard changes and to me it looks like we are semi-randomly preferring member and non-member functions in an inconsistnent and arbitrary manner. We also added std::erase_if as a non-member for std::vector and other containers. I would estimate that you are probably aware of many more of these changes than I am. They've gone in both directions. To me it seems we can't use recent history to inform us. I accept that it may appear arbitrary and inconsistent because I have failed to comprehend the rules that drive the final decisions. Apologies if by asserting that I've achieved nothing this seems to be an emotional reaction. It isn't. While I'm disappointed I couldn't help with the original problem. I consider this eventuality to be part of the acceptable, expected set of outcomes from the effort. If it weren't then I'm not taking enough risk. My perception is that my suggestion doesn't have broad appeal and hence I'm not pushing to make it happen. Other solutions can be explored, or we can leave it as it is. I never intended my experimentation to block progress for other approaches. I also never intended to make any demands about how anyone approached the problem. I'm happy to continue to help, if I am actually doing so, but equally happy to get out of the way and let others come up with something else. Regards, Neil Groves

sob., 21 gru 2024 o 13:54 Neil Groves via Boost napisał(a):

On Sat, 21 Dec 2024 at 12:11, Peter Dimov via Boost

...

wrote:

...

Christian Mazakas wrote:

...

I'm with Andrey, I think the iterator API *is* the useful API. Typically, when I find something in a map, I'll either erase or do something else where reusing the iterator is a key part of everything.

My whole thing was about what users actually wanted vs all the costs of everything else.

What this user has always wanted is to not have to write this:

...

auto it = map.find( key ); if( it != map.end() ) { // do something with it->second }

That's why I'm always defining a helper function `lookup` that returns a pointer, which allows me to write this instead:

if( auto p = lookup( map, key ) ) { // do something with *p }

If there's a built-in way to obtain an optional instead of an iterator, I can use that instead of `lookup`, without changing the syntax.

What I tried to do was provide that with map.equal_range(key) | mapped_value | invoke(fn). And then tidy the syntax, but I accept that wasn't an outstanding success! I collapsed the map.equal_range and mapped_value into one step and then we had map | mapped_values(key) | invoke([](...) {})

In the end what I had for you paraphrasing you snippet above would be:

map | mapped_values(key) | invoke([&](auto&& p) { // do something with p };

Obviously here I've pulled the "if" into the boilerplate code and eliminated the need for p to be optional. I like eliminating the possibility of a dereference of nullopt and I like not having to include either implementation of optional. If we need to have work in the "else" path then this is less awesome and projection to optional is one approach to tackle this. Where we would like to pick a replacement default value when missing that could probably also use syntactic sugar to replace the invoke. The composition possibilities looked interesting to me, and I thought that the common cases could appear approximately as the expressed desired syntax of other approaches with a little syntactic sugar.

I was always open to name changing, so mapped_values could be "lookup" if you wanted. Of course I could go a step further and collapse those two adapters. We also discussed what would be returned, and of course it would be possible to return an optional single element where appropriate. I like the generality of passing the range of values back, especially since that works for multi containers. The optional, to me, looks to be an inferior alternative for an empty or not range, for many of the cases. Perhaps more real world examples would change my mind. Nonetheless I could give the projection to optional no trouble at all. Hence, in the end, I thought I'd pretty much got every issue I'd heard addressed. I realize though that small details to me may be much more important to others. I don't typically confuse my opinion with fact. I'm definitely not young enough to know everything.

I liked not having the explicit if, but totally accept that before it becomes familiar it may look odd. It also takes away the possibility of incorrectly dereferencing a nullopt, while apparently providing a very similar solution to the expressed problem. One of the posted suggestions used the optional map function to get the same advantage.

Thus I went on this journey to see if we could just keep the ranges we already had and get the syntax close to the original expressions. It looked pretty close to me, but it didn't appeal, which is totally fine. It would be boring if we all agreed all the time.

I thus don't have anything concrete to merge that addresses the original poster's request, which is disappointing but totally okay from my perspective.

Hi Neil, I think you may be drawing the wrong conclusions from this thread. Your observations in this thread are very valuable. The positive outcome may be not necessarily the specific changes in the libraries, but generally raising the awareness of the design tradeoffs, things like pizza operator, unified function call syntax, and ranges in general. There are a lot of people that read this thread but do not talk, like myself (until now). I believe you mentioned adding things to Boost.Ranges. I think it would be beneficial, even if there still exist people that choose not to use them. If Boost.Ranges are to be compared to other solutions we need Boost.Ranges to be as good as possible. Thanks, &rzej;

...

Although it strikes me that it should probably be named `try_at` instead

Totally with you on "try_at" versus "try_find", but if we use 0..1 ranges it is just equal_range, that's mapped_value adapted.

...

of `try_find`. `find` is already a try-ing member function, because it doesn't fail, whereas `at` both returns the correct reference type, and can fail.

Of course `try_at` by convention ought to return boost::system::result or std::expected instead of an optional, but that's another story.

Interestingly, and maybe relevant for this discussion, `at` was added to associative containers even though it could have been made a free function instead.

I've looked at the recent standard changes and to me it looks like we are semi-randomly preferring member and non-member functions in an inconsistnent and arbitrary manner. We also added std::erase_if as a non-member for std::vector and other containers. I would estimate that you are probably aware of many more of these changes than I am. They've gone in both directions. To me it seems we can't use recent history to inform us. I accept that it may appear arbitrary and inconsistent because I have failed to comprehend the rules that drive the final decisions.

Apologies if by asserting that I've achieved nothing this seems to be an emotional reaction. It isn't. While I'm disappointed I couldn't help with the original problem. I consider this eventuality to be part of the acceptable, expected set of outcomes from the effort. If it weren't then I'm not taking enough risk.

My perception is that my suggestion doesn't have broad appeal and hence I'm not pushing to make it happen. Other solutions can be explored, or we can leave it as it is. I never intended my experimentation to block progress for other approaches. I also never intended to make any demands about how anyone approached the problem.

I'm happy to continue to help, if I am actually doing so, but equally happy to get out of the way and let others come up with something else.

Regards,

Neil Groves

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Andrey Semashev wrote:

On 12/21/24 15:11, Peter Dimov via Boost wrote:

...

Christian Mazakas wrote: What this user has always wanted is to not have to write this:

auto it = map.find( key ); if( it != map.end() ) { // do something with it->second }

That's why I'm always defining a helper function `lookup` that returns a pointer, which allows me to write this instead:

if( auto p = lookup( map, key ) ) { // do something with *p }

I'll note that with C++17 you can also write this:

if ( auto it = map.find( key ); it != map.end() ) { // do something with it->second }

But each of the three variants seem pretty much equivalent to me, from the code clarity standpoint.

It was admittedly much worse in C++03, where you had to spell out std::map::const_iterator it in its entirety. Plus a typename in front if you happened to be in a template. C++17 makes it almost palatable, except for the need to repeat `it->second`.

Sent from my iPad

On 20 Dec 2024, at 20:37, Neil Groves via Boost wrote:

On Thu, 19 Dec 2024 at 15:30, Christian Mazakas via Boost < boost@lists.boost.org> wrote:

...

The case against a free function gets a lot stronger once you remember that optional in C++ still supports unchecked accesses with trivial syntax and that the most ergonomic and safe way to actually use it is via something like this:

users.find(user_id).map([](auto& user) { user.status = deactivated; return user; });

As you know I've tried seeing how far I could get with some work in Boost.Range. I appreciate this isn't the solution you are dreaming of.

I have working on a private branch (without new member functions in any associative container):

users | mapped_values(user_id) | invoke([](user_type& user) { user.status = deactivated; });

How is this better than for(auto&& user : users) if(user == user_id) deactivate(user); ? My thoughts on the pipeline code: - It took a while to figure out what it was doing. - you can’t easily place a breakpoint in it - it’s only possible with a load of cryptic template expansion - requires a deep knowledge of not only the language and template but also the helper library. Whereas the procedural code is: - absolutely obvious in its intent - easily debuggable - implemented using only the language and no template help at all - trivially teachable

To me, this looks remarkably similar to your example.

This works for std::map, std::multimap, std::unordered_map and std::unordered_multimap. I find this more readable than the original. Of course, this is for one very specific example and I lack support for many of the functions that boost::optional has that one may likely wish for when using a container that has at most 1 mapped_value per key.

mapped_values(key) is a Boost.Range adaptor similar to the existing "map_values" except that it takes a key parameter and applies equal_range to the source range before extracting the mapped_type. I did also find the map adaptors in Boost.Range to be unnecessarily fussy about const-ness propagation. I had to fix this on my branch to make any substantial progress.

I wondered if reading this example line above if it didn't look quite as horrific as perhaps you were imagining?

One unfortunate aspect of the original snippet is the boost::optional choice of the word "map" where we are also working with a concrete "map" container.

There is plenty of work to do if this is of interest to anyone. I would want to ensure there was no performance overhead, and currently I'm not entirely convinced that my initial implementation that uses the member equal_range is optimal. This is an implementation detail, which I would work through.

The value_or ( or front_or ) functionality for non-multi containers can be added in a similar manner, and be broadly usable for range algorithms.

Were this a free function, a user wouldn't be able to create this nice

...

fluent interface that users, by and large, seem to actually like.

For the left-to-right syntax I noticed that there has been a neat looking proposal https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2011r0.html

I'm hoping the left-to-right solution will appear by one or more of the active proposals while we make progress coping with the ways we have.

...

- Christian

If this is of any interest to you I'll look at putting some of this into the develop branch of Boost.Range. If it is not interesting I shall continue on my branch and build a more compelling iteration that I shall land later.

I hope this helps,

Neil Groves

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Fri, Dec 20, 2024 at 11:58 PM Christian Mazakas via Boost < boost@lists.boost.org> wrote:

My whole thing was about what users actually wanted vs all the costs of everything else.

In C++, we have basically unlimited power so we can honestly do basically whatever we want.

I think most users just want `try_find() -> optional`, with `std::optional` and not Boost. Boost is okay as a polyfill, but users have told us ad nauseum that in 2024, they prefer STL versions.

Second, I think the burden on users to opt into the perfectly generic version is too much for the little we're actually doing.

I'm just trying to give my opinion about the potential reception of such efforts. It's fine on paper but I'm not sure we have users clamoring for the feature to manifest in this way.

I fully agree with everything here(except std:: vs boost optional is a bit more complex, see bellow), I am fully aware of costs to enable nice syntax for users. I do not think they are negligible, but I think they are worth it. My experience is also that users prefer std:: alternative if it is in C++. Issue here is more complicated since it is still unclear if optional will be in C++26, and because monadic operations have different member function names. Regarding boost::optional overhead: as mentioned before on my machine adding optional increased preprocessed line count by around 3%(compared to same map header without #include ) Also not sure if ranges code compiles faster than optional without making some realistic compile test cases since I presume optional uses are easier to compile than ranges version. On Sat, Dec 21, 2024 at 11:02 AM Neil Groves via Boost < boost@lists.boost.org> wrote:

So in addition to solving the original problem it reduced the need to teach all the special divergent ways of performing find. That IME is a problem to teach and remember and having no unified optimal syntax for lookup is a pain sometimes in generic code. We have to know it is a map, alter the code to optimize the lookup.

As mentioned before std::find/std::contains do not do dispatch to members so it is unclear if this is easier to teach to people when existing algorithms do not do dispatch. And if we provide find it makes sense to provide all other algorithms with faster member implementations like contains and count.

Ultimately though, we all ended up going in different directions and going nowhere!

Even if we do not agree in this particular use case I can understand that

some users might find :) use for dispatching free function in generic code, although it scares me we are encapsulating complexity.

Thus my syntax examples were to address the OPs proposition and attempt to hit a defined optimal syntax in the problen statement.

You are spot on though. This is all totally pointless.

Not sure why you feel this way, we can disagree, but does that mean we can not learn anything from discussion? I still prefer member function, I will not lie about that, but people here referenced P2011 that was super interesting(although I knew of pizza operator before I never studied it in detail), I have googled up P3021, and many comments here were interesting.

On Sat, Dec 28, 2024 at 12:08 PM Andrzej Krzemienski wrote:

...

About these compile times with boost::optional, in Boost 1.87 Boost.Optional dropped a number of dependencies on other Boost libraries, so I would expect the compile times to improve.

I did this tests recently with modular boost superproject so I am pretty sure those change are already reflected, and as I mentioned for my machine compile overhead was minimal. e21074030123956f4b0ac28fd591f5b253f18af6 (HEAD, tag: optional-2024-10-17 P.S. different hash map implementations might include different headers, so diff might be much larger than one for unordered_flat_map.hpp .