On 25.09.2015 18:37, Andrey Semashev wrote:

On 25.09.2015 17:35, Andrzej Krzemienski wrote:

...

Hi Everyone, I would like to inquire if there would be an interest in Boost in another library for storing optional objects, but working under different design goals.

Compact optional T has (or can have) the same sizeof as T. It uses one indicated value of T to represent the "empty" (or "singular") value. You can declare it like this:

compact_optional<evp_int<int, -1>> oi;

This reads: we have an optional int, with type int inside, where -1 represents the empty value. It can never have a genuine (non-empty value -1). This can be used, for instance, to wrap the std::string::npos into:

compact_optional<evp_int<string::size_type, string::npos>>

With the same memory layout as std::string::size_type, but with the special syntax for managing the singular value.

It is not meant to be an alternative to Boost.Optional: it targets a different application space.

The idea looks interesting - I sometimes have to deal with magic values. Currently I prefer to wrap the object into optional<> or at least make the special value not so magic (i.e. so that the code always does the right thing without checking for the magic value).

However, I'm not sure I agree with your rationale on the reduced interface and possibly the compact_optional naming.

1. You chose not to provide relational operators for compact_optional because you don't know how to order 'empty' values. I think you don't have to make that decision and simply forward the call to the underlying type. I mean, you always have the stored object constructed in some state and as long as it implements operators you can always use them.

2. compact_optional does not provide direct assignment of the values of the stored type, requiring to manually construct a compact_optional-wrapped value. To me, this is too cumbersome to use while I don't see any wins from this restriction. Besides more typing, this essentially requires to use a typedef to declare and use the compact_optional variable.

3. Nitpick: the typical name for the getter operation is get(), not value(). I would also have used empty() to test for the magic value but maybe that makes you feel it like a container.

4. Regarding compact_optional naming. While the class can be used for similar purpose as optional, its interface and behavior are significantly different. Perhaps a different name would be better to avoid confusion (e.g. nullable<>).

5. A suggestion: add evp_zero and evp_empty policies. The first uses literal zero as the special value and can be used with numeric (integer and fp) and pointer types. The second uses a default constructed value as the magic value and a member empty() function to test for magic value. This could be useful with containers, strings and ranges.

6. Also, I think unsafe_raw_value() is a too scary name. There's nothing unsafe about obtaining the special value from the wrapper. I would name it get_raw() (which is in addition to get() which is now value()).

On 25.09.2015 18:55, Marcel Raad wrote:

Andrey Semashev-2 wrote

...

1. You chose not to provide relational operators for compact_optional because you don't know how to order 'empty' values. I think you don't have to make that decision and simply forward the call to the underlying type. I mean, you always have the stored object constructed in some state and as long as it implements operators you can always use them.

This is exactly the problem I sometimes have with Boost.Optional: in some use cases, I want the compiler to remind me that my special value is special and always has to be checked explicitly instead of assuming it's always the lowest possible value. That doesn't apply to operator== and operator!=, of course.

In my practice most of the time you just want some kind of ordering (e.g. for binary lookup). For these cases it's useful for ordering to just work out of the box. When the actual order matters you can always provide your own operator overloads - something you will have to do either way.

On 28.09.2015 01:07, Gavin Lambert wrote:

On 26/09/2015 04:17, Andrey Semashev wrote:

...

On 25.09.2015 18:55, Marcel Raad wrote:

...

Andrey Semashev-2 wrote

...

1. You chose not to provide relational operators for compact_optional because you don't know how to order 'empty' values. I think you don't have to make that decision and simply forward the call to the underlying type. I mean, you always have the stored object constructed in some state and as long as it implements operators you can always use them.

This is exactly the problem I sometimes have with Boost.Optional: in some use cases, I want the compiler to remind me that my special value is special and always has to be checked explicitly instead of assuming it's always the lowest possible value. That doesn't apply to operator== and operator!=, of course.

In my practice most of the time you just want some kind of ordering (e.g. for binary lookup). For these cases it's useful for ordering to just work out of the box. When the actual order matters you can always provide your own operator overloads - something you will have to do either way.

Isn't the solution there to specialize std::less and/or std::hash instead of providing operators? (I vaguely recall there being a nicer way to do this proposed and/or implemented in one of the newer standards, but I don't recall the details.)

That's one alternative, surely. My point is that this should "just work" by default and allow customization when something special is required.

2015-09-25 17:37 GMT+02:00 Andrey Semashev <andrey.semashev@gmail.com>:

On 25.09.2015 17:35, Andrzej Krzemienski wrote:

...

Hi Everyone, I would like to inquire if there would be an interest in Boost in another library for storing optional objects, but working under different design goals.

Compact optional T has (or can have) the same sizeof as T. It uses one indicated value of T to represent the "empty" (or "singular") value. You can declare it like this:

compact_optional<evp_int<int, -1>> oi;

This reads: we have an optional int, with type int inside, where -1 represents the empty value. It can never have a genuine (non-empty value -1). This can be used, for instance, to wrap the std::string::npos into:

compact_optional<evp_int<string::size_type, string::npos>>

With the same memory layout as std::string::size_type, but with the special syntax for managing the singular value.

It is not meant to be an alternative to Boost.Optional: it targets a different application space.

The idea looks interesting - I sometimes have to deal with magic values. Currently I prefer to wrap the object into optional<> or at least make the special value not so magic (i.e. so that the code always does the right thing without checking for the magic value).

Hi Andrey, thank you for your reply.

However, I'm not sure I agree with your rationale on the reduced interface and possibly the compact_optional naming.

1. You chose not to provide relational operators for compact_optional because you don't know how to order 'empty' values. I think you don't have to make that decision and simply forward the call to the underlying type. I mean, you always have the stored object constructed in some state and as long as it implements operators you can always use them.

So this is a trade-off between convenience/expressiveness and the potential to detect unintended semantics at compile-time. If there is sufficient argumentation in favor of adding them, I can always do it. As Rob says, this can also be encoded in the policy.

2. compact_optional does not provide direct assignment of the values of the stored type, requiring to manually construct a compact_optional-wrapped value. To me, this is too cumbersome to use while I don't see any wins from this restriction. Besides more typing, this essentially requires to use a typedef to declare and use the compact_optional variable.

Are you proposing a member function like opt.store_raw_value(v); ?

3. Nitpick: the typical name for the getter operation is get(), not value(). I would also have used empty() to test for the magic value but maybe that makes you feel it like a container.

4. Regarding compact_optional naming. While the class can be used for similar purpose as optional, its interface and behavior are significantly different. Perhaps a different name would be better to avoid confusion (e.g. nullable<>).

I am not particularly tied to name compact_optional. I can be persuaded to rename it. On the other hand, I am not in favor of any names containing "null". I am not an English speaker, but no me "null" sounds like "numeric value zero", which makes sense for the pointer, but not for something that just is not. Maybe "singular" or "special".

5. A suggestion: add evp_zero and evp_empty policies. The first uses literal zero as the special value and can be used with numeric (integer and fp) and pointer types. The second uses a default constructed value as the magic value and a member empty() function to test for magic value. This could be useful with containers, strings and ranges.

Agreed on evp_empty, but I fail to see the advantage of evp_zero over the already existing evp_value_init. The later uses the value initialized T, which already is zero for ints, floats and pointers. Or do you expect the comparison to literal 0 to be faster? Regards, &rzej

On 26 September 2015 at 11:52, Andrzej Krzemienski <akrzemi1@gmail.com> wrote:

I am not particularly tied to name compact_optional.

I'm strongly against the word "optional" appearing the name, for the following reasons: - optional<int> allows me to use every single value that can be stored in an int. This doesn't. - optional<string> allows one to shorten the lifetime of the string it holds. This doesn't. - optional<T> has a nothrow default constructor. This doesn't. At best, it resembles optional only superficially. Please give it a different name. -- Nevin ":-)" Liber <mailto:nevin@eviloverlord.com> (847) 691-1404

On Sat, Sep 26, 2015 at 1:12 PM, Nevin Liber <nevin@eviloverlord.com> wrote:

I'm strongly against the word "optional" appearing the name, for the following reasons:

- optional<int> allows me to use every single value that can be stored in an int. This doesn't. - optional<string> allows one to shorten the lifetime of the string it holds. This doesn't. - optional<T> has a nothrow default constructor. This doesn't.

At best, it resembles optional only superficially. Please give it a different name.

This. However, I do think that there is a place for a separate intrusive_optional as well, though the empty value should not be a valid value of the type, but rather, just be able to take advantage of its storage, and the type should not be constructed while the in the empty state. I.E. an optional reference only takes up the size of a pointer because it uses the "0" value for empty. Similarly, any type that contains a reference (or a pointer that cannot be null) also can take advantage of this by way of an intrusive optional. The user would have to provide a predicate that operates on the raw storage of the value, and access it in a way that is well-formed whether the object is constructed there or not, so it is a little bit trickier to specify properly. -- -Matt Calabrese

On 27.09.2015 14:00, Matt Calabrese wrote:

On Sat, Sep 26, 2015 at 1:12 PM, Nevin Liber <nevin@eviloverlord.com> wrote:

...

I'm strongly against the word "optional" appearing the name, for the following reasons:

- optional<int> allows me to use every single value that can be stored in an int. This doesn't. - optional<string> allows one to shorten the lifetime of the string it holds. This doesn't. - optional<T> has a nothrow default constructor. This doesn't.

At best, it resembles optional only superficially. Please give it a different name.

This. However, I do think that there is a place for a separate intrusive_optional as well, though the empty value should not be a valid value of the type, but rather, just be able to take advantage of its storage, and the type should not be constructed while the in the empty state. I.E. an optional reference only takes up the size of a pointer because it uses the "0" value for empty. Similarly, any type that contains a reference (or a pointer that cannot be null) also can take advantage of this by way of an intrusive optional. The user would have to provide a predicate that operates on the raw storage of the value, and access it in a way that is well-formed whether the object is constructed there or not, so it is a little bit trickier to specify properly.

If the empty state of intrusive_optional means the governed object is not constructed then the usability of this tool will be significantly reduced. Defining the policy of empty state detection would require intricate knowledge of behavior and binary layout of the adapted type, including ABI details such as padding and vtable/virtual inheritance table pointers. For instance, you wouldn't be able to use intrusive_optional with std::string.

On 27.09.2015 23:45, Matt Calabrese wrote:

On Sun, Sep 27, 2015 at 4:23 AM, Andrey Semashev <andrey.semashev@gmail.com> wrote:

...

If the empty state of intrusive_optional means the governed object is not constructed then the usability of this tool will be significantly reduced. Defining the policy of empty state detection would require intricate knowledge of behavior and binary layout of the adapted type, including ABI details such as padding and vtable/virtual inheritance table pointers. For instance, you wouldn't be able to use intrusive_optional with std::string.

Right, which is why you wouldn't use such an an "intrusive_optional" template in that specific case (if you made a string type you certainly could use it, though). I'm not saying intrusive_optional would replace your template,

That's not my template, it's Andrzej's. :)

I'm just agreeing with others that if you do use one of the existing, valid values for the type, it's not really the same kind of abstraction as optional, even though it is useful, so it should probably just have a different name. Places where something like intrusive_optional (which is similar to your type, but more akin to optional) are useful are with types that you have control over.

I suppose you could draw analogy between intrusive_optional & optional and intrusive_ptr & shared_ptr. Fair enough, intrusive_optional requires some amount of control over the adopted type, similar to intrusive_ptr. What's different is the amount of hackishness that is required to support intrusive_optional. intrusive_ptr does not require you to work with the raw storage or know the binary layout of the object while intrusive_optional does. As much as I like the idea of reusing the storage for discriminator, this just feels too fragile to me. Maybe if there was a safer interface for supporting intrusive_optional in user defined types it wouldn't feel that way.

On 27 September 2015 at 15:45, Matt Calabrese <rivorus@gmail.com> wrote:

For instance, I have a "card" type that I use for representing one of 52 possible playing cards. It takes up a byte, even though a byte can hold more that 52 different values. If I put a card in an optional as-is, then it's occupying extra space for the discriminator. I'd like to use optional to represent "not-a-card" because had I baked a "not-a-card" value into the original type itself, then any function that takes an actual card would need an additional precondition and I'd likely want to assert to make sure that the card is valid. With an intrusive_optional abstraction, it makes it easy to just use the spare storage in the byte in order to represent not-a-card only when it is needed, and it takes up no extra storage than what the base abstraction does.

So, your card type already has a disengaged state it knows how to deal with. Why then do you need to wrap your card type inside another template? And if it doesn't understand the disengaged state, then you have to break a class invariant (the "byte" being in the range 1..52) to do it, which just doesn't work. -- Nevin ":-)" Liber <mailto:nevin@eviloverlord.com> (847) 691-1404

On Sun, Sep 27, 2015 at 3:20 PM, Matt Calabrese <rivorus@gmail.com> wrote:

It doesn't. The idea is that the times that you can make such a hook for intrusive_optional are where it is not UB to access that storage regardless of whether or not the object is constructed. For instance, I use an unsigned char to store the value 0 through 51 and I know that this is at the start of the storage location for the card. As the creator of the card type I can guarantee this (this is why it is intrusive). With that knowledge, I can provide some small, default set of functions that operate on raw storage (not an instance of the card type) but that can do so safely whether the object is constructed or not since I can also via hooks guarantee that there will be an unsigned char in that location regardless.

Expanding on this, while it may sound hairy, in practice it should be able to be not too difficult to compose. For instance, if I make a type that contains a card, yet I did not create that card type itself, assuming that I know "card" supports intrusive optional, I would be able to easily point my own intrusive_optional hooks to the location of the "card" that I contain, and forward the functionality along. This leaves you with a higher-level type that again takes up no extra storage for the optional version, and the original type doesn't need a notion of emptiness. -- -Matt Calabrese

On Sun, Sep 27, 2015 at 3:48 PM, Nevin Liber <nevin@eviloverlord.com> wrote:

On 27 September 2015 at 17:20, Matt Calabrese <rivorus@gmail.com> wrote:

...

On Sun, Sep 27, 2015 at 3:08No, in other words you cannot construct a card with a not-a-card state.

Except that every function in its implementation may assume the card is *never* in the not-a-card state.

For instance, one could add ASSERT(1 <= cardIndex && cardIndex <= 52) all over the place, including in the beginning of the body of the destructor, and not expect anything to break.

That's still true. None of these asserts would fire. The value would only ever be "0" (I'm assuming you are implying 0 is the empty state here) when the card is not i na constructed state. In other words, if someone does: "foo = nullopt", it internally checks via raw storage hooks if the object is already logically nullopt, and if not, it destroys the card, and then sets the value via the hook, which operates on raw storage. On Sun, Sep 27, 2015 at 3:48 PM, Nevin Liber <nevin@eviloverlord.com> wrote:

It isn't just about room. The class has to know about it so that it can, at a minimum, weaken its invariants to support it. And if you have to do that, I don't see any need for some generic class to wrap it.

No, it doesn't. That state will never be set while the object is in a constructed state. That is what differs between this and compact_optional (I'm off on a tangent in case that's not clear, and we could probably start a different thread). The difference is in the nature of the hooks. These hooks would deal with raw storage, and do not merely make the overall optional "nullopt" by setting a value in the underlying object. Rather, when setting to nullopt, the object itself is destroyed and we just re-use the storage in a valid and non-conflicting way to identify that we are not in a constructed state. We can check which state the overall object is in (constructed or not constructed) regardless because the hooks operate on raw storage, and with the knowledge that regardless of the underlying object being constructed or not, there is (in this specific case) an unsigned char in a very particular location. -- -Matt Calabrese

2015-09-27 13:00 GMT+02:00 Matt Calabrese <rivorus@gmail.com>:

On Sat, Sep 26, 2015 at 1:12 PM, Nevin Liber <nevin@eviloverlord.com> wrote:

...

I'm strongly against the word "optional" appearing the name, for the following reasons:

- optional<int> allows me to use every single value that can be stored in an int. This doesn't. - optional<string> allows one to shorten the lifetime of the string it holds. This doesn't. - optional<T> has a nothrow default constructor. This doesn't.

At best, it resembles optional only superficially. Please give it a different name.

This. However, I do think that there is a place for a separate intrusive_optional as well, though the empty value should not be a valid value of the type, but rather, just be able to take advantage of its storage, and the type should not be constructed while the in the empty state. I.E. an optional reference only takes up the size of a pointer because it uses the "0" value for empty. Similarly, any type that contains a reference (or a pointer that cannot be null) also can take advantage of this by way of an intrusive optional. The user would have to provide a predicate that operates on the raw storage of the value, and access it in a way that is well-formed whether the object is constructed there or not, so it is a little bit trickier to specify properly.

I have now implemented this capability in compact_optional. See the related docs section: https://github.com/akrzemi1/compact_optional/blob/master/documentation.md#us... Regards, &rzej

5. A suggestion: add evp_zero and evp_empty policies. The first uses

Hi Andrzej, I've been using compact optional extensively for almost 2 months already and found it very useful so I'd like to see an improved version of it in Boost. My use case was an application in which the main data structor is a vector<optional<int>> (with ~O(10^10) elements), and the application expends most of the time processing it. Compared to compact_optional, Boost.Optional requires 2x more memory, and reduces the memory bandwidth by 2x. Using a vector<int> + boost::dynamic_bitset is infinitely better than using Boost.Optional, but complicates the code significantly since there are some values that the ints in that vector cannot never take. Arguably compact_optional is a perfect fit for my use case. Things that I liked using the library so far: 1. being able to customize the sentinel type via policies: sometimes I have a signed integer that can only be positive and want to use -1 as a sentinel, and some times I want to use std::numeric_limits<...>::max(), or some other value. Andrey Semashev wrote: literal zero as the special value and can be used with numeric (integer and fp) and pointer types. The second uses a default constructed value as the magic value and a member empty() function to test for magic value. This could be useful with containers, strings and ranges. I have mixed feelings about these since someone using compact optional should explicitly know what sentinel value they are using (e.g. for things like hashing, serialization, ...). So I would rather like it to strive for making this as explicit and less magical as possible, in particular for integers and fp types. The empty policy seems very convenient though so maybe the library should include some convenient policies somewhere (this would also make for good examples of how to write policies). 2. Tags: this allowed me to strongly type my interfaces very easily, and caught a lot of errors. 3. unsafe_raw_value together with a way to query the sentinel value, which I needed to implement e.g. `std::hash` for compact_optional. The library is WIP, but I also had some points of friction using it: 4. Operators: I want my compact_optional<int> to work "as an int" but fail fast (assert, throw, terminate) on any operation involving the empty state, so I had to implement my own operators for compact optional. Still, other applications have different needs, and one might want the operators to propagate the empty value, to throw an exception instead of calling assert, or to no exist at all and result in a compiler error. Since some operators must be implemented as member functions, there should be a way to customize this using a policy. Other points: 5. Being part of Boost.Optional: I think compact_optional must be part of Boost.Optional. Stateful policies would allow Boost.Optional to be implemented on top of compact_optional, which would allow everybody to experiment with better storage for their own types. For example sizeof(optional<T*>) can be made to be sizeof(T*), and the same could be done for e.g. optional<variant<...>>. Right now experimenting with this is hard because Boost.Optional interface doesn't allow to customize any of this. But basing its implementation on top of compact_optional could change that. 6. Name I really don't mind about compact_optional since 99% of the time one is going to use it through an alias anyways (specifying the type, policy, and tag all the time is just too painful). If Boost.Optional is reimplemented on tope of it, one could honor std::string by calling this basic_optional instead. Vladimir Batov wrote:

To me having allocated -1 or std::max<int> (or whatever) to represent no-value, i.e. "no-int" feels hackish. If, as a developer, I advertise "int" to the user, IMO the user is entitled to expect the full "int" range available. If, as a developer, I do not provide the full range, the proper way (IMO) would be to introduce a new explicit type to represent the supported subset (via enum or proper sensibly-named type -- time, speed, temperature, etc.) In other words, if I cannot provide "int", I should not be saying that I do.

Using the library the best thing that I can say is that I'm providing a "compact_optional<int, some_policy, struct this_is_my_temperature_type_tag> ". Vladimir Batov wrote:

I do have the need to be real frugal when I store data to the disk. But for that purpose I have to be far more economical and size-specific than compact_optional.

I use compact_optional to store data on disk and since sizeof(compact_optional<T>) == sizeof(T) I haven't ran into issues yet. How can one be more economical than that?

Gavin Lambert wrote:

On 28/09/2015 21:05, Andrzej Krzemienski wrote:

...

No, I was suggesting to allow assignment of values of type T (i.e. "opt = v;"). Having a named function like store_raw_value reduces the need in the typedef but the syntax should still be simpler. [...] In a way, this would mean and implicit conversion from T to compact_optional<T>. I do not want any kind of implicit conversion between the two (because the two are something different). I could offer:

opt.raw_value() = some_T;

This is still clunky; either an implicit conversion or simply an additional assignment operator to permit the other syntax is better.

I am against all three options, since right now my code looks like this: opt_t opt_val; opt_val = opt_t{some_T}; which is not that bad. In particular: - I don't use raw_value for assignment, I only use it for when I need unwrapped access to the storage_type independently of the optional being empty or not, e.g., to implement a hash function or for serialization. - additional assignment operator/implicit conversion: this would make the tags less useful, so I am against it since using tags extensively has made my code better. @Andrzej: Right now your implementation allows constructing a compact_optional from the sentinel value, which results in an empty compact_optional. What is the motivation for this? In my fork I've disabled it for the following reasons: - I'd rather use compact_optional's default constructor to explicitly state that i'm constructing an empty optional, this made it easier to reason about my code, - the other constructors from value_type can then hint the compiler that the compact_optional is not empty (but I did not profile so I don't know if this has any impact at all).

On Tue, Sep 29, 2015 at 2:16 AM, Andrzej Krzemienski <akrzemi1@gmail.com> wrote:

2015-09-29 9:11 GMT+02:00 Gonzalo BG <gonzalobg88@gmail.com>:

...

@Andrzej: Right now your implementation allows constructing a compact_optional from the sentinel value, which results in an empty compact_optional. What is the motivation for this?

In my fork I've disabled it for the following reasons: - I'd rather use compact_optional's default constructor to explicitly

state

...

that i'm constructing an empty optional, this made it easier to reason about my code, - the other constructors from value_type can then hint the compiler that the compact_optional is not empty (but I did not profile so I don't know if this has any impact at all).

The motivation for this was to enable an integration of compact_optional with older/other parts of the program that still use magical values:

``` using Index = compact_optional<evp_int<string::size_type, string::npos>> string s = /*...*/ Index i {s.find("substr")}; ```

In the above example, I will not be changing std, but want to change to compact_optional as soon as possible.

But in fact, I could provide a dedicated conversion function for this use case. This aspect is still under consideration, so I may go with your suggestion.

Regards, &rzej

In fact, I think constructing from the magic value is the primary use case for such a class. This makes it easy to integrate with, for instance, a networking protocol that has a magic value meaning "no value here". Now that I've thought about it more, compact_optional is a way of saying that you care exactly how that "not present" state is constructed.

On Sat, Sep 26, 2015 at 7:52 PM, Andrzej Krzemienski <akrzemi1@gmail.com> wrote:

2015-09-25 17:37 GMT+02:00 Andrey Semashev <andrey.semashev@gmail.com>:

...

However, I'm not sure I agree with your rationale on the reduced interface and possibly the compact_optional naming.

1. You chose not to provide relational operators for compact_optional because you don't know how to order 'empty' values. I think you don't have to make that decision and simply forward the call to the underlying type. I mean, you always have the stored object constructed in some state and as long as it implements operators you can always use them.

So this is a trade-off between convenience/expressiveness and the potential to detect unintended semantics at compile-time. If there is sufficient argumentation in favor of adding them, I can always do it. As Rob says, this can also be encoded in the policy.

It can, but I really don't see the reason for it. Can you give an example where operators defined for type T are not sensible to compact_optional<policy<T> >?

...

2. compact_optional does not provide direct assignment of the values of the stored type, requiring to manually construct a compact_optional-wrapped value. To me, this is too cumbersome to use while I don't see any wins from this restriction. Besides more typing, this essentially requires to use a typedef to declare and use the compact_optional variable.

Are you proposing a member function like opt.store_raw_value(v); ?

No, I was suggesting to allow assignment of values of type T (i.e. "opt = v;"). Having a named function like store_raw_value reduces the need in the typedef but the syntax should still be simpler. You may object that this allows the assignment to make the opt object empty (or null, or singular - not sure what terminology you prefer). This may seem counterintuitive at first glance, but only as long as you treat the type as another flavor of optional<>, which is incorrect and is one reason why the name should be changed. What compact_optional really is is just a wrapper that allows to easily distinguish a special value of the governed object from all other values, and in that light there's nothing wrong if assigning this special value to the wrapper makes it singular. Analogously, there's nothing wrong with assigning a nullptr to a pointer.

I am not particularly tied to name compact_optional. I can be persuaded to rename it. On the other hand, I am not in favor of any names containing "null". I am not an English speaker, but no me "null" sounds like "numeric value zero", which makes sense for the pointer, but not for something that just is not. Maybe "singular" or "special".

Well, I'm not a native speaker but AFAIK null does literally mean zero in English. However, in the programming domain I don't see null as something that is necessarily equivalent to a zero value. It's more like a 'special value' to me. E.g. in databases null is a special value that has no connection to zero at all. Even in C++ null pointers are not required to have zero value as the underlying implementation. But I'm not insisting on nullable<>. There's also a close alternative of nilable<>. Singular is the characteristic of a particular value, not the type or range of values, so it's difficult to compose a name from it (I believe, 'singularable' is not a word). Special is too generic, IMHO. We can go a different way: nav_adapter<> (where nav stands for not-a-value) or singular_adapter<>. Although I like it less than nullable.

...

5. A suggestion: add evp_zero and evp_empty policies. The first uses literal zero as the special value and can be used with numeric (integer and fp) and pointer types. The second uses a default constructed value as the magic value and a member empty() function to test for magic value. This could be useful with containers, strings and ranges.

Agreed on evp_empty, but I fail to see the advantage of evp_zero over the already existing evp_value_init. The later uses the value initialized T, which already is zero for ints, floats and pointers. Or do you expect the comparison to literal 0 to be faster?

Oh, it didn't occur to me until your reply that evp_value_init already covers the evp_zero use cases. Ok, good, no need for evp_zero.

On September 26, 2015 5:34:59 PM EDT, Andrey Semashev <andrey.semashev@gmail.com> wrote:

On Sat, Sep 26, 2015 at 7:52 PM, Andrzej Krzemienski <akrzemi1@gmail.com> wrote:

...

I am not particularly tied to name compact_optional. I can be persuaded to rename it. On the other hand, I am not in favor of any names containing "null". I am not an English speaker, but no me "null" sounds like "numeric value zero", which makes sense for the pointer, but not for something that just is not. Maybe "singular" or "special".

Well, I'm not a native speaker but AFAIK null does literally mean zero in English. However, in the programming domain I don't see null as something that is necessarily equivalent to a zero value. It's more like a 'special value' to me. E.g. in databases null is a special value that has no connection to zero at all. Even in C++ null pointers are not required to have zero value as the underlying implementation.

FWIW, in my experience as a native English speaker, null is rarely synonymous with zero. It connotes having no value or (legal) power.

But I'm not insisting on nullable<>. There's also a close alternative of nilable<>. Singular is the characteristic of a particular value, not the type or range of values, so it's difficult to compose a name from it (I believe, 'singularable' is not a word). Special is too generic, IMHO. We can go a different way: nav_adapter<> (where nav stands for not-a-value) or singular_adapter<>. Although I like it less than nullable.

"nav" is a common abbreviation for "navigation" and "navigator", so that would easily cause confusion.

...

...

5. A suggestion: add evp_zero and evp_empty policies. The first uses literal zero as the special value and can be used with numeric (integer and fp) and pointer types. The second uses a default constructed value as the magic value and a member empty() function to test for magic value. This could be useful with containers, strings and ranges.

Agreed on evp_empty, but I fail to see the advantage of evp_zero over the already existing evp_value_init. The later uses the value initialized T, which already is zero for ints, floats and pointers. Or do you expect the comparison to literal 0 to be faster?

Oh, it didn't occur to me until your reply that evp_value_init already covers the evp_zero use cases. Ok, good, no need for evp_zero.

An evp_zero_init (or evp_zero_initialization) would cover zeroes for built-in types and default construction of UDTs. ___ Rob (Sent from my portable computation engine)

On 28/09/2015 21:05, Andrzej Krzemienski wrote:

...

No, I was suggesting to allow assignment of values of type T (i.e. "opt = v;"). Having a named function like store_raw_value reduces the need in the typedef but the syntax should still be simpler. [...] In a way, this would mean and implicit conversion from T to compact_optional<T>. I do not want any kind of implicit conversion between the two (because the two are something different). I could offer:

opt.raw_value() = some_T;

This is still clunky; either an implicit conversion or simply an additional assignment operator to permit the other syntax is better. It's also logical -- an optional type is essentially a superset of the original type (in the case of boost::optional it's a precise superset, while in this case it's technically an equivalent set with one redefined value, but since that value is supposedly invalid for the original type it's still logically a superset). The only real danger that I can see of supporting the direct assignment syntax is if someone explicitly assigns the sentinel value directly. But while that's bad style it shouldn't be undefined behaviour.

On 25.09.2015 19:06, Marc Glisse wrote:

On Fri, 25 Sep 2015, Andrzej Krzemienski wrote:

...

Hi Everyone, I would like to inquire if there would be an interest in Boost in another library for storing optional objects, but working under different design goals.

Compact optional T has (or can have) the same sizeof as T. It uses one indicated value of T to represent the "empty" (or "singular") value. You can declare it like this:

compact_optional<evp_int<int, -1>> oi;

This reads: we have an optional int, with type int inside, where -1 represents the empty value. It can never have a genuine (non-empty value -1). This can be used, for instance, to wrap the std::string::npos into:

compact_optional<evp_int<string::size_type, string::npos>>

With the same memory layout as std::string::size_type, but with the special syntax for managing the singular value.

It is not meant to be an alternative to Boost.Optional: it targets a different application space.

In my opinion, this is not an alternative to Boost.Optional because it should be part of Boost.Optional. For the interface, you could either add a second template parameter to boost::optional specifying the policy, or specialize boost::optional<magic_name<T>>.

I'm not opposed to making it part of Boost.Optional library, but not as a specialization of optional<>. The proposed class is very different from optional<>.

2015-09-26 0:45 GMT+02:00 work <Vladimir.Batov@constrainttec.com>:

Andrzej,

On 2015-09-26 00:35, Andrzej Krzemienski wrote:

...

I would like to inquire if there would be an interest in Boost in another library for storing optional objects, but working under different design goals.

... It uses one indicated value of T to represent the "empty" (or "singular") value.

In other words, your proposed design is intrusive as it carves something from T for its own purposes.

Indeed, I have also considered name "intrusive_optional".

1) IMO non-intrusive design is by far preferred even if it comes with extra-cost (shared_ptr might serve as an example). It's not to say intrusive design has no place. It does... but seems niche as opposed to a generic solution... which Boost seems to be positioning itself (IMO).

Indeed; also compact_optional library comes with the advice that you should try boost::optional first, and only if you have performance problems with it, should you try to use compact_optional instead.

2) Obviously that technique has been used for a long time and it's not immediately obvious to me if the extra interface adds much real value or most importantly clarity/safety. To me having allocated -1 or std::max<int> (or whatever) to represent no-value, i.e. "no-int" feels hackish. If, as a developer, I advertise "int" to the user, IMO the user is entitled to expect the full "int" range available. If, as a developer, I do not provide the full range, the proper way (IMO) would be to introduce a new explicit type to represent the supported subset (via enum or proper sensibly-named type -- time, speed, temperature, etc.) In other words, if I cannot provide "int", I should not be saying that I do.

Ok, I think I see where you are coming from. This is an interesting topic in itself; but for now I only stick to advertising compact_optional. Using the words from your description above, when you have decided to use the hack with "magic value", you can use compact_optional to make this explicit and visible to the type-system: now "int" and "int with magic value" are never confused.

3) I am trying to see if compact_optional might fit in my projects and fail. When in memory, I by far prefer boost::optional for its orthogonality -- it adds functionality without taking anything from T.

And more: boost::optional does not force you to take the time and consider and declare how you want to store the no-value.

I do have the need to be real frugal when I store data to the disk. But for that purpose I have to be far more economical and size-specific than compact_optional.

Indeed. We could say that compact_optional gives you a "generic optimization": it is a fairly generic and mechanical optimization over boost::optional, but to maximally optimize your particular case, you need to do it manually.

...

...

It is not meant to be an alternative to Boost.Optional: it targets a different application space.

I am not sure I immediately agree with you positioning your proposed "compact_optional" as no competitor to the established "optional". After all, your proposed "compact_optional" tries to address the same problem of representing no-value. You even call it similarly. Surely, it for a reason.

If not anything else, this often came out as an extension request to boost::optional and std::experimental::optional. My idea of the two libraries is this: When you design a new data structure, you just use boost::optional to represent optional values. Whet it never causes performance issues, you leave it at that. Another situation is where you already encode a magic value in your type int, you consider changing it to boost::optional, but are concerned about adding potential run-time overhead. Your concerns may be misplaced, but only having a concern (valid or not) is often enough to reject a safer library solution (boost::optional). compact_optional does not give you this concerns: it gives you a different trade-off between type-safety and "performance concerns".

Having said that I can easily be wrong and I am prepared to be convinced otherwise.

V.

Reards, &rzej

On Thu, Oct 1, 2015 at 3:13 AM, Domagoj Saric < domagoj.saric@littleendian.com> wrote:

The approach with different class templates will actually cause uglier user code in the long run (as opposed to a relatively minor _one_time_ hurdle in only some projects that would be broken by a new defaulted parameter) - we'll end up with projects that use optional, pink_optional, compact_optional, smelly_optional and hal9000_optional... with someone having to track and document where and why each of those is used.

It seems you are saying you want to be able to define some optional-specific policy specification called Policy, e.g. NeverNan, and then be able to say optional<float, NeverNan>. This type would behavior similarly, but not identically, to optional<float>. It seems the alternative would be to just define OptionalNeverNan<T> as an alias, using whatever mechanism exists to define it. Users don't have to care if it is aliasing optional<T, NeverNan> or compact_optional<T,NeverNan> or whatever. Calling it optional<T, Policy> instead of something_else<T, Policy> doesn't really help generic code, since generic code is going to use optional<T>; it will have no way of knowing what Policy to use. If you have to tell it the optional policy to use, you could just as well tell it the full optional type to use. That is why the real win comes from specializing optional<T> for certain user-defined types T. You would call your type NeverNan<float> and then (somehow) specialize optional<NeverNan<float>>.