Le 2022-06-02 08:06, Gavin Lambert via Boost a écrit :

On 2/06/2022 17:57, Julien Blanc wrote:

...

I've written and released a (rather small) library which allows defining arrays with arbitrary index mechanisms (hence the name, indexed_array). This especially allows indexing with scoped enums, use non zero-starting indexes, or use sparse indexes. This is not a view, but an array in the sense that it owns the data, and provides the same guarantees and functionalities as std::array, such as triviality propagation, being able to reside in read-only program sections, etc.

How does it compare with Boost.MultiIndex?

I need to fill the feature comparison table, thanks for suggesting including MultiIndex. Unless i'm mistaken, multi-index is intended for dynamic-sized containers, and for providing different ways of accessing the same set of elements. It is never trivial, and cannot reside in a read-only program section (this is an important use case for me, targetting MCUs running program from flash). Its goal is to provide fast access via different ways (i see it as an equivalent of database indexes). On the other side, indexed_array provides a single indexing mechanism, guarantees triviality propagation and contiguity of elements. So, i believe they address different use cases. Regards, Julien

On Thu, Jun 2, 2022 at 8:59 AM Julien Blanc via Boost <boost@lists.boost.org> wrote:

Le 2022-06-02 08:06, Gavin Lambert via Boost a écrit :

...

How does it compare with Boost.MultiIndex?

I need to fill the feature comparison table

Can you also compare it with https://github.com/serge-sans-paille/frozen ? More use-case examples would be nice too. For when it's applicable and useful. --DD

On 6/2/22 13:30, Julien Blanc wrote:

Le jeudi 02 juin 2022 à 11:00 +0200, Maximilian Riemensberger a écrit :

...

On 6/2/22 07:57, Julien Blanc via Boost wrote:

Is there a particular reason to require c++17? I guess some std::array features that got added? I would assume that it should be possible to make it compile under C++14 (maybe using some boost array) since most constexpr things should already be available there.

The first reason it requires C++17 is because this is what i am targetting at the moment. However, making it C++14-compliant would require a few changes:

* the library use template auto syntax in several places, this allows writing indexed_array<float, interval<1, 10>>. In C++14 one would need to write indexed_array<float, interval<int, 1, 10>>, or resort to macros. * it makes use of std::is_invokable to correctly disable the default aggregate constructor if given a safe_arg initializer list. This can be worked around (iirc is_invokable can be emulated in C++14) * it also makes use of std::apply for multi-dimensional indexing. Likewise, this can be emulated in C++14. * there are probably a few others that i'm missing. In short: it can be done, but sticking to C++17 avoided reimplementing several now-standard library features.

I see. The first one is definitely a language limitation. But I guess it's mostly an inconvenience to the users of the library if they are on C++14. The other ones as far as I can see have counterparts in boost that would work in C++14 as well. So I guess it could be done without too much hassle if there is a need and the library would be included in boost. Either way, it's very useful regardless of the std version details. Best, Max

On Fri, 3 Jun 2022 at 08:31, Julien Blanc via Boost <boost@lists.boost.org> wrote:

Le jeudi 02 juin 2022 à 14:00 +0200, Maximilian Riemensberger a écrit :

...

On 6/2/22 13:30, Julien Blanc wrote:

...

[...]

I see. The first one is definitely a language limitation. But I guess it's mostly an inconvenience to the users of the library if they are on C++14. The other ones as far as I can see have counterparts in boost that would work in C++14 as well. So I guess it could be done without too much hassle if there is a need and the library would be included in boost. Either way, it's very useful regardless of the std version details.

I'm already depending on mp11, so adding a dependency to more boost libraries (i have been stuck too long with c++11 and old boost releases, and completely missed that hof provides is_invocable and apply) is not an issue.

I would need, however, to provide an unified syntax for declaring intervals for C++14 and C++17, otherwise a compiler update would break user code. I think this can be done only by using a macro (which would be mandatory only for C++14 code, and provided in C++17 for backward compatibility). Anyhow, I created a github issue to track any progress on this side.

For those interested, i filled up the feature comparison table ( https://julien-blanc-tgcm.github.io/indexed_array/comparison.html ) and added some use cases in the readme file.

Thank you this has helped me. I have one question to add: Are you able to provide one or two motivating use cases? i.e. scenarios in which I would strongly prefer to use this library rather than a std::array and static_cast?

Best regards,

Julien

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Fri, 3 Jun 2022 at 09:19, Julien Blanc <julien.blanc@tgcm.eu> wrote:

Le 2022-06-03 08:44, Richard Hodges via Boost a écrit :

...

On Fri, 3 Jun 2022 at 08:31, Julien Blanc via Boost <boost@lists.boost.org> wrote:

Are you able to provide one or two motivating use cases? i.e. scenarios in which I would strongly prefer to use this library rather than a std::array and static_cast?

You mean, in addition to https://github.com/julien-Blanc-tgcm/indexed_array#sample-use-cases ?

Yes, I'm looking for a real-world use case. The trivial hypothetical cases offered are building a static association between an enum value and some other value. This can be done with switch/case (for example), in about the same amount of typing, at compile time, in a function. For example: constexpr std::uint32_t base_address(i2c_id id) { switch(id) { case i2c_id::i2c1: return 0x1060000u; case i2c_id::i2c2: return 0x1070000u; case i2c_id::i2c3: return 0x1080000u; case i2c_id::i2c4: return 0x1090000u; } } constexpr std::uint32_t offset(i2c_register r) { switch(r) { case i2c_register::set_clock: return 0; case i2c_register::read: return 1; case i2c_register::write: return 2; case i2c_register::ack: return 3; } } https://godbolt.org/z/bYYq65hc6 Another benefit that i did not speak of is that you get type safety for

the index. i.e. you cannot use a wrongly typed index:

indexed_array<char const*, user_messages> messages; std::array<char const*, nb_user_messages> messages_arr; // ... play_message(messages_arr[static_cast<int>(tech_message::goon)]); // boom at runtime play_message(messages[tech_message::goon]); // fails to compile

I also find the code clearer (the static_cast just adds noise in my opinion), but that would be a matter of taste.

Regards,

Julien

On Fri, 24 Jun 2022 at 02:01, Hadriel Kaplan <hadrielk@yahoo.com> wrote:

On Thursday, June 23, 2022, 05:20:01 PM EDT, Richard Hodges via Boost < boost@lists.boost.org> wrote:

...

Yes, I'm looking for a real-world use case. The trivial hypothetical cases offered are building a static association between an enum value and some other value. This can be done with switch/case (for example), in about the same amount of typing, at compile time, in a function.

<snipped example switch-case code>

OK, now show me how to iterate over it. And how to get a view or slice of it (i.e., std::span or range). And how to get the number of elements. :)

I understand that presenting the type with the look and feel of a container is somehow pleasing. The ability might be an interesting academic goal. I am more interested in what real-world use-cases this solves. boost.describe, wise_enum etc solve the case where (for example) I wish to serialise and deserialise my enum value as text. This is useful in the real world as, for example, it allows me to easily and reliably implement a JSON communications protocol that contains my enum. Indexed array still has the maintenance burden of having to maintain both the enum and the associative "array" separately.

I mean ultimately, it's a container. It's not _only_ useful for key'ed lookup retrieval.

You can change the value of whatever the container holds too, for example. Or maybe I'm misunderstanding your point? (could easily be the case)

Anyway, the main use-case or value of it, in my mind anyway, is actually the one when using Boost.Describe. Because when you're using Describe for an enum, you only need to do this for indexed_array:

indexed_array<string, MyEnum> data;

That's powerful, because you do not need to repeat any code, do not need to repeat any sizing/number of enums, etc. If you add new enumerators to your enum, you don't need to change this instantiation of indexed_array. It's all automatic, and you can't screw up.

The arbitrary intervals and non-contiguous use-cases I'm less convinced by. It's too easy to mess up.

And arguably it's not really a "indexed_array" at that point anyway - the underlying container might happen to be a std::array, but that's an implementation detail. From the user's perspective it's more like a "static_map" or some such. Except... it's not actually necessarily sorted, is it? All the examples show them sorted, but I'm not sure it really is if the user doesn't define it that way? (I'm trying _not_ to read the code - only the docs)

Perhaps there is a case when seeking to parse a number of flags that are presented as text and each flag may not appear more than once? enum class NerfEnum : char { wounded = 'W', buffed = 'B'; burning = 'N'; }; BOOST_DESCRIBE_ENUM(NerfEnum, (wounded, buffed, burning)); JSON input: { ... "nerfs": [wounded, buffed, burning] ... } indexed_array might then be used to efficiently store the set of flags associated with this unfortunate character. But in this case, am I not better off with an unsigned int, where each flag is represented by a bit? The "natural code" might be: unsigned flags = 0; for(auto&& value : nerfsjson.as_array()) flags |= to_bit(*deserialise<NerfEnum>(value.as_string()); I could imagine that being able to express this bitset with a container-like interface might make my code more intuitive, but I don't think that indexed_array addresses this case. (I appreciate that such a container would have some of the same criticisms of vector<bool>)

-hadriel

[sorry this turned out to be a long email – hopefully my lovely corporate IT-approved mail client from some company in Redmond won’t botch this up] Ahh, right you want concrete examples of what one would need/do with this thing. Because yeah, it’s not to convert to a string name. People already do that, as we all know. OK, so here are some uses in my employer’s codebase – I’ll just list the first 10 I find, in the order my editor finds them: 1. A container of event-collectors, where the enum key represents event types, and the collectors are objects that stream them to various places or save them to files, etc., per type of event. 2. A container of loggers, where the enum key represents a logging type, and each logger is an object with its own log-level, filenames, etc. 3. A container of subprocess execution commands, where the enum key represents the command type, and the object knows the actual command string, args , etc. 4. A container of hashmaps, that map some VRF prefixes to some internal stuff, and the enum key represents route-advertisement types. (we’re a router company) 5. A container of test-inputs/outputs, where the enum key represents the test-input type, and the test-inputs are various routing messages we use in tests. (there’s actually a whole big bunch of these use-cases, so I’ll skip the rest) 6. A container of worker threads, where the enum key represents the thread name+purpose, and the thread is dedicated for that work. (used in a thread-chain) 7. A container of timer-wheels, where the enum key represents the resolution of the timer-wheel. 8. A container of stats/counters, where the enum key represents a specific stat/counter purpose. (there’s a whole bunch of these too, so I’ll skip the rest) 9. A container of another enum type, where the enum key represents some internal components, and the enum values represent their current state. (up, down, disabled, etc.) 10. A container of function objects, where the enum key represents a rule/action to take, and the function objects perform it. I skipped a bunch of self-similar ones in there, to give a better idea. Some of these are essentially global containers, and some are members of other objects. Obviously we could have created separate C++ variables for each instance instead, instead of using an std::array – because basically we’re using the enum key as a form of programmatic “name” for variables. The reason we do it this way instead, however, is because when you have many of these things it is far easier to write generic code to handle them this way. For example stats/counters: of course at the point in the code that needs to increment/decrement them, that line of code has to identify a specific stat it wants to increment/decrement/whatever, and using an enum to do so vs. variable name is not much different. But when you write the observers of those stats, that need to iterate over them to write them somewhere else, or display them, or reset them all, or whatever, it’s a lot more convenient if they’re already in a container. That code doesn’t need to know variable names. It doesn’t even need to know specific enum values. It just has to know how to access the container. And we have thousands of stats (as do our competitors, and other industries too I’m sure). Obviously some people “register” individual stats into observers, which is fine too, but we don’t need to do that for every stat. Same goes for test inputs/outputs. We don’t write every test by-hand, for each and every test-input/output we need. We just have generic tests that iterate over all inputs, and verify all outputs. (again, I’m sure a lot of people do this – it’s impractical to do otherwise) The other advantage to using enums for these types of things, is once you have some mechanism to convert enums-to-strings, you can use those strings too for the generic code aspects. For example in the test inputs/outputs, the enum’s string is used to print out what’s being tested and is written to the test log, so if it fails we know which one. (googletest does this as well anyway via their macros stringizing stuff, but I’m just mentioning it) Another example is that array of threads – the enum strings become the thread string names used in logging for those threads, for example. And you can do the reverse too – if you can convert strings-to-enums, then you can use strings to find/match entries in those arrays – strings such as might be in your config for example, or even from a command-line. -hadriel From: Boost <boost-bounces@lists.boost.org> on behalf of Richard Hodges via Boost <boost@lists.boost.org> Date: Friday, June 24, 2022 at 12:31 AM I understand that presenting the type with the look and feel of a container is somehow pleasing. The ability might be an interesting academic goal. I am more interested in what real-world use-cases this solves. boost.describe, wise_enum etc solve the case where (for example) I wish to serialise and deserialise my enum value as text. This is useful in the real world as, for example, it allows me to easily and reliably implement a JSON communications protocol that contains my enum.

Le vendredi 24 juin 2022 à 12:46 +0000, Hadriel Kaplan via Boost a écrit :

By the way Julien, I was going to suggest something eventually that might be… controversial, but since Richard brought them up I’ll mention it now:

I think this thing should automatically handle more than just Boost.Describe. I think it should try to handle magic_enum and wise_enum’s generated-enum-info too.

That would make it usable by a much broader user base.

That should not be too hard to do, but could require some additions in these libraries. Basically, what is needed is a way to transform the enumeration values list into a corresponding integer list, which then gets feeded to the default_indexer – and of course a way to do compile- time detection of the fact that the enum has some introspection (like has_describe_enumerators). At that point, all the machinery is already in place. For example, describe integration is only 25 lines of code. Out of the box this should already work for wise-enum: indexed_array<X,  interval<wise_enum::enumerators<MyEnum>::range.front(), wise_enum::enumerators<MyEnum>::range.back()>> But there is no contiguity check, and it would be much simpler to write just indexed_array<X, MyEnum>.

If that’s technically challenging to do, then I think you should approach the authors of them and ask to “normalize” whatever needs to be the same, to make them all usable with this.

I don’t know how they’d respond, but it would be really cool if people could start writing things like index_array that supports all of them.

That would indeed be a nice addition. I'll try to see what can be done here. Best regards, Julien

Le vendredi 24 juin 2022 à 12:46 +0000, Hadriel Kaplan via Boost a écrit :

By the way Julien, I was going to suggest something eventually that might be… controversial, but since Richard brought them up I’ll mention it now:

I think this thing should automatically handle more than just Boost.Describe. I think it should try to handle magic_enum and wise_enum’s generated-enum-info too.

That would make it usable by a much broader user base.

(replying off-list to avoid making too much noise) Just for information, that's done (for both magic_enum and wise enum). Adapting for others enum reflection libraries should be pretty straightforward now with these three examples. Best regards and thanks for supporting the library ! Julien

Yes, I'm looking for a real-world use case. The trivial hypothetical cases offered are building a static association between an enum value and some other value. This can be done with switch/case (for example), in about the same amount of typing, at compile time, in a function.

<snipped example switch-case code> OK, now show me how to iterate over it. And how to get a view or slice of it (i.e., std::span or range). And how to get the number of elements. :) I mean ultimately, it's a container. It's not _only_ useful for key'ed lookup retrieval. You can change the value of whatever the container holds too, for example. OR maybe I'm misunderstanding your point? (could easily be the case) Anyway, the main use-case or value of it, in my mind anyway, is actually the one when using Boost.Describe. Because when you're using Describe for an enum, you only need to do this for indexed_array: indexed_array<string, MyEnum> data; That's powerful, because you do not need to repeat any code, do not need to repeat any sizing/number of enums, etc. If you add new enumerators to your enum, you don't need to change this instantiation of indexed_array. It's all automatic, and you can't screw up. The arbitrary intervals and non-contiguous use-cases I'm less convinced by. It's too easy to mess up. And arguably it's not really a "indexed_array" at that point anyway - the underlying container might happen to be a std::array, but that's an implementation detail. From the user's perspective it's more like a "static_map" or some such. Except... it's not actually necessarily sorted, is it? All the examples show them sorted, but I'm not sure it really is if the user doesn't define it that way? (I'm trying not to read the code - only the docs) -hadriel

Le 2022-06-23 22:41, Hadriel Kaplan a écrit : Hi, Thanks for this feedback.

Ours is purely for enum-based indexing, and only for scoped enums that start at value 0 and increase sequentially – i.e. contiguous, no “holes”. We happen to be able to verify that property of enums at compile-time, and can static_assert that it’s true, so ours just ends up doing a static_cast under-the-hood. (I can’t quite tell what indexed_array does in the end, since it also deals with non-contiguous index values.)

The current strategy is the following: * detect if the set of indexing values is contiguous (ie, no holes). Duplicate values are removed (this is needed to support enums with aliases such as first or last). * If that's the case, use a static_cast and an offset for index computation. If the offset happens to be zero, this is fully optimized by the compiler. * if that's not the case, the current implementation use an mp_foreach loop (iterate over all indexes to recompute the index in the underlying array). This detection is performed at compile time, and has been firstly designed to support described-enums, which we get as an integer list-like. Supporting them without run-time penalty was a design goal. There is indeed a performance penalty with using non-contiguous enums with the default indexer. Since several of my use cases for non-contiguous enums involves mostly either two discontinuous ranges, or a single range + a few single values, i may improve these specific cases in the future (this is an implementation detail that will not change the client-side usage of the library). Regards, Julien