for_each combination/permutation - call for interest

On and off for the past 7 months (well, mostly off), I've been working on a few combination/permutation algorithms. I was inspired by the CUJ article written by John Dibling in the Feb. 2005 issue titled "Extending the STL". The std::lib currently has next_permutation, perv_permutation. But these functions are lacking in two respects: 1. They only allow the entire sequence to be permuted. They do not allow (for example) the ability to find all permutations of 5 items considered only 2 at a time. Consider 5 items 2 at a time from this sequence 1 : a b c d e Permutations 1 : a b 2 : a c 3 : a d 4 : a e 5 : b a 6 : b c 7 : b d 8 : b e 9 : c a 10 : c b 11 : c d 12 : c e 13 : d a 14 : d b 15 : d c 16 : d e 17 : e a 18 : e b 19 : e c 20 : e d When you know that you need to consider only a few items at a time, std::next_permutation is excessively expensive as it will loop through many more iterations. (e.g. consider the permutations of 5 items 5 at a time leads to a list of 120 - vs 20 sequences). 2. std::next/prev_permutation require a sorted sequence and an ordering predicate. This is not always convenient. In addition to permutations of N objects taken K at a time, other useful algorithms would be: * Circular permutations of N objects taken K at a time. * Combinations of N objects taken K at a time. * Reversible permutations of N objects taken K at a time. * Reversible circular permutations of N objects taken K at a time. Out of these 5 algorithms, today I can only present the first 3. The reversible versions are meant to take a sequence over all permutations where an order, and the reverse order are considered to be the same sequence. E.g. 1 2 3 4 is the same permutation as 4 3 2 1 for a "reversible" permutation. This is handy in situations where you are scoring "distances" between items in the list, and it doesn't matter which direction the list is traveled in. Unfortunately I haven't yet found a way to efficiently compute these "reversible" permutations. --- Interface --- I've followed the advice of John Dibling in the CUJ article except that the functor doesn't receive the entire range, but only the first K elements. For example consider: char array[] = {'a', 'b', 'c', 'd', 'e'}; unsigned size_array = sizeof(array)/sizeof(array[0]); const int k = 3; print p = std::for_each_combination(array, array + size_array, k, print()); This calls the print() functor (say p) like: p(array, array+k); for array permuted such that [array, array+k) will hold each combination of 3 items taken from the total of 5 items. And after the for_each_combination call, array will be in its original order. While always permuting the elements into positions [0, k) may at first seem limiting in how the for_each_* algorithms can be implemented, it offers interesting possibilities for the functors. If a functor knows the total length of the sequence (say N), and receives [array, array+k) as a subsequence representing the current combination/permutation, that functor can then call for_each_* on the remaining sequence [array+k, array+N). This allows for a very complex analysis where one considers all combinations (or permutations) of the first k elements, and then simultaneously considers permutations of the remaining elements for each of the outer combinations. One could use this capability to search for solutions to variations of the traveling salesman problem. Instead of minimizing the route of 1 salesman over N cities. You might want to minimize two salesmen's routes over the same set of cities, each city being visited by one and only one salesman. For example you might consider: For each circular-reversible permutation of 10 cities considered 5 at a time: Compute distance around the 5 city route and add to that: For each circular-reversible permutation of the remaining 5 cities considered 5 at a time: Compute distance around the 5 city route. Record minimum combined distance and routes. Naturally this technique can easily be generalized to M salesmen covering N cites, N/M cites covered by each salesman. Unfortunately I currently lack the circular-reversible permutation algorithm and have to instead use the twice-as-expensive circular permutation algorithm. The three algorithms I do have are here: #include <iterator> #include <algorithm> namespace test { namespace detail { template<class BidirectionalIterator, class Function, class Size> Function permute(BidirectionalIterator first, BidirectionalIterator mid, BidirectionalIterator active, BidirectionalIterator last, Size k, Function f, Size n) { if (k == 0) f(first, mid); else { using std::swap; BidirectionalIterator activep1 = active; ++activep1; BidirectionalIterator p = active; ++p; for (Size i = 0; i < n; ++i, ++p) { if (k == 1) f(first, mid); else f = permute<BidirectionalIterator,Function,Size>(first, mid, activep1, last, k-1, f , n-1); if (i != n-1) swap(*active, *p); } std::rotate(active, activep1, last); } return f; } template<class BidirectionalIterator, class Function, class Size> Function combine(BidirectionalIterator first, BidirectionalIterator mid, BidirectionalIterator active, BidirectionalIterator last, Size k, Function f, Size n) { if (k == 0) f(first, mid); else { Size e = n - k + 1; Size nn = n-1; BidirectionalIterator l2 = last; BidirectionalIterator ap1 = active; ++ap1; BidirectionalIterator apk = active; std::advance(apk, k); for (Size i = 0; i < e; ++i, --nn, --l2) { if (k == 1) f(first, mid); else f = combine<BidirectionalIterator,Function,Size>(first, mid, ap1, l2, k-1, f, nn); std::rotate(active, i < e-1 ? ap1 : apk, last); } } return f; } } // detail template<class BidirectionalIterator, class Function, class Size> Function for_each_permutation(BidirectionalIterator first, BidirectionalIterator last, Size k, Function f) { typedef typename std::iterator_traits<BidirectionalIterator>::difference_type difference_type; Size n = static_cast<Size>(std::distance(first,last)); if (k < 1 || n == 0) return f; if (k > n) k = n; BidirectionalIterator mid = first; std::advance(mid, k); return detail::permute<BidirectionalIterator,Function,Size>(first, mid, first, last, k, f, n); } template<class BidirectionalIterator, class Function, class Size> Function for_each_circular_permutation(BidirectionalIterator first, BidirectionalIterator last, Size k, Function f) { typedef typename std::iterator_traits<BidirectionalIterator>::difference_type difference_type; Size n = static_cast<Size>(std::distance(first,last)); if (k < 1 || n == 0) return f; if (k > n) k = n; BidirectionalIterator mid = first; std::advance(mid, k); BidirectionalIterator f2 = first; ++f2; for (Size i = 0, e = n - k + 1; i < e; ++i, --n, ++first, ++f2, ++mid) f = detail::permute<BidirectionalIterator,Function,Size>(first, mid, f2, last, k-1, f, n-1); return f; } template<class BidirectionalIterator, class Function, class Size> Function for_each_combination(BidirectionalIterator first, BidirectionalIterator last, Size k, Function f) { typedef typename std::iterator_traits<BidirectionalIterator>::difference_type difference_type; Size n = static_cast<Size>(std::distance(first,last)); if (k < 1) return f; if (k > n) k = n; BidirectionalIterator mid = first; std::advance(mid, k); return detail::combine<BidirectionalIterator,Function,Size>(first, mid, first, last, k, f, n); } } // test And here's a demo that exercises all 3 algorithms: // demo #include <iostream> #include <iterator> #include <algorithm> class print { public: print() : i_(0) {} unsigned get() const {return i_;} template <class It> void operator()(It first, It last) { typedef typename std::iterator_traits<It>::value_type value_type; std::ostream_iterator<value_type> out(std::cout, " "); std::cout << ++i_ << "\t: "; std::copy(first, last, out); std::cout << '\n'; } private: unsigned i_; }; int main() { char array[] = {'a', 'b', 'c', 'd', 'e'}; unsigned size_array = sizeof(array)/sizeof(array[0]); const int k = 3; std::cout << "Consider " << size_array << " items " << k << " at a time from this sequence\n"; print()(array, array + size_array); { std::cout << "Permutations\n"; print p = test::for_each_permutation(array, array + size_array, k, print()); std::cout << "--- Total of " << p.get() << " calls ---\n"; std::cout << "Sequence left in orginal order\n"; print()(array, array + size_array); std::cout << '\n'; } { std::cout << "Circular Permutations\n"; print p = test::for_each_circular_permutation(array, array + size_array, k, print()); std::cout << "--- Total of " << p.get() << " calls ---\n"; std::cout << "Sequence left in orginal order\n"; print()(array, array + size_array); std::cout << '\n'; } { std::cout << "Combinations\n"; print p = test::for_each_combination(array, array + size_array, k, print()); std::cout << "--- Total of " << p.get() << " calls ---\n"; std::cout << "Sequence left in orginal order\n"; print()(array, array + size_array); std::cout << '\n'; } } The output should be: Consider 5 items 3 at a time from this sequence 1 : a b c d e Permutations 1 : a b c 2 : a b d 3 : a b e 4 : a c b 5 : a c d 6 : a c e 7 : a d b 8 : a d c 9 : a d e 10 : a e b 11 : a e c 12 : a e d 13 : b a c 14 : b a d 15 : b a e 16 : b c a 17 : b c d 18 : b c e 19 : b d a 20 : b d c 21 : b d e 22 : b e a 23 : b e c 24 : b e d 25 : c a b 26 : c a d 27 : c a e 28 : c b a 29 : c b d 30 : c b e 31 : c d a 32 : c d b 33 : c d e 34 : c e a 35 : c e b 36 : c e d 37 : d a b 38 : d a c 39 : d a e 40 : d b a 41 : d b c 42 : d b e 43 : d c a 44 : d c b 45 : d c e 46 : d e a 47 : d e b 48 : d e c 49 : e a b 50 : e a c 51 : e a d 52 : e b a 53 : e b c 54 : e b d 55 : e c a 56 : e c b 57 : e c d 58 : e d a 59 : e d b 60 : e d c --- Total of 60 calls --- Sequence left in orginal order 1 : a b c d e Circular Permutations 1 : a b c 2 : a b d 3 : a b e 4 : a c b 5 : a c d 6 : a c e 7 : a d b 8 : a d c 9 : a d e 10 : a e b 11 : a e c 12 : a e d 13 : b c d 14 : b c e 15 : b d c 16 : b d e 17 : b e c 18 : b e d 19 : c d e 20 : c e d --- Total of 20 calls --- Sequence left in orginal order 1 : a b c d e Combinations 1 : a b c 2 : a b d 3 : a b e 4 : a c d 5 : a c e 6 : a d e 7 : b c d 8 : b c e 9 : b d e 10 : c d e --- Total of 10 calls --- Sequence left in orginal order 1 : a b c d e Note that the functor (print() in this case) can be stateful, and that state is accumulated from call to call, and returned back to the client. In this case the state is a simple line number counter. Anyway, I offer these up on the chance they might be useful, and on the chance that someone might help complete the library to compute "reversible" permutations, which may indeed be quite useful (as in the traveling salesman problem, and variations of it). -Howard

On 9/5/05, Howard Hinnant <hinnant@twcny.rr.com> wrote:
On and off for the past 7 months (well, mostly off), I've been working on a few combination/permutation algorithms.
I am very interested in this and will check out the code you provided in more detail once I finish my current undertakings. As well, a while back I made some algorithms very similar to your combinations algorithm, but with some slight variations. I decided that the combination size is, in practice, often known at compile-time, since it just about always impacts the design of the overall situation. Because of this, I chose to make the combination size be an explicit template argument which was used to provide more convenient calling of the passed-in function object. Instead of having the algorithms call the passed function object with two iterators, you have the option of either having them pass the arguments as a single reference to a complete array of pointers to the elements (which is often useful for metaprogramming), or they pass each argument as an explicit separate function argument (which is good since this is how functions usually are already written, meaning that you don't have to adapt them before using them with the algorithms). The obvious difference between this approach and yours is that this means in code that the size of the combination must be known at compile-time to be used, which is all that I ever encountered in practice. In fact, when originally writing the algorithms, I purposely put off making versions where the combination size wasn't known until runtime until I encountered a situation that needed it, which after a year or so, never happened. While at first having the combination size be a compile-time argument may sound like a downside to some, it actually has some very strong plusses, which is why I decided to make it as opposed to going towards a strictly runtime solution in the first place. Perhaps the main advantage, as I already mentioned, is that the combination size directly affects the conceptual number of arguments passed to the function object. Since my versions of the combination algorithm provide that size at compile-time, rather than always passing a pair of iterators to the function object from inside the algorithm, it can just pass the elements of the combination as separate arguments. This is very beneficial since more often than not, a version taking separate arguments already naturally exists and in order to make it take an iterator range, you would have to adapt it. That step of adaptation is therefore removed. For instance, one operation which I used was the collision of two objects in a physics simulation. I'll demonstrate using a function, though in actuality it was a type with overloaded operator (). void check_collision( object_type left, object_type right ); // 2 being the combination size (colliding two objects at a time) // though it can be left out to default to 2 for_each_combination< 2 >( my_objects.begin(), my_objects.size(), check_collision ); // where my_objects is a container of object_type This is nice, again, since the collision function, or whatever function you would use, is probably already naturally implemented to take the amount of parameters needed for the combination. It also generally wouldn't make sense to check for collision between anything other than two objects at once, so the need for a version which takes an iterator range is simply unnatural. Obviously here, and many other places, this is because the combination size is directly a part of the code's design known at compile-time, so it makes sense to have the argument as a template argument as opposed to being passed at runtime. Again, the downside of this is that combination size must be known at compile-time, though in real-world applications, that is typically all that I personally have ever encountered. Of course, this is not always the case, so both styles of the algorithm would be very nice to have handy. I was going to include the code with this post, but I switched compilers since I last used the algorithms, and after testing it just now, I have a dreaded internal compiler error which I'd like to clean up. I'm currently working on a number of other projects, so it may be a while before I go back and try to figure out what is wrong -- this post was more to point out that having a version with compile-time combination size passing is actually very beneficial, anyway, and the implementation is very trivial if you have any experience with template and preprocessor metaprogramming. -- -Matt Calabrese
participants (2)
-
Howard Hinnant
-
Matt Calabrese