- Boost - lists.preview.boost.org

[test] Applying outstanding pull requests to develop and schedule merge of develop to master
by Jamie Allsop 26 Dec '14

26 Dec '14

Some time ago (more than 2 years) I submitted a set of four complimentary patches for Boost.Test of which one was partially applied. The tickets are: * Test Units (Cases and Suites) in Boost.Test do not capture __FILE__ and __LINE__ at declaration point making it impossible to provide source file linking using external test management tools https://svn.boost.org/trac/boost/ticket/7410 (modified patch applied) * Boost.Test, since boost 1.48 is using the deprecated Boost.Timer class - it should be updated to use the new class https://svn.boost.org/trac/boost/ticket/7397 https://github.com/boostorg/test/pull/15 * Detailed test status is not available in the Boost.Test log (status, assertions, passed) and so live test case status cannot be tracked https://svn.boost.org/trac/boost/ticket/7417 https://github.com/boostorg/test/pull/16 * Support multiple calls to framework::init() allowing wrappers to support running tests using test tools in full systems https://svn.boost.org/trac/boost/ticket/7419 https://github.com/boostorg/test/pull/17 I've updated and resubmitted patches on more than one occasion and also variously created patches against (the now) develop and master so I can actually apply them for myself for each new boost version. Now, the reason for my post is two-fold. First - I'd like to see these patches applied. They cause no user impacts and simply (a) fix broken functionality and (b) *significantly* improve the usefulness of Boost.Test. I am more than happy to help with maintaining Boost.Test since we rely on it heavily and I believe, on the whole it is a good library. Second - it is about time we formulated a schedule for merging develop into master for Boost.Test. Otherwise all the patches in the world are simply pointless. The patches I've submitted, like others will never see the light of day even if they are applied because develop is very different than master and it looks like a merge may never happen. It should be noted that for my patches I have a completely different set that I apply to an actual Boost release because of the large differences between the two branches. To recap - it is time to start actively shepherding patches into Boost.Test develop, but more importantly, it is time we started looking at a schedule for merging Boost.Test develop to master. The longer it is left the worse the impact will be. A lot of good work has been done on develop - the longer it stays there the more people will move away from Boost.Test in their own code and it will only have relevance to Boost libraries themselves. Jamie

3 6

[Test] Command line option --list_content
by Andreas Noll 26 Dec '14

26 Dec '14

Hi everyone, Within our company we heavily use Boost.Test. We use Visual Studio and have written an Visual Studio extension to show all test suites and test cases of a given test executable. This allows our developers to run all tests or only parts of them. This is necessary because we have some lengthy running tests - one could say they are more integration tests than unit tests. What we have to do currently is patching the Boost.Test code to add a command line switch to output a test executable's contents to stdout. The output shows the hierarchy of test suites and test cases and is consumed by our Visual Studio extension. As I have recognized, our patch seems to cover the same functionality as the --list_content switch that already exists in the trunk version of Boost.Test. We would really appreciate that --list_content becomes part of the official boost releases to make our patch obsolete. My questions: 1. What are the reason(s) why-list_content is not (yet) added to the release? Is there any schedule for its release? 2. Is there any way how we can help with that issue? I'm new to the community but maybe there is something our team or me can do to get the feature released. Best regards, Andreas Noll

2 1

[compute] kernels as strings impairs readability and maintainability
by Mathias Gaunard 25 Dec '14

25 Dec '14

Hi, While reading through the code of Boost.Compute to see what it does and how it does it, I often found that the approach used by the library of putting all OpenCL kernels inside of strings was an annoying limitation and made it quite difficult to reason with them, much less debug or maintain them. This has a negative effect on the effort needed to contribute to the library. Has separate compilation been considered? Put the OpenCL code into .cl files, and let the build system do whatever is needed to transform them into a form that can be executed. This would also make it easier to eventually provide a CUDA backend.

7 12

[core] [addressof.hpp] [addressof_test2.cpp] ISO C++ says that these are ambiguous
by Sergey Sprogis 25 Dec '14

25 Dec '14

It looks like he following code fragment from boost/core/addressof.hpp (latest trunk version) violates ISO C++ overloading rules: template<class T> struct addressof_impl { static BOOST_FORCEINLINE T * f( T & v, long ) { return reinterpret_cast<T*>( &const_cast<char&>(reinterpret_cast<const volatile char &>(v))); } static BOOST_FORCEINLINE T * f( T * v, int ) { return v; } }; As a result, libs/core/test/addressof_test2.cpp fails to compile with Oracle Studio 12.4 C++ compiler while gcc-4.8.2 produces just warning here. It's easy to reproduce that behavior of both compilers with 26 lines test case listed below: gcc -c t.cc t.cc:7:33: warning: ISO C++ says that these are ambiguous, even though the worst conversion for the first is better than the worst conversion for the second: [enabled by default] return addressof_impl<T>::f(v,0); ^ CC -c t.cc "t.cc", line 7: Error: Overloading ambiguity between "static addressof_impl<convertible2>::f(convertible2&, long)" and "static addressof_impl<convertible2>::f(convertible2*, int)". t.cc ==================================== template<class T> struct addressof_impl{ static T *f(T &v,long){} static T *f(T *v,int){} }; template<class T> T *addressof(T &v){ return addressof_impl<T>::f(v,0); } template<class T> void scalar_test(T* =0){ T *px=new T(); T &x =*px; (addressof(x)==px)? 1 : 0; } class convertible{ public: convertible(int=0){} }; class convertible2{ public: convertible2(int=0){} operator convertible2 *()const{ return 0; } }; void foo(){ scalar_test<convertible>(); scalar_test<convertible2>(); } ============================== It would be nice if boost/core/addressof.hpp file will be changed somehow to fit the proper ISO C++ requirements. Sergey

3 2

[compute] Some questions
by Andrey Semashev 23 Dec '14

23 Dec '14

Hi, I have no experience with OpenCL or GPU computing in general, so bear with me if my questions sound silly. I have a few questions regarding Boost.Compute: 1. When you define a kernel (e.g. with the BOOST_COMPUTE_FUNCTION macro), is this kernel supposed to be in C? Can it reference global (namespace scope) objects and other functions? Other kernels? 2. When is the kernel compiled and uploaded to the device? Is it possible to cache and reuse the compiled kernel? 3. Why is the library not thread-safe by default? I'd say, we're long past single-threaded systems now, and having to always define the config macro is a nuisance. 4. Is it possible to upload the data to process to the device's local memroy from a user-provided buffer, without copying it to boost::compute::vector? Same for downloading. What I'd like to do is move some of data processing to the GPU while the rest is performed on the CPU (possibly with other libraries), and avoid excessive copying. 5. Is it possible to pass buffers in the device-local memory between different processes (on the CPU) without downloading/uploading data to/from the CPU memory? 6. Is it possible to discover device capabilities? E.g. the amount of local memory (total/used/free), execution units, vendor and device name? Thanks.

3 4

[compute] Review period extended till December 30 + sum up
by Antony Polukhin 23 Dec '14

23 Dec '14

Hi, Review period of Compute library extended till December 30. Some notes and answers to them that were pointed out during review so far: * Some of the algorithms could be tuned for a specific hardware - Library author is working on a auto-tuning solution and improves existing algos * Why OpenCL C API used instead of C++ API? - C++ API had issues, C API allows a bit more control. No C++ API for OpenCL 2.0 * API is not N4105 compatible, library is not N*** compatible - This could be easily fixed by a library that will use Compute as a backend * Some of the algorithms return futures while other work with command queues - This is because of the OpenCL design + async chains/futures would be probably added later * Not all the types of the Khronos API are supported - A few types are not documented yet, there's a patch in the works which would allow the Khronos C++ types to be passed into Compute algorithms * A few type related errors could be detected at compile time - This will be fixed, more asserts would be added to runtime only checkable places. Report issues to the tracker https://github.com/kylelutz/compute/issues * On small data sets CPU algorithm would work faster than a GPGPU. How about implicitly dispatching algo on CPU in those cases? - The call on whether to execute the algorithm should be left up to the user. While library author agrees that this would be a useful feature, he just don't think Compute is the right place for that logic. * How about providing way to do chains of async operations - This is a big task that will be solved some day. * How about providing Boost.ASIO like error handling via throw and error_code - Implementing an approach like ASIO's wouldn't be that difficult. Thanks to all the reviewers for spending their time and providing useful comments so far! -- Best regards, Antony Polukhin

1 0

[compute] Define user functions using lambda expressions
by Asbjørn 22 Dec '14

22 Dec '14

Hi, I'm probably missing something obvious, but how would I go about making a custom function using a lambda expression? Say I want to declare some function "sqr" which returns "_1 * _1", and then use this in a transform() lambda expression, ie transform(..., 2.0 * sqr(_1) - 3.0 * _1 + 5.0, ...); to evaluate 2x^2 - 3x + 5. If I try to use BOOST_COMPUTE_FUNCTION() I can't use the resulting function in a lambda expression it seems. Cheers - Asbjørn

2 2

[compute] some comments
by Mathias Gaunard 22 Dec '14

22 Dec '14

Hi, I do not know if I'll be doing a full review, but I have some comments from having skimmed through the code and documentation. From a quick glance, from all the algorithms that are supported, quite a few are not easy to implement with the OpenCL programming model efficiently. I think the documentation should say more about the implementation strategies of each algorithm, its complexity and how much it is parallelized.

2 1

[icl] insert_iterator and elements
by Jeff Flinn 22 Dec '14

22 Dec '14

interval_set has overloaded insert methods for both intervals and elements. Is there a reason there insert iterator lacks this ability. The line in the code below *itr = 4; fails to compile. This could be made to compile by: - adding an implicit constructor 'interval( const element& )' for the underlying (discrete) intervals, initializing to [element, element+1), or [element, element]. - adding an overloaded operator=( const element& ) - adding an element_insert_iterator class I've encountered 2 domains now where I need to construct an interval_set<int> from a range of ints. The most natural ways to accomplish this (for me) are: boost::copy( ints, inserter(int_interval_set) ); or boost::copy_range<int_interval_set>( ints ); Of course I can transform the incoming ints, but this seems a common enough usage pattern for the library to provide. Especially since the lack of this facility is asymmetric to the insert overloads of interval_set. #include <iostream> #define BOOST_ICL_USE_STATIC_BOUNDED_INTERVALS #include <boost/icl/interval_set.hpp> #include <boost/range/algorithm/copy.hpp> int main() { typedef boost::icl::interval_set<int> IdSet; typedef IdSet::interval_type IdInterval; IdSet ids; ids.insert(3); boost::icl::insert_iterator<IdSet> itr(ids, ids.end()); *itr = IdInterval(1,2); //*itr = 4; //compile error no viable operator = boost::copy(ids, std::ostream_iterator<IdInterval>(std::cout)); std::cout << "\n"; return 0; }

1 0

[compute] review
by Pavan Yalamanchili 22 Dec '14

22 Dec '14

*Design and Implementation* The library provides an STL-like library for OpenCL devices. Although there are other libraries offering similar behavior, Boost.Compute is the most complete and the most generalized. The directory structure is well organized and easy to dig through to understand the implementation or debug an issue. Others have pointed out the existence of the OpenCL C++ wrapper and the fact that Compute library does not use it. Here are a few reasons why I think the author was *right* in his design decisions. - The C++ wrapper begins to look similar to the original OpenCL C interface once you start using it in a generalized fashion. - cl.hpp is monolithic (single file 12k+ LOC) and is not modular enough to only include the parts that you need. - Compute and cl.hpp have two different goals. The biggest selling point of Compute is the set of algorithms it supports not its C++ wrapper around OpenCL. Having said that, I think the Compute library might be better off with an additional interface that accepts the OpenCL-C++ wrapped objects. *Documentation* The documentation is comprehensive and the API is documented well. There are a few improvements that can be done. For example, the reference is at the bottom of the TOC and is hard to see immediately. The API reference in the TOC can also be expanded out a bit more to show the general categories of algorithms that are supported. *Potential Usefulness* It is fairly easy to transition from applications using vector algorithms in STL to Compute. The wide availability of OpenCL devices makes the library useful to a large user base. *Domain Knowledge* I am a developer of the ArrayFire library[1]. I am the lead engineer of a company that specializes in this domain. I consider myself to be fairly knowledgeable in this domain. *Experience with the library* ArrayFire depends on Boost.Compute for a few algorithms in our OpenCL backend. We explicitly and implicitly (via ArrayFire) test Boost.Compute on a variety of hardware / compilers / operating systems. Compilers and Operating systems we use: - gcc 4.8, 4.9 on various Linux distributions - clang 3.4 on OSX - Visual Studio 2013 on Windows. We have found some bugs through our usage that have been mostly resolved by the author or by patches sent by us. *Conclusion* The experience overall has been fairly positive. But there are certainly *some* rough edges that need resolving. An issue that will plague any OpenCL library is the performance portability across various devices. It would be nice if the author can talk about how he plans to eventually address this issue. Overall, Boost.Compute will be a great addition to Boost. But before it is accepted the following issues need to be addressed. - The tests need to be a bit more comprehensive. - Due to the general nature of OpenCL, there needs to be a list of "officially supported" devices. - Make sure all the tests are passing on the supported devices. *Links* [1] https://github.com/arrayfire/arrayfire

2 1