Le 14/12/2016 à 11:20, Florian Lindner a écrit :

Am 12.12.2016 um 09:24 schrieb Florian Lindner:

...

Hello,

I need to conditionally decativate tests and use a custom decorator for that, like that:

virtual void apply(bt::test_unit& tu) { [...] if (std::find(_ranks.begin(), _ranks.end(), rank) == _ranks.end()) { std::cout << rank << " Disable, because of rank, test: " << tu.full_name() << std::endl; tu.p_default_status.value = bt::test_unit::RS_DISABLED; return; } }

(i'm happy to provide full source code, if anyone is interested) As you can see it's deativated on some MPI ranks only.

However, if a user uses -t "testname" it is activated again.

Is there a way to really deactivate the test? Or delete it from the test tree?

Seems to be next to impossible:

test_suite.remove(tu.p_id); has no effect.

bt::framework::deregister_test_unit(&tu); causes a SIGSEGV within boost.

tu.p_default_status.value = bt::test_unit::RS_DISABLED; as I said above, only disables if not enabled otherwise.

Any ideas anyone to completely disable / delete a test case from within a decorator?

Thanks, Florian

/// Boost.Test decorator that makes the test run only on specfic ranks class OnRanks : public bt::decorator::base { public:

explicit OnRanks(const std::vector<int> & ranks) : _ranks(ranks) {}

private:

virtual void apply(bt::test_unit& tu) { size_t rank = precice::utils::Parallel::getProcessRank(); size_t size = precice::utils::Parallel::getCommunicatorSize();

if (_ranks.size() > size) { auto& test_suite = bt::framework::master_test_suite(); // std::cout << "Removed: " << tu.full_name() << " with ID " << tu.p_id // << " from test suite " << test_suite.full_name() << std::endl; test_suite.remove(tu.p_id); // bt::framework::deregister_test_unit(&tu); // tu.p_default_status.value = bt::test_case::RS_DISABLED; return; }

if (std::find(_ranks.begin(), _ranks.end(), rank) == _ranks.end()) { auto& test_suite = bt::framework::master_test_suite(); // std::cout << "Removed: " << tu.full_name() << " with ID " << tu.p_id // << " from test suite " << test_suite.full_name() << std::endl; test_suite.remove(tu.p_id); // bt::framework::deregister_test_unit(&tu); // tu.p_default_status.value = bt::test_case::RS_DISABLED; return; } }

virtual bt::decorator::base_ptr clone() const { return bt::decorator::base_ptr(new OnRanks(_ranks)); }

std::vector<int> _ranks;

};

What about, instead of trying to remove a test when some conditions are met, adding it when those conditions are not met? I would rather go for an easy solution that is just declaring the test in the init_ of boost.test: http://www.boost.org/doc/libs/1_62_0/libs/test/doc/html/boost_test/adv_scena... and then for declaring tests manually: http://www.boost.org/doc/libs/1_62_0/libs/test/doc/html/boost_test/tests_org... The disabled tests features are by design this way: we want some tests disabled by default except on some conditions, and we want to be able to override those conditions from the runtime command line. If a test *cannot* run because of some conditions (say GPU is of the wrong type), then it should be disabled on this condition as you are doing with your class, but also the test itself need to check that (like catching an error because of the instance of the GPU is of the wrong type). For your case, it can just be a BOOST_TEST_REQUIRE on the right ranking (if needed). Hope this helps, Raffi

Le 16/12/2016 à 16:31, Florian Lindner a écrit :

Am 14.12.2016 um 17:25 schrieb Raffi Enficiaud:

...

Le 14/12/2016 à 11:20, Florian Lindner a écrit :

...

Am 12.12.2016 um 09:24 schrieb Florian Lindner:

...

Hello,

I need to conditionally decativate tests and use a custom decorator for that, like that:

virtual void apply(bt::test_unit& tu) { [...] if (std::find(_ranks.begin(), _ranks.end(), rank) == _ranks.end()) { std::cout << rank << " Disable, because of rank, test: " << tu.full_name() << std::endl; tu.p_default_status.value = bt::test_unit::RS_DISABLED; return; } }

(i'm happy to provide full source code, if anyone is interested) As you can see it's deativated on some MPI ranks only.

However, if a user uses -t "testname" it is activated again.

Is there a way to really deactivate the test? Or delete it from the test tree?

Seems to be next to impossible:

test_suite.remove(tu.p_id); has no effect.

bt::framework::deregister_test_unit(&tu); causes a SIGSEGV within boost.

tu.p_default_status.value = bt::test_unit::RS_DISABLED; as I said above, only disables if not enabled otherwise.

You cannot do that: - inside a precondition, if you start modifying the test tree, this would lead to hazardous behaviour - the apply is not meant to be used. The public api is the one described here: http://www.boost.org/doc/libs/1_62_0/libs/test/doc/html/boost_test/tests_org...

...

...

Any ideas anyone to completely disable / delete a test case from within a decorator?

[snip]

We have a lot of tests, and there are many which should just run on one rank as well as many that run on multiple ranks, testing parallelized scenarios. I would REALLY like to go with automatic test registration, instead of declaring so many tests in the init function.

Ok

...

[snip]

I understand your design rationale. Please try to understand mine. The precondition decorator does what I need, except that is flags the test as failed if unmet. I just require a method to skip / delete / do not execute a test if a certain runtime condition is met and do NOT flag this test as failed.

Ok, this is a bug in the backlog: https://svn.boost.org/trac/boost/ticket/12095 I believe solving this would satisfy your requirements. Best, Raffi