# Boost.Outcome.v2 REVIEW REPORT In conclusion to the review for Boost.Outcome(v2) (19-Jan to 28-Jan, 2018): The library is ACCEPTED (conditions below). Acceptance is principally based on: (1) Reviewers found the library useful to address a current recognized need (2) New idioms are enabled that reviewers found compelling (3) The submission satisfies all Boost Library Requirements Concerns over accepting the library raised in the review discussion include: (a) New idioms proposed are not compelling (b) Requires C++14 (prefer compatibility with C++11) (c) Library may evolve beyond the current design (d) Library is unnecessarily complex Reviews submitted during the review period: *- Vinícius dos Santos Oliveira -- ACCEPT (Fri-26-Jan-2018) *- Andrzej Krzemienski -- ACCEPT (CONDITIONAL) (Fri-26-Jan-2018) *- Bjorn Reese -- ACCEPT (CONDITIONAL) (Sun-28-Jan-2018) *- Daniela Engart -- ACCEPT (Sun-28-Jan-2018) *- John P Fletcher -- ACCEPT (CONDITIONAL) (Sun-28-Jan-2018) In addition, after the review period concluded (but within the duration had the review period been extended) further reviews were submitted: *- Vinnie Falco -- REJECT (Tue-30-Jan-2018) *- Emil Dotchevski -- REJECT (Tue-30-Jan-2018) *- Glen Fernandes -- REJECT (Tue-30-Jan-2018) *- Paul A Bristow -- WOULD ACCEPT (?not a review?) (Wed-31-Jan-2018) *- Rob Stewart -- NOT ACCEPT (Sat-03-Feb-2018) Conditions for acceptance: It is expected: *- Changes are made to use Boost-standard names for macros and namespaces *- Docs are updated to be clear regarding: *- how library treats default-constructed values *- salient attributes of objects, and how “spare_storage” is treated *- Documentation is integrated into Boost framework and release process *- Library is distributed under the Boost Software License (BSL) The remainder of this report contains context and analysis contributing to this decision, including basis for why acceptance purports to be constructive and beneficial to the Boost community, and broader C++ community. # MOTIVATION: REAL-WORLD USE TODAY The prime motivation for acceptance is: *- Reviewers have real-world use cases _today_ for which they found Outcome to be an effective and best available alternative; and which is consistent with current-need and expectations; and which is consistent with ongoing C++ Standard evolution efforts.
From the Library Author: <quote> “Outcome is really an abstraction layer for setting per-namespace rules for when to throw exceptions. Exception throwing is absolutely at the heart of Outcome. That's why Outcome != Expected, and why it ICEs older compilers, and why C++ 14 is needed.”
However it looks like Outcome provides a solution for most of the
Exhibited behavior is two-fold: (1) Expected errors are handled locally with low overhead (e.g., deterministic / predictable with low-latency) (2) Unexpected errors are type-erased into an 'exception_ptr' and pushed up the call stack (e.g., exception-throw stack-unwind) For example, server-side code that is expected to handle a lot of failures, where “stop-the-world” is never suitable, may always handle errors locally. In contrast, most other system-wide code may 'throw' an error, which should be handled within some caller-context. # AT ISSUE: DESIGN AND IDIOMS What appears to be debated in this review are the Outcome library design and idioms; and not the quality of implementation (although the implementation is criticized as complex -- see below). For example, even reviewers that voted to reject commented that the library seems sound, and seems useful for some cases. This merits repeating: This review has the highly positive characteristic (despite the accompanying discomfort) of debating the idioms that challenge what already exists, and which forces re-evaluation of our historical approaches. Many of these discussions might otherwise be summarized (and which has a fair chance of being agreed upon by all parties) as: *- Error handling can be done with exceptions, or with branch-testing on error instances; and some algorithms or constraints may favor one over the other for technical or compositional reasons. It is this highly pragmatic observation that is at the core of the Outcome library submission. Outcome enables a new idiom consistent with other pattern explorations deemed by the C++ community as useful, as demonstrated through acceptance into the C++ standard for 'std::optional<>' and the pending 'std::expected<>'. Outcome enables value-transport of a multi-modal data object '<T|error|exception>' across API boundaries that provides the benefit of static compile-time type-checking, but places the explicit burden upon the user for 'visit / interrogation' of that multi-modal data object. Outcome is intended for use in program areas with harsh resource constraints, where many inconveniences are expected, and tradeoffs are part of the design decision for creating and handling explicit 'out' values that include failure information. These constraints may include: interaction among one-or-more modules compiled without exception-handling enabled; deterministic or low-latency execution requirements; and use of custom 'error' and/or 'exception' objects that are expected to be populated and tested within the local context (such as to perform explicit conditional branching for error handling or recovery). During the review there was some confusion regarding evaluating Outcome in the context of “generalized use” across an entire codebase, versus “localized use” where the algorithm intends to machine specific cases for statically-bound stateful handling of errors. For example, it is not expected that Outcome might be used across all APIs across all modules within a system. (Recall that current C++ community guidance might otherwise suggest 'std::error_code' or 'exception-throw' might be used across all APIs for all modules within a system, if such consistency is permitted.) Rather, Outcome intends to address that nexus where an algorithm interacts with several modules, each with differing 'error' and 'exception' practices, where the library consistently transfers '<T|error|exception>' with compile-time type enforcement and runtime behavior and performance guarantees. Design decisions within Outcome v2 are well-considered, and faithfully reflect feedback raised during the (quite extensive) v1 review. Indeed, none of the issues raised during the v1 review were again raised within the v2 review. And, it is noted that two reviewers previously voting REJECT on v1 now vote ACCEPT on v2. Further, it should be noted that Outcome v2 has evolved into a collaborative effort with assistance from several contributors, with validation and feedback in multiple real-world codebases with specific engineering constraints. In this review, reviewers found the idioms specifically useful (beyond alternatively available tools / idioms), and consistent with practices in existing codebases and in other languages. Further, the proposed direction appears to be consistent with evolving idiomatic practice within the C++ Standard enabling new idiomatic algorithm expression, as evidenced by multi-mode types such as 'std::optional<>' and the (proposed and evolving) 'std::expected<>'. Idioms provided through Outcome are also demonstrated to be useful / successful in other languages. As noted in the review, Outcome enables handling of failure with idioms similar to those found in Rust, a language which also supports RAII but which does not support exceptions (and reviewers commented that these idioms are sufficiently successful that exceptions are not viewed as an interesting nor desired mechanism in Rust). One reviewer commented that it is not compelling to attempt to replicate Rust error handling idioms, which (as a different language) naturally offers different idioms. And instead, offered that the natural idiom in C++ is to throw (and catch) exceptions. However, it can be noted that the Outcome design approach is somewhat agnostic to the use of 'exceptions', but rather relies upon a multi-modal data object that the user must explicitly unpack (similar to 'std::optional<>' or 'std::expected<>'), and which makes explicit guarantees about unified transport and handling of '<error|exception>'. The core of the dispute seems to be over the value of localized reasoning for handling errors: Exceptions are good for “distant” handling (to seamlessly transfer handling to a parent context), while explicit error/result instances (possibly customized) encourage local reasoning for discrete handling within the local algorithm context. Complicating the issue is that (of course) both are accepted practice in differing environments and with differing engineering constraints: Throwing exceptions may be most expressive to avoid algorithmic edge cases or ensure errors are actually handled; but alternatively it may be more expressive to explicitly handle errors locally when it is not desirable to delegate the failure to an upper level. # DISAGREEMENT: ERROR / EXCEPTION HANDLING Even prior to considering the Outcome library submission, error handling idioms and practices remain contentious and confusing within the C++ community (as well as within the Boost community). Recent threads on the Boost email-list continue to highlight the ongoing confusion and disagreement even over the proper or idiomatic use of 'std::error_code', which boasts a decade of real-world experience and approaches a decade in the C++standard. One might think such discussions should by now be resolved; but no, we observe that even semantic agreement of proper-or-intended behavior for 'std::error_code' was not reached on these recent Boost email discussions. We also observe that discussion regarding 'error' or 'exception' handling can at times be clouded within the evolving C++ Standard itself, with discussions of 'throw'/'no-throw' (e.g., “dual”) APIs, semantics for 'wide' or 'narrow' contracts, and even the occasional (and “over-simplified”) discussion of “for vs. against” exception handling (a topic that alone is worthy of consuming a surprising number of evenings and libations). The Library Author can be congratulated (or scolded) for exploring work or attempting Boost community review in such a contentious space. However, these topics are also fundamental motivations in the creation of Outcome itself: It provides a mechanism to unify 'error' _and_ 'exception' handling to enable unified localized reasoning across third-party modules; and directly exposes the resulting issues for cross-module error handling by charging the user with explicit unpacking of the '<T|error|exception>' object, but with the benefit of static binding (e.g., compile-time type checking). The implication is that in such a library review attempting to address a portion of the C++ landscape already marked with disagreement regarding current-and-alternative approaches, we cannot expect but to see charged commentary where issues of “global reasoning” over a “generalized use case” were inevitably raised to sometimes overshadow the discussion of a library that specifically intends to address “localized reasoning” with specific performance constraints and behavior guarantees. (Although compared to the v1 review, we might consider some threads to have missed opportunity to be yet more colorful.) The beclouded discussion appeared at times to exhibit violent agreement: Quoting reviewer (voting to reject): practical
cases, while leaving the general case unsolved. Boost.Outcome is a set of tools (rather than just one) and you are expected to choose one that best solves your particular problem.
Quoting response by library author:
I was just about to say the same thing, but more pointed. <...> Knowing that a piece of code will never, ever see stack unwinding lets you skip handling stack unwinding. Hence `noexcept`.
Furthermore, unlike with exception specifications which were unhelpfully checked at runtime, Outcome fails at compile time. .... You can't successfully compile code if your E types don't have interop specified for them.
Outcome is of particular use in a lower-level layer of code (closer to bare metal, or in server contexts), where explicit deterministic error handling warrants increased tedium or inconvenience in explicit checks for success/failure.
An observer might note that the reviewer (voting to reject) and library author have a shared view of what the library proposes to do; but the reviewer desires a generalized solution, rather than an interop-solution that provides specific performance constraints and behavioral guarantees. They agree on what it does. They disagree regarding the value and likelihood of a future possible generalized solution to perform a similar function, which is not currently proposed for review. The (perhaps unstated) concept is that no generalized solution may be possible with today's language, or perhaps ever: Both 'error' objects and 'exception' throws fundamentally serve different use cases: (a) 'error / status' instance (i.e., “opt-in”): A discrete object that can be inspected (or ignored), such as to perform conditional processing (b) 'exception' throw (i.e., “opt-out”): A control transfer-to-caller that avoids accidental instruction execution (e.g., stack-unwind) and which cannot be ignored Proponents of (a) talk of contract and execution simplicity, and of determinism / efficiency. Proponents of (b) talk of composition simplicity, and avoidance of edge cases due to liberation from local tedium. A further complication is due to how arbitrary (implementation-specific) data is returned for failure handling: It can be encoded into the type system (increasing coupling across APIs), or may be type-erased (such as done by 'std::error_code'). Noted in the review is that 'exception' throws are a special case of type erasure, as the C++ runtime performs the type erasure without impacting the declared API (thus providing a very large part of the convenience for using exceptions). The Outcome library attempts to unify '<error|exception>' handling for localized reasoning, and (re-)throw only in specific well-defined contexts. It is not a generalized pattern, but a unified mechanism for discrete handling of those scenarios where the design choice is explicitly made to perform localized reasoning of failure from dependence upon heterogeneous modules that may perform surprising (and evolving) behavioral changes for failure-notification. A generalized solution intends to provide uniformly simpler code, or generalized idioms. This might not (ever) be possible when we explicitly talk about localized reasoning of error handling within the local context: By definition, we want to enumerate and handle discrete failure cases specific to the local algorithm; so the best we can do is to utilize (compile-time) type-checks that verify our unwrapping-and-interrogation of our '<T|error|exception>' instance that bubbles up from within some far-away dependency that chooses the most inopportune time to exhibit surprising changes in behavior. Also by definition, localized reasoning requires localized tedium to _handle_ specific error cases, which are also enumerated locally. In such cases, a generalized solution does not apply. Much of the concern over Outcome acceptance appears to be based on concerns of the effects of the library on the ecosystem. It is true that users might wrongly apply localized reasoning tools to (widespread) generalized use. However, that same issue already exists with the C++ community's dichotomous split between 'error' or 'exception' handling, and with our Groundhog-Day revisiting of the same irreconcilable patterns that bring us the “dual-APIs” for “throw/no-throw” in the C++ Networking TS and Filesystem TS (and presumedly, many more Standard libraries to come). Of particular note is the assertion raised during the review that: *- The error being exceptional or not depends upon context, and not the algorithm. For example, connection failure on a game client and a game server are both backed by the same function; but Outcome permits the context to convert an 'error' into an 'exception', or not (based on caller-context). To quote one reviewer (voting to accept):
Boost.Outcome is not to “replace exception handling in your programs”: It is used to cover those isolated places where exception handling proves inferior to manual control flows.
From the library author: As the last section of the tutorial covers, there is a non-source-intrusive mechanism for externally specifying interoperation rules to handle libraries using one policy interoperating with libraries with different policies. This lets Eve stitch together the Alice and Bob libraries without having to modify their source code, and without affecting any other libraries. I personally think this Outcome's coup de grace and why it's the only scalable choice for large programs considering using
We might now expand our Matrix Of Confusion for preferring 'error' or 'exception' to include: (a) “Generalized-pattern” vs. “Localized-reasoning” (b) Expected vs. Non-Expected failure (c) Dependency upon subsystem providing 'error' vs. subsystem providing 'exception' While technically a “matrix”, commonly this is (quite) multi-dimensional: Frequently we depend upon many subsystems, each of which make very different decisions for how they relay disappointment, and where each individual subsystem will *change* that decision in surprising ways merely when we perform a version update. This is the brittleness that Outcome intends to address. Outcome is merely a mechanism to enable 3rd party module inter-operation. It does not make the decision for whether 'error' or 'exception' instances are provided by some subsystem, nor does it care. Rather, it is a unification mechanism that enables an algorithm to be _authored_ with specific performance and behavioral guarantees when reliance is upon one-or-more modules that _made_ that 'error' vs. 'exception' decision in a manner your specific use case finds unfortunate. Lastly, reviewers most critical of Outcome point to 'exceptions' as the C++ language mechanism to be leveraged in design, such as to enforce strong guarantees of object invariants (e.g., RAII permits ctors to throw to ensure invariants are not violated), and thus the Outcome library is not needed. However, despite this language feature, alternative idioms such as private ctors and 'make_xxx()' factory functions are not uncommon to also enable successful instantiation with internal state upholding invariant values; and we again must concede that modules employ differing decisions to compile with exception handling enabled. Thus, it seems reasonable that a library such as Outcome might exist to present a unifying interface across 3rd party code with differing design decisions, rather than hope for a simpler world where a single (exception-based) approach is mandated system-wide across modules. # CONCERN: CUSTOMIZATION POINTS (COMPLEXITY) Outcome attempts to bridge vocabulary types for '<error|exception>' handling among subsystems, including: C++ with or without exceptions enabled, C subsystems, and modules providing customized errors and/or customized exceptions. These goals necessarily complicate the design beyond a simple 'variant<>' value-type with no such customization points. This complexity was commented on by reviewers that voted to reject, suggesting: (1) Complexity is too high for the features provided; (2) Additional complexity to support C compatibility is unnecessary; and (3) Complexity is unnecessary if the library required subsystems not using exceptions to be wrapped behind a C-style API and to compile only those wrapped-subsystems with exception handling disabled. Further, other indirect comments suggested: (4) Complexity is lessened if Outcome assumed use of only 'std::error_code' (not user-customized error instances), and the type-erased payload provided through an exception (e.g., 'throw'). In response, the Outcome library premise is that a simpler world does not exist (where such customization is unnecessary): Despite nearly a decade of experience, the C++ community has yet to adopt 'std::error_code' as the ubiquitous mechanism to identify error (where instead bespoke error types remain in widespread use); and even in modern C++ codebases, it is not uncommon to compile with exceptions disabled. A heterogeneous landscape exists, and likely will continue to exist. For both practical and technical reasons, it is expected that domain-specific customization for error and exception handling will continue to be required, such as to address engineering constraints and needs for 'constexpr'; for small-embedded environments with severe restrictions on memory allocation; and for context-specific payload transferred or accumulated into journals / logs as a part of error handling. It is even expected that the C++ Standard itself will adapt to its own evolving idioms exposed through new language features, and (future) standard types that we might assume will be consistently applied across standard library APIs. This underscores the need for such customization points, where proposals such as Outcome attempt to provide an effective mechanism within a rather bleak current landscape with guaranteed future changes. In the face of this (currently-seen and continually expected) evolution, we have today's practical real-world issues: Commonly today's systems are composed of 3rd party modules that must interoperate in a well-behaved and deterministic manner, where '<error|exception>' customization becomes a system requirement. Of particular note is that in large codebase environments, current practice is to adopt customization policies to impose consistent and bespoke rules across the entire codebase, and across modules. Prior to Outcome, this customization is often done with preprocessor macros. In these environments, Outcome is viewed as a positive evolution. Today's Outcome customization points exist “out-of-the-box” to be compatible with existing module-specific 'error' or 'exception' needs, and to provide API/source-compatibility with a possible future-compatible 'std2::error_code'. Reviewers critical of the provided customization points want type-erased error state, but the library author asserts that this is hard to do in a manner that is (1) efficient, and (2) permits evolution (such as to support user-specific error values, or a possible future 'std2::error_code'). Further, the library author asserts that the library is not in fact complex; but that it can be overwhelming to the uninitiated merely because it enables customization of '<E>', and customization enables possibilities (which by definition implies complexity); and that such customization is characteristic of most (all?) vocabulary libraries. Despite concerns over complexity, v2 is greatly simplified over v1 where changes were directed through v1 review feedback; and provides a single-header version with reasonable compile-time overhead. Using (integrating) the library should not be difficult, and explicit efforts to establish ABI stability should make extended use over evolving codebases possible, and not-hard. Because enabling cross-module '<error|exception>' handling is fundamental to the library, it is conceivable that the library offers the minimal interface with the minimal possible complexity to do this job. And, this is a job that is (1) needed by real-world users; and (2) unavailable through alternative mechanisms (and which would not be provided through a simplified value-type composed of 'variant<T,error,exception>'). this sort of
error handling.
In this context where Outcome may possibly exhibit minimal sufficient complexity to address its intended target of cross-module '<error|exception>' transport, one might recall the Fred Brooks quote:
“The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstracts away its essence.” -- Fred Brooks, “No Silver Bullet” (1986)
# DISAGREEMENT: REQUIRES C++14 (versus C++11) Outcome requires C++14 and is identified as failing on some toolchains exhibiting non-conforming C++ Standard behavior. The review raised concerns that requiring C++14 (rather than C++11) would limit the library's suitability for Boost inclusion. In this context, it might be suggested that an error variant is needed by everyone, so we might reject a C++14 implementation in the hope that a future C++11 compatible library will be proposed. It is noted that Outcome v1 supported old compilers back to clang 3.1 and gcc 4.9, and had many workarounds to suppress non-conforming compiler behavior; and used preprocessor metaprogramming to work around compile-time selected CRTP (because of compile-time costs, and issues on older compilers). However, these approaches were rejected in the v1 review as too complex; and reviewers were not persuaded by concerns raised by the library author that their removal would demand dropping compatibility with older toolchains. Outcome v2 removed these workarounds based on v1 feedback, resulting in a requirement for newer toolchains. Similar discussions have been raised in other contexts regarding Boost distribution packaging, and the possibility of forking “pre/post” C++11 (e.g., “modern C++”) libraries into separate Boost distributions. This is an unfortunately complex issue, as Boost already contains libraries supporting varying levels of “minimum” requirements including varying support for compilers exhibiting non-conforming behavior for C++98, C++03, C++11, C++14, and C++17 (and others). Some libraries such as Boost.Hana or those reliant upon heavy 'constexpr' behavior demand the very-latest C++17 toolchains. Further, it is recognized that in some cases backward-compatibility can be undesirable, or fundamentally limiting to the library's usefulness. For example, at a recent 'BoostCon' / 'C++Now' conference an attempt was made to evolve the 'Boost.Date_Time' API to use the seemingly highly suitable C++17 'structured binding' declaration to decompose 'date / time' objects. Unfortunately, this work was abandoned: Due to use by other Boost libraries restricted to a previous C++ Standard, and the discovery that 'Boost.Date_Time' library semantics fundamentally changed when using this new language feature, it was concluded to not be feasible to support both the legacy API and new API using structured bindings. It appears in this case a new library must be authored without consideration of backward-compatibility to enable this evolution for what appears to be an otherwise obvious or natural expressiveness to decompose 'date / time' objects using 'structured binding' declarations. It is noted that C++11 is already eclipsed by C++14 and C++17, and that current Boost library requirements merely require C++ Standard conforming behavior, whereby the library must clearly document those platforms that are supported. Further, it is noted that a v1 implementation supporting older tool chains was rejected (so v2 now requires C++14). As this is a complex issue that demands consideration of many tradeoffs (including feasibility and behavior fundamental to the library itself), it is expected that the Library Maintainer constantly review and evolve these decisions as the C++ language evolves; toolchains are updated; and users raise issues or provide contributions. # CONCERN: LIBRARY FUTURE EVOLUTION A concern is raised that upon acceptance into Boost, Outcome may evolve beyond the design reviewed for acceptance. Further, a specific concern was raised regarding Outcome exploring evolution of an 'std::error_code' (which we might call a proposed 'std2::error_code'). Within the context of Outcome, the 'error' type exists as a mere customization point by which the user may supply a bespoke definition, or incidentally use the 'std::error_code' provided by the C++11 Standard. Both of these are seen today in common practice. It is expected that users (and perhaps the library author) will continue to explore possible 'error' implementations for specific purposes (such as exploration of an 'error' for small-embedded that does not rely upon allocation machinery). This is viewed as a necessary and healthy advancement of the science, for which the library (by design) conveniently empowers the end-user to parameterize domain-specific 'error' types into 'result<>' or 'outcome<>'. Indeed, it is hoped (and expected) that a greater amount of user-experimentation or flirtation with 'error' value transfer will be performed (not less) in the context of user-specific needs or engineering constraints: This is a fundamental benefit from having Outcome as a design option for authoring interfaces across module boundaries. Regarding a possible future “drift” from its clearly-stated mission, this review considers the library as submitted; and defers to the Boost community regarding policies and procedures for handling libraries included in the Boost distribution (which exist in varying states of evolution and maintenance). # FINAL THOUGHTS The Boost community is stronger for enabling and exploring idioms, and for not shying away from the difficult (and sometimes contentious) effort to discover new approaches that address the dark corners that stress our real-world systems. These are (perhaps) the only noble efforts that someday may lead to new best practices. If experience (otherwise known as, “painful memory of limitation”) is a prime driver for the design and implementation choices made by developers, then it is unreasonable in a review such as this to expect agreement on all fronts. Indeed, our systems have differing constraints and evolutionary / scaling prospects, and we fear different things. However, it seems important to keep in mind that this disagreement continues to be necessary and expected: Rather than pretending to hide within an echo chamber, the Boost Community does the “hard work” of challenging perspectives, pushing the envelope, and questioning assertions in the face of a constantly shifting technological landscape and evolving C++ language. Dissenting or critical reviews are essential, and desired. The long Boost history includes many examples of speculative and risky approaches, which in hindsight are now considered common and "best" practice. Few C++ developers today can practice professionally without at least passing knowledge of template-metaprogramming, which used to be an esoteric ritual only within the confines of Boost. Outcome was designed under the proposition that cross-module 'error' / 'exception' handling in today's systems is unnecessarily brittle and problematic. Reviewers found Outcome to effectively address a serious concern of transporting different '<error|exception>' instances across modules. It attempts to solve a hard problem, which spans 3rd-party module composition, while adhering to specific performance constraints and behavioral guarantees. Its use is demonstrated to be effective in some environments exhibiting severe engineering restrictions. As such, it is not expected that all users everywhere will have direct need for Outcome. However, it is hoped that evolution of similar idioms will eventually permit the broader C++ community to consider this domain of cross-module stateful control transfer to be a “solved problem” (or certainly “less-brittle” or “less-problematic”). Great thanks to the Boost Community for their tremendous efforts, disciplined review, and detailed exploration of topics in considering this library submission. Sincerest appreciation to Niall Douglas (Author of the Outcome Library) and other contributors for pushing the boundary for '<error|exception>' handling and for submitting this work to the Boost Review process. --charley Boost.Outcome (v2) Review Manager