On 2 Jul 2014 at 17:59, Beman Dawes wrote:
You must remember that from the perspective of ISO, the only engineering standards which exist are other ISO standards. That in this case equals POSIX. This is why I said that Windows support cannot be standardised and would have to be omitted. pletely agree on this sentiment.
You have been misinformed.
I believe I used loose language. But I am not misinformed. For your reference, I used to serve as the ISO SC22 mirror convenor for Ireland, though you have many more years of ISO experience than I do.
ISO rules are somewhat different for dealing with documents that are not ISO standards, but mostly that is just a matter of careful drafting. For example, see the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards, section 6.6.3 Use of trade names and trademarks.
The C++ standard and its TRs and TSes support Windows, as well as a lot of other operating systems, and that is one of the criteria for evaluating proposals to the committee. Any proposal that cannot support a common operating system would likely be dead on arrival.
I'll come back to this at the end.
Look at the Filesystem TS for an example. It mentions Windows and several other operating systems by name, and has a compliance section that explains how that TS copes with differences between platforms.
As ISO would call it, it's all about "normative references" which can only be to other ISO documents (source: http://www.iec.ch/standardsdev/resources/draftingpublications/writing_ editing/directives/normative_references.htm). As POSIX is an ISO standard, it can be normatively referenced. Everything else goes into a Bibliography. What this turns into is that you can have an informative section which explains similarities and differences between the standard and implementations and other non-ISO standards, but which forms no part of the standard itself. You can also claim that there is no substantive difference between the standard and some named implementation(s), but again this is informative. While you *could* standardise a non-normatively referenced item aka some proprietary technology, you would receive a lot of heat from ISO and the potential for JTC1 to reject it at a plenary session which would be deeply embarrassing. The reason why is because there is a formal process for proprietary technologies to become standardised which involves patent agreements etc. There are also other ISO working groups to consider, so if a WG21 Networking TS standardised the Windows IOCP model I could rightly see the Austin Working Group going bananas over the potential consequences on POSIX, never mind the ADA WG, or even the Ruby WG. Let's apply this to the Filesystem TS. Windows provides most of a semantic equivalence to POSIX in filesystem, so file handles, reading and writing, file paths referencing a universal file system namespace (apart from a drive letter, differences in allowed characters and the slash being opposite they're very semantically close, much closer than say file paths on DEC VMS), ability to use unicode (albeit in different binary formats), they all map onto the same thing, and these are what Filesystem standardises. This is because this is the common subset of Windows *to* POSIX, not *with* POSIX. Let's briefly cover what Filesystem does not currently standardise. Memory mapping files on Windows is similar to POSIX, though with a nasty gotcha difference [1]. Symbolic links are also similar, though also with a nasty gotcha difference [2]. The Win32 API doesn't expose an atomic stat() call, even though the NT kernel does, so metadata can be racy which is another nasty gotcha. Off the top of my head, the only substantial divergence of Windows from POSIX is security and access permissions [3], and of course file locking where the Windows implementation is not stupidly broken. So, in the Filesystem TS, you can standardise an API which if applications use will produce identical effects on all systems (apart from access perms on Windows [see below]), and you can claim this legitimately as it is true (sans bugs of course) for the subset of POSIX-equivalent Filesystem functionality you have standardised. As your informative, you can point out minor differences about platform specifics and deviations from POSIX, and all is good. But let us be very clear here, the "gold standard" aimed at is POSIX. If Windows provides a feature not provided by POSIX - and in Filesystem there are many e.g. async file i/o - you cannot standardise it in an ISO TS without significant political consequences. If you look at access perms, note the direction of travel: the Filesystem TS can standardise a POSIX feature even though it isn't available on Windows and that's permitted. The opposite is not permitted.
The C++ standard and its TRs and TSes support Windows, as well as a lot of other operating systems, and that is one of the criteria for evaluating proposals to the committee. Any proposal that cannot support a common operating system would likely be dead on arrival.
And here we arrive onto the rub of the matter. Windows originally borrowed the 4.4 BSD network stack, and as a result has a closely similar socket API to POSIX - if used *synchronously*. Unfortunately, all i/o (with a few exceptions) on Windows is *always async* with the synchronous functions acting as wrappers of the underlying async implementation. Windows in fact pretends to be a micro-kernel and provides a 98% [4] async i/o API as if it were micro-kernel. Thus, on Windows, ASIO uses async sockets whilst on POSIX it uses non-blocking sockets. This produces a ton of semantic difference which is why there is such a plethora of POSIX-specific and Windows-specific class types in the ASIO reference page (http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/reference.ht ml, bottom of page). Now, a Networking TS standardising ASIO can simply say "implementation defined" whenever these async vs. non-blocking semantic differences turn up, and I suspect that is exactly what will happen because all the alternatives are harder. This implies that Windows-specific support aka the whole true async socket semantic model aka cool stuff like IOCP (i/o completion ports) cannot be standardised. What will result in the Networking TS will still be useful for some C++ network applications, but I suspect that real C++ network applications are going to remain on ASIO because it lets you code around the semantic differences. Does this explain my original claim? I am not claiming that a standardised ASIO in a Networking TS would not function on Windows. I *am* claiming that so much Windows-specific aka async-specific functionality would have to be left unspecified that no serious C++ networking user would bother using the functionality in the TS. So what would I propose instead? What I personally think is needed here is that WG21 provide a canonical Networking TS implementation library rather than writing a specification and leaving it up to STL implementators. That canonical implementation would be a substantially rejigged and slimmed down all C++14 subset of ASIO [5]. This of course is very new territory - I am not aware of WG21 ever providing a canonical library implementation before. But WG21 has already been breaking new ground with letting library writers go direct to std::experimental and skip getting a new STL type into Boost first and later submit for standardisation after some experience is gained. I note the fact that activity in Boost dropped substantially around the same time WG21 allowed this, and that "all the action" in new library features appears to be happening around WG21 processes instead of via Boost. Personally I think this is the best way of standardising ASIO into a Networking TS. But I appreciate that such an approach is fraught with dangers, not least who will pay the engineers who write and test such a canonical library? What if there is substantial disagreement in the direction of refactor? What if real users just don't care, and keep using original ASIO? Still, it's better than every STL having a separate toy implementation of ASIO that is useless. [1]: munmap() will unmap all maps falling into the range supplied. UnmapViewOfFile() only unmaps the precise map given. This can be a nasty source of bugs. [2]: Derereferencing a link takes different semantics to POSIX unfortunately. It works similarly enough most won't notice, until they do. [3]: xattr ACL support on Linux and BSD looks similar enough to Windows ACLs that a common subset of functionality should be possible. Of course, POSIX famously demurred on standardising these in the 1990s, but ACLs are now pretty much standard in implementation across Linux, BSD and Windows. They are ripe for being formally standardised into POSIX. [4]: A glaring counterexample is that CreateFile() which is used to open file handles cannot be executed asynchronously. Neither can CloseHandle() nor FlushFileBuffers(). There are others, but these three are the worst offenders, especially as CreateFile() is incredibly slow (~30k ops/sec). [5]: ASIO could do with explicit support for micro-kernel operating systems, by which I mean you can completely dispense with threads on such systems. Something which annoyed me with the QNX port of ASIO is that it used the POSIX multiplexing API which was woefully inefficient when all syscalls in a micro-kernel OS can be multiplexed at source, and therefore the ASIO io_service could be very tightly integrated into the core QNX message passing system. Such an adaptation I think would not require much change to ASIO. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/