Re: [boost] [serialization]A question about the implementation of XML_name in basic_xml_oarchive.ipp
On 9/28/05, David Abrahams <dave@boost-consulting.com> wrote:
Hmm... Also, is the apparent dependency on ASCII encoding truly portable?
Caleb Epstein wrote:
Doubtful. Wouldn't testing for std::isalnum || '-' || '_' be a better idea? Perhaps not quite as performant (once the lookup table was made static), but certainly more portable and simpler to read.
Simon Buchan wrote:
In most implementations, the is*()'s are implemented using exactly the same method.
Yes, but the table will be different on an EBCDIC implementation than they are on an ASCII implementation. The point is that the specified table hard codes ASCII, so when somebody runs it on an IBM mainframe it will give the wrong answer. -- Martin Bonner Martin.Bonner@Pitechnology.com Pi Technology, Milton Hall, Ely Road, Milton, Cambridge, CB4 6WZ, ENGLAND Tel: +44 (0)1223 441434
Martin Bonner wrote:
On 9/28/05, David Abrahams <dave@boost-consulting.com> wrote:
Hmm... Also, is the apparent dependency on ASCII encoding truly portable?
Caleb Epstein wrote:
Doubtful. Wouldn't testing for std::isalnum || '-' || '_' be a better idea? Perhaps not quite as performant (once the lookup table was made static), but certainly more portable and simpler to read.
Simon Buchan wrote:
In most implementations, the is*()'s are implemented using exactly the same method.
Yes, but the table will be different on an EBCDIC implementation than they are on an ASCII implementation. The point is that the specified table hard codes ASCII, so when somebody runs it on an IBM mainframe it will give the wrong answer.
This may be irrelavant anyway: http://www.w3.org/TR/REC-xml/#charsets 2.2 Characters [Definition: A parsed entity contains text, a sequence of characters, which may represent markup or character data.] [Definition: A character is an atomic unit of text as specified by ISO/IEC 10646:2000 [ISO/IEC 10646]. Legal characters are tab, carriage return, line feed, and the legal characters of Unicode and ISO/IEC 10646. The versions of these standards cited in A.1 Normative References were current at the time this document was prepared. New characters may be added to these standards by amendments or new editions. Consequently, XML processors MUST accept any character in the range specified for Char. ] Character Range [2] Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] /* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */ The mechanism for encoding character code points into bit patterns MAY vary from entity to entity. All XML processors MUST accept the UTF-8 and UTF-16 encodings of Unicode 3.1 [Unicode3]; the mechanisms for signaling which of the two is in use, or for bringing other encodings into play, are discussed later, in 4.3.3 Character Encoding in Entities. The interesting bit is the last sentance. (I didn't want to take it out of context)
This is on the right track - but not quite there. This function checks characters in xml_tag. Surely the tag can't contain a tag. Note that there is another problem which as already come up. An stll:string can contain a character such as a '\0' while an xml file cannot. When I was making this xml_archive implementation I did considert making all sthe string variables as hex character data. I rejected this for a couple of reasons. There are a bunch of other issues here. Many of which I forget. It has much to do with the following and how they interact and map (or don't map) to each other. utf8 utf16 codecvt_facets wchar char current(global) locale xml characters which are legal/illegal stl:string legal characters - which is all of them but doesn't match the above etc. I made some compromises to push things through to the end. If one wants the best xml representation my advice is to use xml_w?archive. This saves and loads utf-8. On gcc platforms this is mapped to 32(!) bit characters, on other machines to 16 bit characters. When saving / loading to stl:string variables these are mapped to the multi-byte character set of the current locale, which may be different from the locale under which the archive was created. So if you want to be really safe, use stl::wstring throughout your code and besure that you don't have any prohibited characters in these strings. Robert Ramey P.S. EBCDIC On an ebcic machine I would expect that following to be necessary. one's xml_archive is in utf-8 or utf16. The stream would be opened with a codecvt facet that maps utf8 or ascii, or whatever to ebcdic. The routine in question should be altered to function with the local character set be it ascii or ebcdic. RR Simon Buchan wrote:
Martin Bonner wrote:
On 9/28/05, David Abrahams <dave@boost-consulting.com> wrote:
Hmm... Also, is the apparent dependency on ASCII encoding truly portable?
Caleb Epstein wrote:
Doubtful. Wouldn't testing for std::isalnum || '-' || '_' be a better idea? Perhaps not quite as performant (once the lookup table was made static), but certainly more portable and simpler to read.
Simon Buchan wrote:
In most implementations, the is*()'s are implemented using exactly the same method.
Yes, but the table will be different on an EBCDIC implementation than they are on an ASCII implementation. The point is that the specified table hard codes ASCII, so when somebody runs it on an IBM mainframe it will give the wrong answer.
This may be irrelavant anyway: http://www.w3.org/TR/REC-xml/#charsets 2.2 Characters
[Definition: A parsed entity contains text, a sequence of characters, which may represent markup or character data.] [Definition: A character is an atomic unit of text as specified by ISO/IEC 10646:2000 [ISO/IEC 10646]. Legal characters are tab, carriage return, line feed, and the legal characters of Unicode and ISO/IEC 10646. The versions of these standards cited in A.1 Normative References were current at the time this document was prepared. New characters may be added to these standards by amendments or new editions. Consequently, XML processors MUST accept any character in the range specified for Char. ] Character Range [2] Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] /* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */
The mechanism for encoding character code points into bit patterns MAY vary from entity to entity. All XML processors MUST accept the UTF-8 and UTF-16 encodings of Unicode 3.1 [Unicode3]; the mechanisms for signaling which of the two is in use, or for bringing other encodings into play, are discussed later, in 4.3.3 Character Encoding in Entities.
The interesting bit is the last sentance. (I didn't want to take it out of context)
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (3)
-
Martin Bonner -
Robert Ramey -
Simon Buchan