[Locale] Security bug announcement - UTF-8 validation

Hello, Boost.Locale library in Boost 1.48 to 1.52 including has a security flow. boost::locale::utf::utf_traits accepted some invalid UTF-8 sequences. Applications that used these functions for UTF-8 input validation could expose themself to security threats as invalid UTF-8 sequece would be considered as valid. This bug is fixed in upcoming Boost 1.53. For more details see: https://svn.boost.org/trac/boost/ticket/7743 Users who can't upgrade to the latest versions may apply the following patch to fix the problem. http://cppcms.com/files/locale/boost_locale_utf.patch Regards, Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

On Friday 04 January 2013 06:15:27 Artyom Beilis wrote:
Hello,
Boost.Locale library in Boost 1.48 to 1.52 including has a security flow.
boost::locale::utf::utf_traits accepted some invalid UTF-8 sequences.
Applications that used these functions for UTF-8 input validation could expose themself to security threats as invalid UTF-8 sequece would be considered as valid.
This bug is fixed in upcoming Boost 1.53.
For more details see: https://svn.boost.org/trac/boost/ticket/7743
Users who can't upgrade to the latest versions may apply the following patch to fix the problem.
Perhaps, this should be announced in 1.53 release notes?

________________________________ From: Andrey Semashev <andrey.semashev@gmail.com> On Friday 04 January 2013 06:15:27 Artyom Beilis wrote:
Hello,
Boost.Locale library in Boost 1.48 to 1.52 including has a security flow.
boost::locale::utf::utf_traits accepted some invalid UTF-8 sequences.
Applications that used these functions for UTF-8 input validation could expose themself to security threats as invalid UTF-8 sequece would be considered as valid.
This bug is fixed in upcoming Boost 1.53.
For more details see: https://svn.boost.org/trac/boost/ticket/7743
Users who can't upgrade to the latest versions may apply the following patch to fix the problem.
Perhaps, this should be announced in 1.53 release notes?
It is in release notes quoting: Locale: * Security related bug fix, some invalid UTF-8 sequences where accepted as valid #7743 Also maybe it need to be more Release managers, maybe we need to make it bolder? Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

On 1/4/2013 6:28 AM, Artyom Beilis wrote:
From: Andrey Semashev <andrey.semashev@gmail.com> On Friday 04 January 2013 06:15:27 Artyom Beilis wrote:
Hello,
Boost.Locale library in Boost 1.48 to 1.52 including has a security flow.
boost::locale::utf::utf_traits accepted some invalid UTF-8 sequences.
Applications that used these functions for UTF-8 input validation could expose themself to security threats as invalid UTF-8 sequece would be considered as valid.
This bug is fixed in upcoming Boost 1.53.
For more details see: https://svn.boost.org/trac/boost/ticket/7743
Users who can't upgrade to the latest versions may apply the following patch to fix the problem.
Perhaps, this should be announced in 1.53 release notes?
It is in release notes quoting:
Locale: * Security related bug fix, some invalid UTF-8 sequences where accepted as valid #7743 Also maybe it need to be more
Release managers, maybe we need to make it bolder?
Yes, I think this warrants a bolder announcement, like the one we did last release for the potentially breaking result_of change. Here I'm thinking of the red warning on the front page, not necessarily a separate page describing the issue. The red warning could simply link directly to the 1.53 release notes. Daniel, thoughts? -- Eric Niebler BoostPro Computing http://www.boostpro.com

----- Original Message -----
From: Eric Niebler <eric@boostpro.com>
It is in release notes quoting: Locale: * Security related bug fix, some invalid UTF-8 sequences where accepted as valid #7743 Also maybe it need to be more Release managers, maybe we need to make it bolder? Yes, I think this warrants a bolder announcement, like the one we did last release for the potentially breaking result_of change. Here I'm thinking of the red warning on the front page, not necessarily a separate page describing the issue. The red warning could simply link directly to the 1.53 release notes. Daniel, thoughts? -- Eric Niebler BoostPro Computing http://www.boostpro.com
What is more disturbing me that we do not have **standard and ready** to go way of handling such situation. I think we need a general policy what to do if some bug that may affect application security or introduce a potential vulnerability to an application is discovered. It is not the first time (and of course it would not be the last time) that such a situation happens. For example, there is a bug in UUID that was fixed in 1.43 should get much more serious attention: https://svn.boost.org/trac/boost/ticket/3971 It is uncommon case when generation on unpredictable UUID is used ad application relay on that (for example session key). Also, I'm not sure if the author was aware how critical this bug was, but such a bug should be treated much more seriously that small line in Boost 1.43 UUID notes. Probably potential vulnerabilities should be: 1. Published in a central place, including the information about which Boost versions are affected. 2. Exact security risk should be described. 3. A patch that can fix them should be given. You should remember, that for example many Linux distributions deliver older Boost version and support of for a long time. End even distributions with a short release cycle need to provide security updates for their packages for at least for about year or two, but sometimes for much longer period as like RHEL, Debian or Ubuntu LTS. The fact that Boost does release bug fixes for older versions makes the work of package maintainers for Linux distributions much harder. Such central documentation and page should be available and keep a track of all potential vulnerabilities, and of course it should not be tied to release cycle. Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

On 4 January 2013 19:42, Eric Niebler <eric@boostpro.com> wrote:
Yes, I think this warrants a bolder announcement, like the one we did last release for the potentially breaking result_of change. Here I'm thinking of the red warning on the front page, not necessarily a separate page describing the issue. The red warning could simply link directly to the 1.53 release notes.
Daniel, thoughts?
Sure, whatever you want. Just let me know what you want to add.

On Friday 04 January 2013 06:15:27 Artyom Beilis wrote:
Hello,
Boost.Locale library in Boost 1.48 to 1.52 including has a security flow.
boost::locale::utf::utf_traits accepted some invalid UTF-8 sequences.
Applications that used these functions for UTF-8 input validation could expose themself to security threats as invalid UTF-8 sequece would be considered as valid.
This bug is fixed in upcoming Boost 1.53.
For more details see: https://svn.boost.org/trac/boost/ticket/7743
Users who can't upgrade to the latest versions may apply the following patch to fix the problem.
Perhaps, this should be announced in 1.53 release notes?

On 05/01/13 01:15, Artyom Beilis wrote:
Hello,
Boost.Locale library in Boost 1.48 to 1.52 including has a security flow.
boost::locale::utf::utf_traits accepted some invalid UTF-8 sequences.
Applications that used these functions for UTF-8 input validation could expose themself to security threats as invalid UTF-8 sequece would be considered as valid.
This bug is fixed in upcoming Boost 1.53.
For more details see: https://svn.boost.org/trac/boost/ticket/7743
Users who can't upgrade to the latest versions may apply the following patch to fix the problem.
http://cppcms.com/files/locale/boost_locale_utf.patch
Regards,
Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Hello, Pardon my ignorance, but how would an invalid UTF-8 sequence cause a security threat? All I can think it would do is create garbage. I don't mean every day security threats, I mean any. Thanks, Jookia.

On 04/01/13 16:00, Jookia wrote:
Hello,
Pardon my ignorance, but how would an invalid UTF-8 sequence cause a security threat? All I can think it would do is create garbage.
Different software treat malformed UTF-8 sequences differently. One piece of software may consider that the sequence contains some special characters while others might not. This can be used for SQL injection among others.

On 04 January 2013 15:01 Jookia [mailto:166291@gmail.com] wrote :-
Hello,
Pardon my ignorance, but how would an invalid UTF-8 sequence cause a security threat? All I can think it would do is create garbage.
I don't mean every day security threats, I mean any.
Thanks, Jookia.
I'm not an expert in this field but I believe that invalid utf8 sequences have been used for several well documented attacks - the most common have been to disguise paths / url's to avoid validation routines which would discard these url's automatically - ie a HTTP get request for /../somefile which could (and has for some servers in the past) end up returning somefile which is living outside of the expected directory tree of retrievable documents. http://www.technicalinfo.net/papers/URLEmbeddedAttacks.html Alex
participants (7)
-
Alex Perry
-
Andrey Semashev
-
Artyom Beilis
-
Daniel James
-
Eric Niebler
-
Jookia
-
Mathias Gaunard