_CRT_SECURE_NO_DEPRECATE would best be defined by bjam when TOOLS=vc-8_0, as many other Boost libraries get the same warnings. The deprecations are purely for security (potential buffer overflow) reasons. The help subsystem installed with the VC8 beta explains all. Lookup gets_s in the index, and then follow the link to "Security Enhancements in the CRT", to get up to speed quickly. - Keith MacDonald "Jonathan Turkanis" <technews@kangaroologic.com> wrote in message news:da472i$7d7$1@sea.gmane.org...

I think defining _CRT_SECURE_NO_DEPRECATE is something the end-user has to do, since by the time my headers are included the standard iostreams headers may already have been included.

Do you know any more details about the deprecation of these member functions? I was just guessing that the deprecation was for security reasons; can you confirm this? I'd also like to know what functions are supposed to replace them.

Thanks for your help!

Jonathan

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

I'm not sure about copyright restrictions on posting excerpts, but this article explains what Microsoft are trying to achieve: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncode/html... I'm fairly sure that theVC8 beta 1 documentation covers this issue, but why couldn't you install beta 2?

I dont think defining _CRT_SECURE_NO_DEPRECATE in a header is acceptable.

I also agree. This should be defined in the bjam config file for TOOLS=vc-8_0, as I said earlier. Is that David Abraham's area of responsibility? - Keith MacDonald "Jonathan Turkanis" <technews@kangaroologic.com> wrote in message news:da4hpf$ton$1@sea.gmane.org...

Keith MacDonald wrote:

...

_CRT_SECURE_NO_DEPRECATE would best be defined by bjam when TOOLS=vc-8_0, as many other Boost libraries get the same warnings.

The deprecations are purely for security (potential buffer overflow) reasons. The help subsystem installed with the VC8 beta explains all. Lookup gets_s in the index, and then follow the link to "Security Enhancements in the CRT", to get up to speed quickly.

Unforunately, my beta 1 expired today, and I wasted about 8 hours last week trying to install beta 2 and then reinstalling beta 1.

If there's a discussion specifically about deprecated member functions, could you please post an excerpt?

Jonathan

After uninstalling beta 1 you may want to run the cleanup tool at http://go.microsoft.com/fwlink/?linkid=47065 I know a few people who have had troubles installing beta 2 before running this. On 7/2/05, Jonathan Turkanis <technews@kangaroologic.com> wrote:

Keith MacDonald wrote:

...

I'm not sure about copyright restrictions on posting excerpts, but this article explains what Microsoft are trying to achieve:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncode/html...

...

I'm fairly sure that theVC8 beta 1 documentation covers this issue,

As I mentioned somewhere, my VC8 beta 1 conveniently expired at the same time I noticed the deprecation warnings.

...

but why couldn't you install beta 2?

If only I knew! The first few times I tried to install it, the installer crashed. I tried again last night, and it fails with a bunch of error messages; I guess there's still residue in the registry from the failed installations. I really don't care that much about beta 2; what I'm worried about is the release version, which I'm afraid won't install either.

Jonathan

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

-- Cory Nelson http://www.int64.org

Keith MacDonald wrote:

...

I dont think defining _CRT_SECURE_NO_DEPRECATE in a header is acceptable.

I also agree. This should be defined in the bjam config file for TOOLS=vc-8_0, as I said earlier.

This will only help end users who build their applications with bjam. Also, a user may want to enable the deprecation warnings, because some of them may be useful.

Is that David Abraham's area of responsibility?

I think it's the responsibility of whoever volunteers, but I'm not sure it's a good idea.

- Keith MacDonald

Jonathan

Keith MacDonald wrote:

I was thinking of the build of the entire library, when it would look more professional if all these warnings were suppressed.

This might be useful for other libraries.With Iostreams, the .cpp files don't use these functions.

If headers contain calls to the deprecated functions, the Boost developers will have to decide on a policy for the library release, and document it. My vote would be to leave it up to the end user to define _CRT_SECURE_NO_DEPRECATE, as their company policy may be to disallow use of such software, so they need to be aware of it.

I think it's okay to disable deprecation warnings locally, if this is possible. (I imagine you can use pragma warning, with puch and pop, put I haven't checked.) But most uses of istream::read, etc. will occur in user code.

- Keith MacDonald

Jonathan

Philipp Thomas wrote:

* Keith MacDonald (boost@mailclan.net) [20050702 01:06]:

...

Lookup gets_s in the index, and then follow the link to "Security Enhancements in the CRT", to get up to speed quickly.

Great, anothoer non-standardized function, just what the world needs :(

They are submitted for standardization if that make you happier. Personally, I can't help but be reminded of the adage "The good thing about standards is that there are so many". j

Cory Nelson wrote:

The deprecated functions/methods I've seen all take buffers as arguments but not their sizes.

That's one reason I'm puzzled by the deprecation of istream::read and streambuf::sgetn, which take buffer size parameters.

I dont think defining _CRT_SECURE_NO_DEPRECATE in a header is acceptable.

I agree. Jonathan

Bo Persson wrote:

"Jonathan Turkanis" <technews@kangaroologic.com> skrev i meddelandet news:da4hie$tav$1@sea.gmane.org...

...

Cory Nelson wrote:

...

The deprecated functions/methods I've seen all take buffers as arguments but not their sizes.

That's one reason I'm puzzled by the deprecation of istream::read and streambuf::sgetn, which take buffer size parameters.

No, they don't. They take requested number of characters to read. What if the buffer is smaller? :-)

Heh, okay. These 'safe' functions require a little cooperation from the user.

They have definitely gone overboard in this case!

According to John's post, they've come to their senses, at least partially.

Bo Persson

Jonathan

John Maddock wrote:

...

That's one reason I'm puzzled by the deprecation of istream::read and streambuf::sgetn, which take buffer size parameters.

According to issue FDBK25976 Microsoft agree that it should not be marked as deprecated, so this is likely to be fixed in the final release (the full URL for issue is

http://lab.msdn.microsoft.com/productfeedback/viewfeedback.aspx?feedbackid=3...)

but I'm not sure if that URL actually works unless you're logged on.

It's works for me. Thanks for the reference -- it's exactly on point.

Interestingly these API's aren't documented as deprecated in the SDK as far as I can see, so I wonder if someone just made a mistake?

I think it has to be a mistake to deprecate essential member functions without introducing replacements.

It would probably be a good idea if someone would volunteer to give Microsoft's headers a once over and see if there are any other API's that are needlessly deprecated. Certainly there plenty of room for frustration here - I've seen deprecated warning from std::copy and other std lib algorithms (and no *not* using pointers), as well as the C API's.

Maybe they should just deprecate everything, to be on the safe side.

John.

Jonathan

Jeff Garland wrote:

On Sat, 2 Jul 2005 10:41:00 -0600, Jonathan Turkanis wrote

...

...

Interestingly these API's aren't documented as deprecated in the SDK as far as I can see, so I wonder if someone just made a mistake?

I think it has to be a mistake to deprecate essential member functions without introducing replacements.

I haven't been following this thread closely, but are you saying there aren't some sort of safe replacements?

Apparently so. For many of the deprecated functions there's a recommended safe version with the _s suffix, and most of these are proposed for standardization (see, e.g., http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1055.pdf.) But in the cases I'm talking about, there seem to be no new 'safe' versions. In fact, it's hard see how there could be, since the existing functions already provide a buffer size parameter. For example, what would basic_istream::read_s look like?

...

...

It would probably be a good idea if someone would volunteer to give Microsoft's headers a once over and see if there are any other API's that are needlessly deprecated. Certainly there plenty of room for frustration here - I've seen deprecated warning from std::copy and other std lib algorithms (and no *not* using pointers), as well as the C API's.

Maybe they should just deprecate everything, to be on the safe side.

I really don't think we should disparage them for this effort. This is a beta , after all, and it's sort of the point of beta stuff to shake out these sort of issues. I've seen the cost of using some of these unsafe functions in big software systems. You can create incredibly subtle, and hard to find intermitant bugs by doing strcpy into stack-based buffers.

I'm not against the _s functions, by any means, and I'm willing to cut microsoft some slack in a beta product. But it looks like whoever sprinkled the __declspec's all over the place didn't have a clear idea what he was doing.

So the fact that Microsoft is trying to raise awareness and move things forward -- I find it hard to argue against this. This particular approach might appear 'heavy handed', but it has us talking ;-)

About the wrong issues, though ;-) Jonathan

Jeff Garland wrote:

Jonathan Turkanis wrote

...

Jeff Garland wrote:

...

Jonathan Turkanis wrote

...

Apparently so. For many of the deprecated functions there's a recommended safe version with the _s suffix, and most of these are proposed for standardization (see, e.g., http://www.open- std.org/jtc1/sc22/wg14/www/docs/n1055.pdf.)

Yep -- looked at that, but it doesn't seem to cover all the deprecated functions -- which is not good :-(

...

But in the cases I'm talking about, there seem to be no new 'safe' versions. In fact, it's hard see how there could be, since the existing functions already provide a buffer size parameter. For example, what would basic_istream::read_s look like?

Right, got it -- that doesn't make sense. Should we submit a formal bug report?

Judging by John's post, it looks like that's already been done. Jonathan

Bronek Kozicki wrote:

And BTW, I thought that term "deprecated library features" is actually used by the C++ standard, has clear meaning,

Not exactly... 'deprecated' is used in the standard to describe certain language features, such as implicit conversion of string literals to array-of-non-const-char. However, the meaning is never clearly defined. c.f. core issue 223 [and probably heading off-topic for Boost lists now] -- AlisdairM

Hi everyone, I wanted to take a few moments to address the range of points raised in this thread, and to provide a little background. Herb provided a great summary of our motivations here. We want to help developers write more secure code, and we want to do that by default, so that the out-of-the-box experience is as secure as we can make it. We've reviewed lots of code in the last few years as part of the intensive security focus at Microsoft. Though we've already used these lessons to secure our own code, we wanted to help them be applied to other code too. To make this happen, we added "deprecation" warnings to a bunch of standard APIs. Most warnings tell you what function to use instead. For example, the warning for strcpy will point you at strcpy_s, which takes a buffer size as a second parameter. As you can imagine these kinds of changes mostly impact the C standard library. The C++ standard Library is in much better shape. It was designed well from the ground up and does not have extensive security problems. But there are areas (such as use of pointers for iterators) where it's possible to write dangerous unbounded constructs similar to a strcpy. We try to issue deprecation warnings on each of these constructs. While doing this, we're not making any compromise on producing a product that conforms to the standard -- we've improved our conformance in VS8 over our high level in VS7.1, mostly in minor ways. And while we do warn by default on insecure usages, we always provide a way to disable these warnings, and (in builds after the Beta 2 build most of you are using), these warning texts include specific advice on how to disable the warning if it is unwanted. We want to make sure it is as easy as possible to write a portable product. Though these changes are currently language extensions, we're also working to make them available to the whole C and C++ community, not just users of the Microsoft product. We've been working for a couple of years to standardize the changes to the C library in the C standard committee - and this process has been very productive and helpful. It should also lead to several non-Microsoft implementations of these functions. Note that this standards submission includes all the deprecated functions present in the C standard itself. We deprecated other Microsoft extensions, such as _splitpath, which has been replaced by _splitpath_s, but have not submitted these to standardization since the original standard does not cover these areas. We're hoping to submit the C++ library changes for similar standardization at a later date - once we have more experience of using them and broader feedback from the community - and once we've completed the standardization work on the C portions of the library. As a general point, if anyone comes across something deprecated that they think should not be, please do open a bug. As someone noted earlier in the thread, a couple of these did slip through, and we have recently un-deprecated them. We've applied this library to pretty large codebases, so we have some experience, but your experience would be very useful to us. I wrote a long article on these issues for MSDN magazine. It has more details and examples of these issues. http://msdn.microsoft.com/msdnmag/issues/05/05/SafeCandC/default.aspx As an initial step when upgrading a project, _CRT_SECURE_NO_DEPRECATE (for C functions) and _SCL_SECURE_NO_DEPRECATE (for C++) is a great way to ensure your application compiles. But after that initial phase, I would encourage you to think about how to remove these warnings without disabling them. In many cases, you can make portable code changes to resolve the warning without taking any dependency on our extensions. In others, you may need to write extra helper code or make non-portable changes, but the security benefit is worth it, especially for library code like boost. If someone in the boost community decides to take this on as a project, I would be glad to work with them to help them understand the issues involved and work through any problems. As others have noted, both istream::read and sgetn were mistakenly deprecated, and this has been fixed in our latest builds. I just checked the current sources. [[[ Aside: Someone asked why sgetn got deprecated. Here is a short version of the reason. Consider a much simpler C function, memcpy memcpy(d, s, cnt); You can make an argument that cnt is a count of the source, size of destination or "both". A reasonable function to add is memcpy_s, which clarifies the distinction memcpy_s(d, dsize, s, scnt); There has been debate and feedback on how much safer this form is. Having a consistent pattern where a destination pointer is followed by its size is a significant aid to secure code reviews, static analysis of code, and consistency among buffer writing code. On the other side, many find one integer parameter here to be "enough". Early in our implementation we deprecated memcpy (and other cases like this), and added memcpy_s. Later in our implementation, we chose to undeprecate memcpy (and its ilk), and keep memcpy_s as an alternative for those who want it. sgetn got deprecated (and undeprecated) for similar reasons. Essentially, we changed our implementation for a few functions based on customer and user feedback, and when we did that, we missed sgetn. ]]] When you see a warning from std::copy, it means that we believe you've given us an "unchecked" iterator - either one that is a pointer, or one that we cannot establish is "checked". If you have custom iterators, you can use traits to mark them as checked. If you have code samples that warn where you think they shouldn't please submit an issue in the product feedback centre, and we'll investigate and fix if possible. The sooner we get these, the better, as we are getting close to our release. Someone noted that their b2 wouldn't install, perhaps because of beta 1 detritus. If you can't install b2, you likely won't be able to install the final version either. I suggest you open a feedback issue and get suggestions on how to clean out your machine. Wherever we have deprecated something, we have a recommended replacement. Often, the compiler warning will tell you exactly what this replacement is. For a complex case (passing a pointer to std::copy), it's not always possible for us to be this precise, but you will get general advice. I will keep an eye on this thread, but please feel free to follow up with me directly as well if you need more information Martyn Lovell Development Lead Visual C++ Libraries aka martynl@microsoft.com

Pete, Thanks for the feedback. We too spent some time debating the granularity of deprecation. We do understand that different developers will take different positions on which functions are 'dangerous'. What we wanted to do was draw on our experience of real security holes we've fixed and deprecate the functions which are a common source of security problems. This ensures that, by default, we're not encouraging developers to use functions which we know to be unsafe. Gets may be the very unusual case here - an API which it is almost impossible to use safely. However, both fopen and strcpy share a similar property - that it is possible to use them safely, but easy and common to misuse them. Strcpy has the obvious well-understood buffer overrun risks, while fopen has the similarly well-understood file-access-permissions problems. The list of functions we chose to deprecate was honed after working with developers inside and outside of Microsoft, after looking at lots of code fixes made to resolve security problems and after considering the impact of each deprecation on the codebase of your application. It's changed significantly during the lifetime of the project as we've received feedback - we've both added and removed deprecations. Personally, I would not want to use any of the current deprecated functions in an application that I wanted to ship and secure. Because the cost of ensuring they are used correctly is much higher than the cost of using the safer ones to start with. Giving deprecation-subsets based on specific security problems would not have been easy (since some functions have more than one problem), and would have complicated the use of this feature significantly. Instead, we decided to support a default set of deprecations that match our understanding of what is 'unsafe', and then allow you to create your own deprecations if you don't like ours. If you have your own list of functions to deprecate, you can always replace ours. Because #pragma deprecate is a general compiler feature, you can deprecate any sub or superset of functions that you wish to. It's also very easy for you to distribute a header file with your preferred set of deprecations for all your projects. If we find there are common deprecation-subsets that we don't support, then we'd definitely want to revisit the issue of deprecation-subsets in our next version. However, our experience so far has been that customers either want more deprecated than we have (for example, because they also want to deprecate some Win32 functions, or some library functions), or they don't want any deprecation at all (for example, because they are writing portable POSIX code). If you have specific functions that you think should not be deprecated, we'd definitely be interested to hear which they are and why. We are still in the phase of the product where we can respond to feedback, and as was noted elsewhere in this thread, we've recently found a couple of cases where we had wrongly deprecated something.

The choice between portable but "deprecated" and non-portable but secure isn't very appealing either, but that's a separate issue.

This is definitely an issue we want to address better in future. We have already been working thru standardization to try to ensure there are portable and safe choices available.

I'd prefer the undeprecated version of sprintf to be snprintf instead of sprintf_s, and the undeprecated version of swprintf to be swprintf (the standard version of which wasn't insecure, only the Microsoft nonstandard variation was.)

First I should note that in VC8 for the first time we provide a standard-conformant swprintf (with the extra int parameter, and the other standard-conformant behaviours). We also provide a #define to revert to the 'old' VC7 swprintf. Second, I think it's important to note that neither C99 snprintf nor C90 swprintf are safe - because both allow the highly-risky %n construct. At the last C standard committee meeting we had a representative from CERT who reemphasized to us how important it was to address the %n issue, which has been exploited in the real world. Third, even if you ignore the %n issue, our experience with strncpy has taught us that these functions need two integer parameters, not one, so that programmer-intended truncation can be distinguished from buggy buffer overrun. snprintf_s addresses both of these issues. --- Let me end by encouraging you to turn on deprecation for a project you care about, and see whether it deprecates anything inappropriate. I'd be delighted to get any feedback you might have on our default deprecations, so that we can ensure we've got the correct list. Thanks again, Martyn Lovell Development Lead Visual C++ Libraries