Microsoft To Banish Memcpy() 486
kyriacos notes that Microsoft will be adding memcpy() to its list of function calls banned under its secure development lifecycle. This reader asks, "I was wondering how advanced C/C++ programmers view this move. Do you find this having a negative impact on the flexibility of the language, and do you think it will restrict the creativity of the programmer?"
No - there are plenty of safer alternatives (Score:5, Insightful)
Lame story (Trying for flamebait here?)
Should have been done 30 years ago. (Score:2, Insightful)
This should have been done thirty years ago.
A half-measure, at best... (Score:1, Insightful)
AFAIK memcpy() is only one of the many ways in which you can hang any half-awake coder himself in C (some manage it when awake, but let's say charitable - I'm ignoring the chorus asking "why?").
Well done for thinking about security, boohoo for going straight to the press and trying to milk it instead of bloody DOING something for a change. Trust comes from casual "oh that? Yeah, we thought that was unsafe so we improved it" instead of trumpeting high and wide "look everyone, we bought a new padlock. Look how shiny it is, and how big" whilst still having a wooden backdoor with a simple latch only.
Do, don't talk. Show me. We've 20 years of marketing so pardon me for being cynical.
=Smidge=
They should go one better... (Score:5, Insightful)
...and pop up a message box asking the user to confirm they want to copy the memory, and if they press OK then they should have to enter a captcha.
Seriously though, how is it supposed to make your code safer if you pass the size you think your destination buffer is? With memcpy, that size is implicitly greater or equal to the copy size and it's the caller's responsibility to make sure this is the case. Putting bounds checking into the copy function is ridiculous if you're responsible for passing the bounds yourself, and it goes against basic good design. I'm surprised they aren't passing the source buffer size too, just to be extra safe. Also, what happened to the __restrict keyword? It's strangely absent from the memcpy_s function declaration.
=Smidge=
Re:No - there are plenty of safer alternatives (Score:3, Insightful)
And they aren't even removed, but (by default) a warning is issued when using them. I'd say it's a good move - passing the size of the destination buffer is usually not that complicated.
#define memcpy memmove (Score:3, Insightful)
I have often used memcpy instead of strcpy when I have known the length of the strings, and also known the destination to be large enough.
I'm guessing many developers will just #define memcpy to something else and continue as nothing happened.
then why are you using C? (Score:2, Insightful)
If you consider memcpy too dangerous then you should be using something besides C. If you're using C++ and memcpy then you really do need to know what both you and the compiler are doing.
The difference bewteen memcpy() and strcpy() (Score:3, Insightful)
The problem with strcpy() and sprintf() and like functions is that you don't know when calling them the length of the source to be copied into the supplied buffer. But with memcpy() you specify this length.
Frequently, the size of the target is calculated at run time, so bugs in memcpy() tend to be in the area of this calculation, rather than in not checking if the source fits the target.
Any lack of memcpy() would be easy to overcome, just use
memcpy_s (dst, len, src, len)
which is functionally identical to
memcpy (dst, src, len)
Re:No - there are plenty of safer alternatives (Score:4, Insightful)
safe versions - if you prefer to blindly program away, not worrying about where your objects end up in memory. But - what is "safe"? Is there any replacement for properly testing all I/O from all possible sources?
Re:Should have been done 30 years ago. (Score:5, Insightful)
Why? I can see some justification on the strXXX functions where you don't know how many bytes are going to be copied unless you call strlen first, but in memcpy you pass how many bytes to copy in as a parameter. So this is to protect programmers who can't do math?
Re:Isn't security the programmer's responsibility? (Score:1, Insightful)
>Isn't security the programmer's responsibility?
You must be kidding. The average code monkey has failed miserably at that responsibility, time after time, for decades.
>(FP btw)
Fail.
Silly and useless (Score:5, Insightful)
This is nothing like sprintf. In sprintf there is no way to know how much data will be created ahead of time, so limit on buffer size is useful to make sure there is no buffer overrun.
With memcpy it is *precisely* known how much data will be copied. It is right there, 3rd parameter. If a developer can't do "if (sizetocopy = sizeofdstbuffer)", it is just as unlikely that he will be able to properly state that additional parameter that specifies the destination buffer size.
Of course if Microsoft is so concerned with security, why the heck did it take them years to add snptinf()? All this is is another attempt to make crossplatform development that much harder (much like all those "obsolete" POSIX functions that will barf warnings unless you use a cryptic define).
That said, if this silliness ever becomes a rule, I have an easy solution:
#define memcpy(dst, src, size) memcpy_s((dst), (src), (size), (size))
Problemo solved, now let's go actually write some real code.
Re:No mention of memmove... (Score:3, Insightful)
Okay, I'm obviously missing something here. How is having an extra parameter for the destination size any safer? I always thought the third parameter to memcpy was the amount of data to copy, and since obviously it should never be set to anything larger than the size of the destination, how will having the destination size explicitly passed in help any?
Or are we just talking about a convenience feature that will make it easier for lazy programmers?
When will MS learn? (Score:5, Insightful)
Yes, you read that right. Microsoft is deprecating parts of an ISO Standard all by themselves. Not that this should surprise anyone. I would have absolutely no objection to them proposing to WG14 to deprecate those functions; heck, I'd encourage it! But besides going out and deciding to 'deprecate' parts of the standards, the replacement functions actually violate those same standards.
And the warnings are irritating. You can't write a nice cross-platform library without either spewing tons of warnings or having to put in a bunch of #defines to shut the compiler up. And if you do that, your users get irritated if they depend on these warnings because you just turned them off (and of course, if you don't, they'll complain that your library is unsafe).
Screw Microsoft.
Re:No mention of memmove... (Score:2, Insightful)
instead of
If they've been screwing up and using the wrong size for the number of bytes to copy, what's going to stop them from screwing up and putting the wrong size for the size of the destination buffer? Nothing! Now the coders that have been using something like
for the last parameter for years will have to change their code. Oh well...it's job security for someone.
Re:Not for C++ (Score:3, Insightful)
If = is ambiguous, then you must have a habit of abusing it. While you can overload = to mean anything you want, I suppose, it would seem like you should try to preserve the general notion of the assignment operator in C and C++, which is that = never modifies the right-hand side of the equal sign, only the left hand side.
"Does it mean duplicate the contents, transfer the contents and clear the original copy, or just swap the contents of the items, which might be quicker."
I would say if you are trying to stay true to the way the operator behaves in the language when you overload it, the last two operations would NOT be overloaded to the equal sign, because they modify the right-hand argument. Give them a meaningful method name instead.
Workaround (Score:2, Insightful)
Re:What an idiotic idea. (Score:3, Insightful)
Re:When will MS learn? (Score:5, Insightful)
In case anyone is curious, this is the type of thing that coppro is talking about:
c:\Program Files\Microsoft Visual Studio 8\VC\include\io.h(318) : see declaration of 'close'
Message: 'The POSIX name for this item is deprecated. Instead, use the ISO C++ conformant name: _close. See online help for details.'
Now, as far as I know, no ISO body has deprecated functions like close(2), open(2), read(2), and write(2). And I've always heard that methods that start with an underscore are internal compiler functions and shouldn't be called directly. I don't know why the MS compiler writers think they can do this, but it is really annoying to get hundreds of warnings like this when compiling. In addition, it hides legitimate warnings that could indicate real problems.
As to the article in question, I can't think of any good reason why memcpy(3C) would be considered unsafe, since it specifies the amount of memory to copy. Sure, you could use it to copy outside the bounds of dst, but that's just calling it incorrectly. It's not like sprintf(3C) where you could easily accidentally write outside the bounds of the string.
easy fix (Score:5, Insightful)
Just write a one-liner that replaces all calls to memcpy with a call to memcpy_s, duplicating the size parameter.
I'm only half-joking. This is exactly how people will (mis)use memcpy_s. If you want safe memory access, you need to ban the entire C language. For those cases where you need C, you'll just have to make sure your programmers know what they're doing.
How to easily ... (Score:4, Insightful)
How to easily make your code compliant with the new safety requirements:
#define memcpy(dest,src,len) memcpy_s(dest,len,src,len)
Re:Think about it (Score:3, Insightful)
Some of these reactions are quite funny.
The goal of asking you to specify the length of the destination buffer is to force you to think about the data you're working with *while* you're writing the code and not afterwards in an unconnected security audit. Furthermore, it provides documentation to other people reading the code who may not have the same mental model of what's going on as you do. And as usual "other people" includes you, six months after you wrote the code.
Uh-huh. Because everyone is not just going to add a line of code to one of the base headers:
#define memcpy(dst, len, src) memcpy_s((dst),(len),(src),(len))
C is not a safe language, and stupid programmers will always find ways to mess it up. There are safer languages where you can't hang yourself as easily, and if you don't understand C, you should use them. Microsoft can't fix this problem.
Re:No - there are plenty of safer alternatives (Score:5, Insightful)
It's a psychological thing. Having a separate parameter for the size of the destination buffer forces the programmer to think about what that size is. Too often programmers call memcpy passing the size of the data that needs copying and forget to check that the destination is big enough. And that's why we see so many buffer overflows.
If you never make this mistake continue to use memcpy. I don't care and neither does Microsoft.
The wole thing is just a bunch of nonsense (Score:5, Insightful)
Firstly, the specification of C anf C++ standard library is governed by the corresponding standard commitee. Microsoft has absolutely no authority to "banish" anything from neither C nor C++. They can deprecate it in their .NET code, C# etc., but it has absolutely no relevance to C and C++ languages. So, why would the author of the original question direct it to "advanced C and C++" programmers is beyond me. In general, C and C++ programmers will never know about this "interesting" development.
Secondly, the tryly unsafe and useless functions in the C standard library are the functions like "gets", which offer absolutely no protection agains buffer overflow, regardless of how careful the develoiper is. Functions like 'memcpy', on the other hand, offer sufficient protection to a qualified developer. There's absolutely no sentiment against these functions in C/C++ community and there is absolutely no possiblity of these functions to get deprecated as long as C language exists.
Re:malloc() and free() (Score:2, Insightful)
Well there's a reason Java and C# have GCs to free you from having to worry about calling malloc and free..
Shooting themselves in the foot. (Score:3, Insightful)
Now, all that's going to happen is that programmers are going to write their own memcpy-like routines using a quicky for-loop or something. It'll be just as bug prone, and harder to detect via automated source code analysis.
Re:No - there are plenty of safer alternatives (Score:2, Insightful)
It would have been nicer if it returned the number of non-null bytes copied, so you could do a quick compare with len to check for the unterminated case.
Re:No - there are plenty of safer alternatives (Score:5, Insightful)
It still will not help.
If they are a sloppy enough programmer not to look at what is going on, and to ensure the size of the destination, they will be sloppy enough to use the same dratted variable in both spots, drool all over the keyboard and move on to the next sloppy bit of code.
Re:No - there are plenty of safer alternatives (Score:4, Insightful)
Re:When will MS learn? (Score:4, Insightful)
That's correct, because ISO C++ never included those functions in the first place. POSIX != ISO C. (Not that MSVC is on any kind of reasonable schedule for keeping up with ISO standards, but that's a whole different issue...)
Basically MS is deprecating their own terrible implementation of some POSIX compatibility. This is actually required for ISO C compliance: the compiler is not supposed to define a bunch of extraneous functions in the global namespace, because they might conflict with your names. Once those functions are removed entirely (and I believe you can #define them away right now) you can implement your own compatibility functions for software you're porting to Windows.
Now, this is all entirely separate from the SDL warnings GP is complaining about, which show up when you use standard ISO C functions like strcpy, sprintf, and apparently now memcpy. Which, honestly, I wish weren't quite so irritatingly implemented, although I'm torn because using those functions really is terrible.
It's not really that worth getting up in arms about, though, because JESUS CHRIST there's a compiler flag to disable the warnings, just put it in your makefile and quit bitching already!
Re:No - there are plenty of safer alternatives (Score:4, Insightful)
Technically one size argument is enough, but in a large enough software project the code that allocates the destination buffer is maintained separately from the code that copies into it. Any failure in communication (e.g. building against an outdated library) will lead to someone's linker writing a binary with code that will overrun a buffer.
With an explicit destination size parameter, the buffer copy code is no longer as sensitive to changes at the allocation site. A breakdown in communication will lead to a binary that produces a controlled runtime error instead of a buffer overrun.
Microsoft Can't Deprecate POSIX Functions (Score:1, Insightful)
Besides:
#pragma warning( disable: 4996 )
Thanks
Re:No - there are plenty of safer alternatives (Score:3, Insightful)
That's physically impossible, even given infinite time. Read up on the halting problem.
No it's not. A computer is not a Turing machine - it has finite memory (=> finite # of states), an algorithm has to halt or visit a state it's already been in.
And to expand on the GP for those that didn't RTFA, they replaced Memcpy with a memcpy that forced you to state the size of the destination buffer, which is a constant time operation, and a much needed one. So this only forces C coders to make their code a little more clear.
Fair enough, well done MS. But their new memcpy can be lied to (memcpy_s(dst, 9999, src, 40)) and guys who aren't keeping track of (and checking) their remaining destination size are the guys likely to lie to memcpy_s
Re:No - there are plenty of safer alternatives (Score:5, Insightful)
What Microsoft is trying to do here is to eliminate a low hanging fruit of software security that has led to hundreds if not thousands of buffer overflow conditions and associated vulnerabilities/exploits.
They might be trying, but they are failing, because the mistake that leads to the error in the first place (miscalculating destination buffer size) has the same effect (buffer overrun) whether you use memcpy() or memcpy_s().
Re:Let Java do it for you. (Score:3, Insightful)
If you wrote a program that used 8+ gigs of memory that means you're an incompetent code monkey.
I have an hourly job that processes about 8GB of input data files. We found that copying the data to a tmpfs filesystem instead of leaving it on a HDD cut the work time from 20 minutes to about 30 seconds because the job necessarily requires an enormous number of random seek()s. Since mmap()ing a file on tmpfs is roughly identical to read()ing the whole file into RAM, I guess that makes me an incompetent code monkey.
Of course, my boss who got a 40x speedup in exchange for $250 worth of RAM might see things differently from your inexperienced little self. Sometimes the Real World throws pretty big datasets at you.
Re:No - there are plenty of safer alternatives (Score:3, Insightful)
So copying with the destination buffer size specified makes it safe? How so?
If I knew that the value I want to copy won't fit into the destination buffer, I wouldn't copy it there. Simple as that. Oh, because the size of the source might be variable? Then maybe the coder should do a size_ssize_d?size_s:size_d as the size argument. Taking a variable that isn't under the coder's full control as a size in any memory manipulation operation is asking for trouble anyway.
Do you think it will be safer now that you "force" the programmer to specify the maximum destination size? If people are sloppy enough to not check sizes, they'll simply pass the same parameter twice and the security you add is zero.
Re:No - there are plenty of safer alternatives (Score:3, Insightful)
When it comes to programming languages, that approach just means either lots more dead or broken code, or a lot less code AND a lot less good code.
There's a higher percentage of C programs where "an attacker can execute arbitrary code of the attacker's choice", compared to say Java or Python programs. Just a look at Bugtraq over the years.
I'm half joking but there might be fewer than 10 people in the world who can write secure, reliable AND useful C programs (useful to more than just one person
Re:No - there are plenty of safer alternatives (Score:5, Insightful)
Re:No - there are plenty of safer alternatives (Score:5, Insightful)
Whilst you are correct, if Microsoft is going to essentially replace the standard C library with one that has an incompatible API, why not just call it a new library and have done with it?
Or, better yet, if security really was the goal, develop a C-like language that was secure by design?
By simply making things awkward for people to write portable code, all they do is ensure that there are multiple code bases for projects (which increases the opportunity for error) or ensures that people won't write portably. Which is a more likely goal, given who we are talking about.
Re:No - there are plenty of safer alternatives (Score:4, Insightful)
I understand the problem you are describing, but I fail to see how this solution addresses it. If there is already a disconnect between the programmer doing the copying and the programmer doing the allocating, then making the programmer doing the copying repeat himself is not going to fix the problem.
The only problem this function solves is buffer over flows caused by a programmer calculating a number of bytes to copy at runtime (e.g. by reading it from a Content-Length header) and failing to check the calculated value against what he believes is the actual size of the buffer. If the value that he believes to be the size of the buffer is wrong, changing from memcpy to memcpy_s will not catch the mistake. In other words, changing from memcpy to memcpy_s will only protect against sloppy programmers, and if they don't understand what the function is supposed to be protecting them from (which is likely) they'll probably just use the same value for copy_size and dst_size anyway (or switch to memmove), which will completely defeat the purpose of blacklisting memcpy in the first place.
Not to mention, if you're doing any pointer arithmetic and writing to an offset some number of bytes past *buffer, then passing the size of *buffer doesn't really help, unless the function is smart enough to know that (I don't see how it could be unless we pass that as a parameter as well), or the user is smart enough to calculate the remaining size of *buffer. If the user is one of the sloppy programmers that this function is meant to protect against in the first place, I think that is highly unlikely, don't you?
Re:No - there are plenty of safer alternatives (Score:3, Insightful)
And then why don't you make it compile to non-native code, so you can do code analysis at runtime? Might as well give it a good standard library that uses all the features so people would try writing stuff. Of course, you can't name it C then, maybe you should give it a catchier name with some punctuation or other pun on the language.
Hey, wait a minute... [microsoft.com]
Re:No - there are plenty of safer alternatives (Score:3, Insightful)
Who do you think has to write those high level wrappers? memcpy is one of the most ridiculously popular functions in systems level C/C++ code, especially for copying arrays or sub-arrays, where it can be much faster than a hand-written loop. You can wrap it in a function for every type you need, but that's still a lot of memcpy you have to write properly. Fortunately it's easy and this whole argument is moot.
Re:No - there are plenty of safer alternatives (Score:4, Insightful)
First of all, memcpy IS a libary call.
"but they should have to explain their need and the benefit over using a higher level wrapper to lots and lots of people."
One source tree, many O/S's. Memcpy is a ANSI C library call, I have been using it for more than 20yrs without a problem. IF MS want to pop up a warning that tells me my source will compile on gcc I can't stop them from doing so.
Re:No - there are plenty of safer alternatives (Score:1, Insightful)
Congratulations! Your bad programming silently truncates data when the source is too large for the destination. This, of course, is yet another software defect, which may cause your program to behave incorrectly, crash, or be exploitable by an attacker.
Depending on the application, a truncation may be exploitable by causing a buffer overrun elsewhere in the program. So no, your code is not magically safe from buffer overruns. Perhaps you should be programming in C#/Java, since this stuff is clearly too hard for you.
Mixed views... (Score:3, Insightful)
I guess Microsoft is trying to get people that don't understand C++ to program better in the language. IMHO, this will not solve fundamental problems with people programming C++, just cause them to learn the language slower by not having an experience working through a bastard of a bug. I think the world (and not just MS) needs to realize that writing a piece of complex software is difficult. Bugs like an errant memcpy of a subclass into a baseclass instance are *very* easy bugs to solve if you're looking for them. Memory overwrites are easy to detect if you are looking for them. Bad pointers are easy to detect if you're looking for them.
Such problems will always exist in software. I've found a fair number in my own code. What we *really* need to do is train a better caliber of programmer. OTOH, Microsoft seems hell-bent on trying to make writing software easy to do