C string handling refers to a group of functions implementing operations on
strings in the C standard library. Various operations, such as copying,
concatenation, tokenization and searching are supported.
The only support for strings in the C programming language itself is that the
compiler will translate a quoted string constant into a null-terminated string,
which is stored in static memory. The C standard library, however, provides a
large number of commonly used functions designed to manipulate these
null-terminated strings. Definitions
A string is a contiguous sequence of code units terminated by the first zero
code. In C, there are two types of strings: string, which is sometimes
called byte string which uses the type chars as code units, and wide string
which uses the type wchar_t as code units.
A common misconception is that all char arrays are strings, because string
literals are converted to arrays during the compilation phase. It is important
to remember that a string ends at the first zero code unit. An array or string
literal that contains a zero before the last byte therefore contains a string,
or possibly several strings, but is not itself a string. Conversely, it is
possible to create a char array that is not null-terminated and is thus not a
string: char is often used as a small integer when needing to save memory.
The term pointer to a string is used in C to describe a pointer to the initial
byte of a string. In C, pointers are used to pass strings to functions.
Documentation will often use the term string to mean pointer to a string.
The term length of a string is used in C to describe the number of bytes
preceding the zero byte. strlen is a standardised function commonly used to
determine the length of a string. A common mistake is to not realize that a
string uses one more unit of memory than this length, in order to store the zero
that ends the string. Character encodings
Each string ends at the first occurrence of the zero code unit of the appropriate
kind. Consequently, a byte string can contain non-NUL characters in ASCII or
any ASCII extension, but not characters in encodings such as UTF-16. The
encodings that can be stored in wide strings are defined by the width of
wchar_t. In most implementations, wchar_t is at least 16 bits, and so all
16-bit encodings, such as UCS-2, can be stored. If wchar_t is 32-bits, then
32-bit encodings, such as UTF-32, can be stored.
Variable-width encodings can be used in both byte strings and wide strings.
String length and offsets are measured in bytes or wchar_t, not in
"characters", which can be confusing to beginning programmers. UTF-8 and Shift
JIS are often used in C byte strings, while UTF-16 is often used in C wide
strings when wchar_t is 16 bits. Truncating strings with variable length
characters using functions like strncpy can produce invalid sequences at the end
of the string. This can be unsafe if the truncated parts are interpreted by code
that assumes the input is valid. Support for Unicode literals such as
char foo[512] = "φωωβαρ";(UTF-8) or wchar_t foo[512] = L"φωωβαρ"; is
implementation defined, and may require that the source code be in the same
encoding. Some compilers or editors will require entering all non-ASCII
characters as \xNN sequences for each byte of UTF-8, and/or \uNNNN for each
word of UTF-16. Overview of functions
Most of the functions that operate on C strings are declared in the string.h
header, while functions that operate on C wide strings are declared in the
wchar.h header. These headers also contain declarations of functions used
for handling memory buffers; the name is thus something of a misnomer.
Functions declared in string.h are extremely popular since, as a part of
the C standard library, they are guaranteed to work on any platform which
supports C. However, some security issues exist with these functions, such
as potential buffer overflows when not used carefully and properly, causing the
programmers to prefer safer and possibly less portable variants, out of which
some popular ones are listed below. Some of these functions also violate
const-correctness by accepting a const string pointer and returning a non-const
pointer within the string. To correct this, some have been separated into two
overloaded functions in the C++ version of the standard library.
In historical documentation the term "character" was often used instead of
"byte" for C strings, which leads many to believe that these functions somehow
do not work for UTF-8. In fact all lengths are defined as being in bytes
and this is true in all implementations, and these functions work as well with
UTF-8 as with single-byte encodings. The BSD documentation has been fixed to make
this clear, but POSIX, Linux, and Windows documentation still uses
"character" in many places where "byte" or "wchar_t" is the correct term.
Functions for handling memory buffers can process sequences of bytes that
include null-byte as part of the data. Names of these functions typically start
with mem, as opposite to the str prefix. = Constants and types=
= Functions= Multibyte functions
"state" is used by encodings that rely on history such as shift states. This is
not needed by UTF-8 or UTF-32. UTF-16 uses them to keep track of surrogate
pairs and to hide the fact that it actually is a multi-word encoding.
= Numeric conversions= The C standard library contains several
functions for numeric conversions. The functions that deal with byte strings
are defined in the stdlib.h header. The functions that deal with wide strings
are defined in the wchar.h header. The strtoxxx functions are not
const-correct, since they accept a const string pointer and return a non-const
pointer within the string. Also, since the Normative Amendment 1, atoxx
functions are considered subsumed by strtoxxx functions, for which reason
neither C95 nor any later standard provides wide-character versions of
these functions. Popular extensions
Replacements = strcpy / strcat=
Despite the well-established need to replace strcat and strcpy with functions
that do not allow buffer overflows, no accepted standard has arisen. This is
partly due to the mistaken belief by many C programmers that strncat and
strncpy have the desired behavior; however, neither function was designed
for this, and the behavior and arguments are non-intuitive and often written
incorrectly even by expert programmers. As part of its 2004 Security Development
Lifecycle, Microsoft introduced a family of "secure" functions, such as strcpy_s
and strcat_s; these functions were later standardized with some minor changes,
and are now part of C11 and ISO/IEC WDTR 24731. These functions perform runtime
integrity checks of their arguments; if the checks fail, a user-specified
"runtime-constraint handler" function is called. If the user has not specified
such a function, the default behavior is implementation-defined. Microsoft's C
runtime will abort the program when the constraints are violated. Some functions
perform destructive operations before calling the runtime-constraint handler;
for example, strcat_s sets the destination to the empty string, which
can make it difficult to recover from error conditions or debug them. These
functions attracted considerable criticism because initially they were
implemented only on Windows, and at the same time warning messages started to be
produced by Microsoft Visual C++, suggesting the programmers to use these
functions instead of standard ones. This has been speculated by some to be an
attempt by Microsoft to lock developers into its platform. Although open-source
implementations of these functions are available, these functions are not
present in common Unix C libraries. More popular strlcat and strlcpy
functions date from 1999 or earlier; they have been criticized on the basis
of encouraging the use of C strings and creating more problems than initially
trying to solve. Consequently, they have not been included in the GNU C library,
although they are implemented in OpenBSD, FreeBSD, NetBSD, Solaris, Mac
OS X, and QNX. The lack of GNU C library support has not stopped various library
authors from using it and bundling a replacement, among other SDL, GLib,
ffmpeg, rsync, and even internally in the Linux kernel. Open source
implementations for these functions are available.
= memcpy / memmove= strcpy is typically used to copy C
strings when the string length isn't known in advance of the copy, but if the
length is known then memcpy is faster because it doesn't continually look for
the 'nul' end-of-string terminator during the copy operation. Also, memcpy
is faster than memmove, because memmove must handle over-lapping memory copying.
Though memcpy is fast, it can be optimized for slightly faster speed and
slightly reduced size for preferred input parameters, such as size is
greater than zero, and both pointers aren't NULL. This is useful in speed
critical code, such as Interrupt handler functions.
A vast majority of code doesn't check the return value from the memcpy
function. In many situations, such as using static buffers, the source and
destination addresses are known at compile time, so NULL address checking
can be safely removed. Most of the time, one or more bytes are
copied. The for() and while() are pre-test loop operators which require an
initial test and branch, where as do-while() will move one byte before a
post-test is performed at the bottom of the loop, thus executing fewer
instructions. The pre-decrement may be slightly faster than post-decrment on
some older processor architectures. If the "copy size" is a known fixed
size, the loop can be "unrolled" to save the loop overhead.
In 1983, a loop unrolling concept was created, now known as Duff's device,
which copies a power-of-2 amount of bytes per loop cycle. The algorithm can
be optimized for 2, 4, 8, 16, 32, ... bytes. Though the original "Duff's
device" algorithm copies up to 8 bytes per loop, the following optimized
example copies 4 bytes per loop. In situations where the source and
destination pointers are located on aligned address boundaries, more than
one byte can be copied per loop using the processor native memory width, such
as 4 bytes per loop in 32-bit processors. Even higher widths are
possible on some processors, such as 64-bits or 128-bits per loop.
If copying a very large block of memory, a DMA controller can be used to increase
the speed. See also
C syntax § Strings – source code syntax, including backslash escape
sequences String functions
Null-terminated string Notes
References External links
Fast memcpy in C, multiple C coding examples to target different types of
CPU instruction architectures