Page History

Understanding the difference between text mode and binary mode is important when using functions that operate on file streams. (See FIO14-C. Understand the difference between text mode and binary mode with file streams for more information.)

Section Subclause 7.21.9.2 of the C Standard [ISO/IEC 9899:2011] specifies the following behavior for fseek() when opening a binary file in binary mode:

...

In addition, footnote 268 of Section subclause 7.21.3 has this to saysays:

Setting the file position indicator to end-of-file, as with fseek(file, 0, SEEK_END), has undefined behavior for a binary stream (because of possible trailing null characters) or for any stream with state-dependent encoding that does not assuredly end in the initial shift state.

Seeking to the end of a binary stream in binary mode with fseek() is not meaningfully supported and is not a recommended method for computing the size of a file.

Section Subclause 7.21.9.4 of the C Standard [ISO/IEC 9899:2011] specifies the following behavior for ftell() when opening a text file in text mode:

...

Consequently, the return value of ftell() for streams opened in text mode should never be used for offset calculations other than in calls to fseek().

POSIX [Open Group 2008IEEE Std 1003.1:2013] provides several guarantees that the problems described in the C Standard do not occur on POSIX systems.

First, the fopen() page page says:

The character 'b' shall have no effect, but is allowed for ISO C standard conformance.

This guarantees that binary files are treated the same as text files in POSIX.

Second, the fwrite() page page says:

For each object, size calls shall be made to the fputc() function, taking the values (in order) from an array of unsigned char exactly overlaying the object. The file-position indicator for the stream (if defined) shall be advanced by the number of bytes successfully written.

This indicates means that the file position indicator, and consequently the file size, is directly based on the number of bytes actually written to a file.

...

This code example attempts to open a binary file in binary mode and use fseek() and ftell() to obtain the file size.   This code is noncompliant on systems that do not provide the same guarantees as POSIX. On these systems, setting the file position indicator to the end of the file using fseek() is not guaranteed to work for a binary stream, and consequently, the amount of memory allocated may be incorrect, leading to a potential vulnerability.

...

If the code needs to handle large files, it is preferable to use fseeko() and ftello() because, for some implementations, they can handle larger file offsets than fseek() and ftell() can handle.   If they are used, the file_size variable should have type off_t to avoid the possibility of overflow when assigning the return value of ftello() to it. This solution works only with regular files.

...

This compliant solution uses the Windows _filelength() function to determine the size of the file on a 32-bit OSoperating system.   For a 64-bit OSoperating system, consider using _filelengthi64 instead.

int fd;
long file_size;
char *buffer;

_sopen_s(&fd, "foo.bin", _O_RDONLY, _SH_DENYRW, _S_IREAD);
if (fd == -1) {
  /* Handle error */
}

file_size = _filelength(fd);
if (file_size == -1) {
  /* Handle error */
}

buffer = (char*)malloc(file_size);
if (buffer == NULL) {
  /* Handle error */
}

/* ... */ 

...

This compliant solution uses the Windows GetFileSizeEx() function to determine the size of the file on a 32- or 64-bit OS.operating system:

HANDLE file;
LARGE_INTEGER file_size;
char *buffer;

 
file = CreateFile(TEXT("foo.bin"), GENERIC_READ, 0, NULL, 
                   OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (INVALID_FILE_HANDLE == file) {
  /* Handle error */
}
 
if (!GetFileSizeEx(file, &file_size)) {
  /* Handle error */
}
 
/* 
 * Note: 32-bit portability issue with LARGE_INTEGER
 * truncating to a size_t .
 */
buffer = (char*)malloc(file_size);
if (buffer == NULL) {
  /* Handle error */
}

/* ... */ 

...

This noncompliant code example attempts to open a text file in text mode and use fseek() and ftell() to obtain the file size.:

FILE *fp;
long file_size;
char *buffer;

fp = fopen("foo.txt", "r");
if (fp == NULL) {
  /* Handle error */
}

if (fseek(fp, 0 , SEEK_END) != 0) {
  /* Handle error */
}

file_size = ftell(fp);
if (file_size == -1) {
  /* Handle error */
}

buffer = (char*)malloc(file_size);
if (buffer == NULL) {
  /* Handle error */
}

/* ... */ 

...

The Visual Studio documentation for ftell()  [MSDN] states:

The value returned by ftell may not reflect the physical byte offset for streams opened in text mode, because text mode causes carriage return-linefeed translation. Use ftell with fseek to return to file locations correctly.

...

Because binary files are treated the same as text files in POSIX, you can use either compliant solution can be used for determining the size of a binary file under POSIX to determine the size of a text file as well.

...

Understanding the difference between text mode and binary mode with file streams is critical when working with functions that operate on them. Setting the file position indicator to end-of-file with fseek() has undefined behavior for a binary stream. In addition, the return value of ftell() for streams opened in text mode is useful only in calls to fseek(), not for determining file sizes or for any other use. As such, fstat() or other platform-equivalent functions should be used to determine the size of a file.

Automated Detection

Bibliography

...

...

Image Modified Image Modified Image Modified