File and path names containing particular characters or character sequences can cause problems when used in the construction of a file or path name:

  • Leading dashes: Leading dashes can cause problems when programs are called with the file name as a parameter because the first character or characters of the file name might be interpreted as an option switch.
  • Control characters, such as newlines, carriage returns, and escape: Control characters in a file name can cause unexpected results from shell scripts and in logging.
  • Spaces: Spaces can cause problems with scripts and when double quotes are not used to surround the file name.
  • Invalid character encodings: Character encodings can make it difficult to perform proper validation of file and path names. (See IDS11-J. Perform any string modifications before validation).
  • Namespace prefixing and conventions: Namespace prefixes can cause unexpected and potentially insecure behavior when included in a path name.
  • Command interpreters, scripts, and parsers: Characters that have special meaning when processed by a command interpreter, shell, or parser.

As a result of the influence of MS-DOS, 8.3 file names of the form xxxxxxxx.xxx, where x denotes an alphanumeric character, are generally supported by modern systems. On some platforms, file names are case sensitive, and on other platforms, they are case insensitive. VU#439395 is an example of a vulnerability resulting from a failure to deal appropriately with case sensitivity issues [VU#439395].  Developers should generate file and path names using a safe subset of ASCII characters and, for security critical applications, only accept names that use these characters.

Noncompliant Code Example

In the following noncompliant code example, unsafe characters are used as part of a file name.

File f = new File("A\uD8AB");
OutputStream out = new FileOutputStream(f);

A platform is free to define its own mapping of the unsafe characters. For example, when tested on an Ubuntu Linux distribution, this noncompliant code example resulted in the following file name:

A?

Compliant Solution

Use a descriptive file name, containing only the subset of ASCII previously described.

File f = new File("name.ext");
OutputStream out = new FileOutputStream(f);

Noncompliant Code Example

This noncompliant code example creates a file with input from the user without sanitizing the input.

public static void main(String[] args) throws Exception {
  if (args.length < 1) {
    // Handle error
  }
  File f = new File(args[0]);
  OutputStream out = new FileOutputStream(f);
  // ...
}

No checks are performed on the file name to prevent troublesome characters. If an attacker knew this code was in a program used to create or rename files that would later be used in a script or automated process of some sort, the attacker could choose particular characters in the output file name to confuse the later process for malicious purposes.

Compliant Solution

This compliant solution  uses a whitelist to reject file names containing unsafe characters.  Further input validation may be necessary, for example, to ensure that a file or directory name does not end with a period.

public static void main(String[] args) throws Exception {
  if (args.length < 1) {
    // Handle error
  }
  String filename = args[0];

  Pattern pattern = Pattern.compile("[^A-Za-z0-9._]");
  Matcher matcher = pattern.matcher(filename);
  if (matcher.find()) {
    // File name contains bad chars; handle error
  }
  File f = new File(filename);
  OutputStream out = new FileOutputStream(f);
  // ...
}

 

Exceptions

FIO99-J-EX0: A program may accept a file or path name that uses "unsafe" characters provided that the developer has determined that the file is not used in a restricted sink such as a command interpreter, shell, parser,logger, or other complex subsystem that attaches a particular meaning to these characters.

Risk Assessment

Failing to use only a safe subset of ASCII can result in misinterpreted data.

Rule

Severity

Likelihood

Remediation Cost

Priority

Level

IDS05-J

medium

unlikely

medium

P4

L3

Automated Detection

ToolVersionCheckerDescription
The Checker Framework

2.1.3

Tainting CheckerTrust and security errors (see Chapter 8)

Related Guidelines

Bibliography

ISO/IEC 646-1991

ISO 7-Bit Coded Character Set for Information Interchange

[Kuhn 2006]

UTF-8 and Unicode FAQ for UNIX/Linux

[Wheeler 2003]

5.4, "File Names"

[VU#439395] 

 


  

12 Comments

  1. I can't tell if this rule is saying that the following characters:

    % & + , - . : = _
    

    can or must not be used in filenames.

    There is further confusion introduced by the discussion of case sensitivity. If we are not requiring all upper or lower case letters in filenames, I would suggest removing this discussion as inappropriate, misleading, and confusing in the context of this rule.

    1. I made some of the changes I threatened to make above. There are still a couple of alternate explanations for disallowing characters other than letters, numbers, and portable punctuation including scripts and name-separator characters.

    2. I removed the colon from the list of valid characters because it is a reserved character under Windows (http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx

      The following reserved characters:

      • < (less than)
      • > (greater than)
      • : (colon)
      • " (double quote)
      • / (forward slash)
      • \ (backslash)
      • | (vertical bar or pipe)
      • ? (question mark)
      • * (asterisk)

       

      The apple documentation (https://documentation.apple.com/en/finalcutpro/usermanual/index.html#chapter=3%26section=4%26tasks=true) lists the following:

      File separators
      : (colon)
      / (forward-slash)
      \ (backward-slash)
      You cannot use colons (:) in the names of files and folders because Mac OS 9 (Classic) uses this character to separate directories in pathnames. In addition, some applications may not allow you to use slashes (/) in the names of items.
      These characters are directory separators for Mac OS 9, Mac OS X, and DOS (Windows) respectively.
      Special characters not included in your native alphabet
      ¢ ™
      These characters may not be supported or may be difficult to work with when exported to certain file formats, such as EDL, OMF, or XML.
      Punctuation marks, parentheses, quotation marks, brackets, and operators
      . , [ ] { } ( ) ! ; " ' * ? < > |
      These characters are often used in scripting and programming languages.
      White space characters such as spaces, tabs, new lines, and carriage returns (the last two are uncommon)
       
      White space is handled differently in different programming languages and operating systems, so certain processing scripts and applications may treat your files differently than expected. The most conservative filenames avoid all use of whitespace characters, and use the (underscore) _ character instead.

      Any reason we should just whittle down this list to alphanumeric, periods (which are necessary for 8.3 files, and '_' characters?

  2. the link of VU#439395 points to Java Bibliography page.

    I changed the link to point to Bibliography page of C Coding Standard space, but it looks like

    Confulence recognizes this link as external site, instead of within another space.

    anyone please fix the link (two occurrences) as appropriate...

     

  3. The exception needs more work. What is a restricted sink, in this context?

    1. I expanded a bit, mostly by repeating text from the introduction.

  4. It is unclear what "name-space separation characters" is meant to indicate.  My best guess is this:

    http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#namespaces

    If this is the intent, it might be better to say "namespace prefixing and conventions" or something like that.

  5. There are certain constraints on file names that aren't supported by this rule.  For example from http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#namespaces again:

    Do not end a file or directory name with a space or a period. Although the underlying file system may support such names, the Windows shell and user interface does not. However, it is acceptable to specify a period as the first character of a name. For example, ".temp".

    This constraint cannot be enforced by simply using a safe subset of ASCII.  

    Perhaps this should be called "Use conservative file naming conventions" which is definitely a concise but not precise title, but we can be more precise in the rule which I think should say use 8.3 with these characters "[^A-Za-z0-9._]" but don't end in a period. Even the code to check for this will be sort of complex.

  6. This rule may also belong in FIO, because creating file names is half the equation.

  7. I am now thinking that this rule should be demoted to a recommendation, or scrapped altogether. Points to consider:


    We can't come up with a subset of characters that are safe for all filesystems. (That is, a safe one that was sufficiently limited would be too restrictive to be useful). Java's write-once-run-anywhere phillosophy betrays us here. We would have to forsake platform independence here and say: Respect the limitations of filenames supported by your filesystem. It's not enough to focus on platforms, but rather on filesystems. You could mount an HFS+ filesystem on a Linux box, and then you have to deal with Mac-style filename restrictions on the Linux platform.

    This also applies to the C & C++ analogues to this rule (although they be merely recommendations)

    Finally, all the bad cases cited in the intro are addressed by other rules:

    Leading dashes are only a problem when an untrusted filename is used in a shell command, which violates IDS07-J. Do not pass untrusted, unsanitized data to the Runtime.exec() method. The same applies to spaces and control characters, such as newlines, carriage returns, and escape.

    As for invalid character encodings and namespace prefixing, the solution is addressed well in FIO16-J. Canonicalize path names before validating them.


    We could demote this to a recommendation (the C & C++ analogues are recommendations) and therefore we don't have to define a precise specification of what filenames are allowable. But I'm not sure this rule contains any info not covered by other rules.

    1. I'm not against that idea.  I see the main problem being that there are other solutions, such as escaping weird characters in names.  Let's go ahead and change to guideline and also move to the File I/O section at the same time.