Skip to end of metadata
Go to start of metadata

Multiuser systems allow multiple users with different privileges to share a file system. Each user in such an environment must be able to determine which files are shared and which are private, and each user must be able to enforce these decisions.

Unfortunately, a wide variety of file system vulnerabilities can be exploited by an attacker to gain access to files for which they lack sufficient privileges, particularly when operating on files that reside in shared directories in which multiple users may create, move, or delete files. Privilege escalation is also possible when these programs run with elevated privileges. A number of file system properties and capabilities can be exploited by an attacker, including file links, device files, and shared file access. To prevent vulnerabilities, a program must operate only on files in secure directories.

A directory is secure with respect to a particular user if only the user and the system administrator are allowed to create, move, or delete files inside the directory. Furthermore, each parent directory must itself be a secure directory up to and including the root directory. On most systems, home or user directories are secure by default and only shared directories are insecure.

Similar to shared files, file links can be swapped out and may not always point to the intended location. As a result, file links in shared directories are untrusted and should not be operated on (see FIO15-J. Do not operate on untrusted file links).

Device Files

File names on many operating systems may be used to access device files. Device files are used to access hardware and peripherals. Reserved MS-DOS device names include AUX, CON, PRN, COM1, and LPT1. Character special files and block special files are POSIX device files that direct operations on the files to the appropriate device drivers.

Performing operations on device files intended only for ordinary character or binary files can result in crashes and denial-of-service (DoS) attacks. For example, when Windows attempts to interpret the device name as a file resource, it performs an invalid resource access that usually results in a crash [Howard 2002].

Device files in POSIX can be a security risk when an attacker can access them in an unauthorized way. For instance, if malicious programs can read or write to the /dev/kmem device, they may be able to alter their own priority, user ID, or other attributes of their process or they may simply crash the system. Similarly, access to disk devices, tape devices, network devices, and terminals being used by other processes can also lead to problems [Garfinkel 1996].

On Linux, it is possible to lock certain applications by attempting to read or write data on devices rather than files. Consider the following device path names:

/dev/mouse
/dev/console
/dev/tty0
/dev/zero

A Web browser that failed to check for these devices would allow an attacker to create a website with image tags such as <IMG src="file:///dev/mouse"> that would lock the user's mouse.

Shared File Access

On many systems, files can be simultaneously accessed by concurrent processes. Exclusive access grants unrestricted file access to the locking process while denying access to all other processes, eliminating the potential for a race condition on the locked region. The java.nio.channels.FileLock class may be used for file locking. According to the Java API, Class FileLock [API 2014], documentation,

A file lock is either exclusive or shared. A shared lock prevents other concurrently running programs from acquiring an overlapping exclusive lock but does allow them to acquire overlapping shared locks. An exclusive lock prevents other programs from acquiring an overlapping lock of either type. Once it is released, a lock has no further effect on the locks that may be acquired by other programs.

Shared locks support concurrent read access from multiple processes; exclusive locks support exclusive write access. File locks provide protection across processes, but they do not provide protection from multiple threads within a single process. Both shared locks and exclusive locks eliminate the potential for a cross-process race condition on the locked region. Exclusive locks provide mutual exclusion; shared locks prevent alteration of the state of the locked file region (one of the required properties for a data race).

The Java API [API 2014] documentation states that "whether or not a lock actually prevents another program from accessing the content of the locked region is system-dependent and consequently unspecified."

Microsoft Windows uses a mandatory file-locking mechanism that prevents processes from accessing a locked file region.

Linux implements both mandatory locks and advisory locks. Advisory locks are not enforced by the operating system, which diminishes their value from a security perspective. Unfortunately, the mandatory file lock in Linux is generally impractical for the following reasons:

  • Mandatory locking is supported only by certain network file systems.
  • File systems must be mounted with support for mandatory locking, which is disabled by default.
  • Locking relies on the group ID bit, which can be turned off by another process (thereby defeating the lock).
  • The lock is implicitly dropped if the holding process closes any descriptor of the file.

Noncompliant Code Example

In this noncompliant code example, an attacker could specify the name of a locked device or a first in, first out (FIFO) file, causing the program to hang when opening the file:

String file = /* Provided by user */;
InputStream in = null;
try {
  in = new FileInputStream(file);
  // ...
} finally {
  try {
    if (in !=null) { in.close();}
  } catch (IOException x) {
    // Handle error
  }
}

Noncompliant Code Example

This noncompliant code example uses the try-with-resources statement (introduced in Java SE 7) to open the file. The try-with-resources statement guarantees the file's successful closure if an exception is thrown, but this code is subject to the same vulnerabilities as the previous example.

String filename = /* Provided by user */;
Path path = new File(filename).toPath();
try (InputStream in = Files.newInputStream(path)) {
  // Read file
} catch (IOException x) {
  // Handle error
}

Noncompliant Code Example (isRegularFile())

This noncompliant code example first checks that the file is a regular file (using the NIO.2 API) before opening it:

String filename = /* Provided by user */
Path path = new File(filename).toPath();
try {
  BasicFileAttributes attr =
      Files.readAttributes(path, BasicFileAttributes.class);

  // Check
  if (!attr.isRegularFile()) {
    System.out.println("Not a regular file");
    return;
  }
  // Other necessary checks
  
  // Use
  try (InputStream in = Files.newInputStream(path)) {
    // Read file
  }
} catch (IOException x) {
  // Handle error
}

This test can still be circumvented by a symbolic link. By default, the readAttributes() method follows symbolic links and reads the file attributes of the final target of the link. The result is that the program may reference a file other than the one intended.

Noncompliant Code Example (NOFOLLOW_LINKS)

This noncompliant code example checks the file by calling the readAttributes() method with the NOFOLLOW_LINKS link option to prevent the method from following symbolic links. This approach allows the detection of symbolic links because the isRegularFile() check is carried out on the symbolic link file and not on the final target of the link.

String filename = /* Provided by user */;
Path path = new File(filename).toPath();
try {
  BasicFileAttributes attr = Files.readAttributes(
      path, BasicFileAttributes.class, LinkOption.NOFOLLOW_LINKS);

  // Check
  if (!attr.isRegularFile()) {
    System.out.println("Not a regular file");
    return;
  }
  // Other necessary checks

  // Use
  try (InputStream in = Files.newInputStream(path)) {
    // Read file
  };
} catch (IOException x) {
  // Handle error
}

This code is still vulnerable to a time-of-check, time-of-use (TOCTOU) race condition. For example, an attacker can replace the regular file with a file link or device file after the code has completed its checks but before it opens the file.

Noncompliant Code Example (POSIX: Check-Use-Check)

This noncompliant code example performs the necessary checks and then opens the file. After opening the file, it performs a second check to make sure that the file has not been moved and that the file opened is the same file that was checked. This approach reduces the chance that an attacker has changed the file between checking and then opening the file. In both checks, the file's fileKey attribute is examined. The fileKey attribute serves as a unique key for identifying files and is more reliable than the path name as on indicator of the file's identity.

The SE 7 Documentation [J2SE 2011] describes the fileKey attribute:

Returns an object that uniquely identifies the given file, or null if a file key is not available. On some platforms or file systems it is possible to use an identifier, or a combination of identifiers to uniquely identify a file. Such identifiers are important for operations such as file tree traversal in file systems that support symbolic links or file systems that allow a file to be an entry in more than one directory. On UNIX file systems, for example, the device ID and inode are commonly used for such purposes.

The file key returned by this method can only be guaranteed to be unique if the file system and files remain static. Whether a file system re-uses identifiers after a file is deleted is implementation dependent and consequently unspecified.

File keys returned by this method can be compared for equality and are suitable for use in collections. If the file system and files remain static, and two files are the same with non-null file keys, then their file keys are equal.

As noted in the documentation, FileKey cannot be used if it is not available. The fileKey() method returns null on Windows. Consequently, this solution is available only on systems such as POSIX in which fileKey() does not return null.

String filename = /* Provided by user */;
Path path = new File(filename).toPath();
try {
  BasicFileAttributes attr = Files.readAttributes(
      path, BasicFileAttributes.class, LinkOption.NOFOLLOW_LINKS);
  Object fileKey = attr.fileKey();

  // Check
  if (!attr.isRegularFile()) {
    System.out.println("Not a regular file");
    return;
  }
  // Other necessary checks

  // Use
  try (InputStream in = Files.newInputStream(path)) {

    // Check
    BasicFileAttributes attr2 = Files.readAttributes(
        path, BasicFileAttributes.class, LinkOption.NOFOLLOW_LINKS
    );
    Object fileKey2 = attr2.fileKey();
    if (!fileKey.equals(fileKey2)) {
      System.out.println("File has been tampered with");
    }

    // Read file
  };
} catch (IOException x) {
  // Handle error
}

Although this code goes to great lengths to prevent an attacker from successfully tricking it into opening the wrong file, it still has several vulnerabilities:

  • The TOCTOU race condition still exists between the first check and open. During this race window, an attacker can replace the regular file with a symbolic link or other nonregular file. The second check detects this race condition but does not eliminate it.
  • An attacker could subvert this code by letting the check operate on a regular file, substituting the nonregular file for the open, and then resubstituting the regular file to circumvent the second check. This vulnerability exists because Java lacks a mechanism to obtain file attributes from a file by any means other than the file name, and the binding of the file name to a file object is reasserted every time the file name is used in an operation. Consequently, an attacker can still swap a file for a nefarious file, such as a symbolic link.
  • A system with hard links allows an attacker to construct a malicious file that is a hard link to a protected file. Hard links cannot be reliably detected by a program and can foil canonicalization attempts, which are prescribed by FIO16-J. Canonicalize path names before validating them.

Compliant Solution (POSIX: Secure Directory)

Because of the potential for race conditions and the inherent accessibility of shared directories, files must be operated on only in secure directories. Because programs may run with reduced privileges and lack the facilities to construct a secure directory, a program may need to throw an exception if it determines that a given path name is not in a secure directory.

Following is a POSIX-specific implementation of an isInSecureDir() method. This method ensures that the supplied file and all directories above it are owned by either the user or the system administrator, that each directory lacks write access for any other users, and that directories above the given file may not be deleted or renamed by any users other than the system administrator.

public static boolean isInSecureDir(Path file) {
  return isInSecureDir(file, null);
}
public static boolean isInSecureDir(Path file, UserPrincipal user) {
   return isInSecureDir(file, user, 5);
}

/**
 * Indicates whether file lives in a secure directory relative
 * to the program's user
 * @param file Path to test
 * @param user User to test. If null, defaults to current user
 * @param symlinkDepth Number of symbolic links allowed
 * @return true if file's directory is secure.
 */
public static boolean isInSecureDir(Path file, UserPrincipal user,
                                    int symlinkDepth) {
  if (!file.isAbsolute()) {
    file = file.toAbsolutePath();
  } if (symlinkDepth <=0) {
      // Too many levels of symbolic links
      return false;
    }

  // Get UserPrincipal for specified user and superuser
  FileSystem fileSystem =
      Paths.get(file.getRoot().toString()).getFileSystem();
  UserPrincipalLookupService upls = 
      fileSystem.getUserPrincipalLookupService();
  UserPrincipal root = null;
  try {
    root = upls.lookupPrincipalByName("root");
    if (user == null) {
      user = upls.lookupPrincipalByName(System.getProperty("user.name"));
    }
    if (root == null || user == null) {
      return false;
    }
  } catch (IOException x) {
    return false;
  }

  // If any parent dirs (from root on down) are not secure,
  // dir is not secure
  for (int i = 1; i <= file.getNameCount(); i++) {
    Path partialPath = Paths.get(file.getRoot().toString(),
                                 file.subpath(0, i).toString());

    try {
      if (Files.isSymbolicLink(partialPath)) {
        if (!isInSecureDir(Files.readSymbolicLink(partialPath),)) {
                           user, symlinkDepth - 1)
          // Symbolic link, linked-to dir not secure
          return false;
        }
      } else {
        UserPrincipal owner = Files.getOwner(partialPath);
        if (!user.equals(owner) && !root.equals(owner)) {
          // dir owned by someone else, not secure
          return false;
        }
        PosixFileAttributes attr =
            Files.readAttributes(partialPath, PosixFileAttributes.class);
        Set<PosixFilePermission> perms = attr.permissions();
        if (perms.contains(PosixFilePermission.GROUP_WRITE) ||
            perms.contains(PosixFilePermission.OTHERS_WRITE)) {
          // Someone else can write files, not secure
          return false;
        }
      }
    } catch (IOException x) {
      return false;
    }
  }

  return true;
}

When checking directories, it is important to traverse from the root directory to the leaf directory to avoid a dangerous race condition whereby an attacker who has privileges to at least one of the directories can rename and re-create a directory after the privilege verification of subdirectories but before the verification of the tampered directory.

If the path contains any symbolic links, this routine will recursively invoke itself on the linked-to directory and ensure it is also secure. A symlinked directory may be secure if both its source and linked-to directory are secure. The method checks every directory in the path, ensuring that every directory is owned by the current user or the system administrator and that all directories in the path prevent other users from creating, deleting, or renaming files.

On POSIX systems, disabling group and world write access to a directory prevents modification by anyone other than the owner of the directory and the system administrator.

Note that this method is effective only on file systems that are fully compatible with POSIX file access permissions; it may behave incorrectly for file systems with other permission mechanisms.

The following compliant solution uses the isInSecureDir() method to ensure that an attacker cannot tamper with the file to be opened and subsequently removed. Note that once the path name of a directory has been checked using isInSecureDir(), all further file operations on that directory must be performed using the same path. This compliant solution also performs the same checks performed by the previous examples, such as making sure the requested file is a regular file, and not a symbolic link, device file, or other special file.

String filename = /* Provided by user */;
Path path = new File(filename).toPath();
try {
  if (!isInSecureDir(path)) {
    System.out.println("File not in secure directory");
    return;
  }

  BasicFileAttributes attr = Files.readAttributes(
      path, BasicFileAttributes.class, LinkOption.NOFOLLOW_LINKS);

  // Check
  if (!attr.isRegularFile()) {
    System.out.println("Not a regular file");
    return;
  }
  // Other necessary checks

  try (InputStream in = Files.newInputStream(path)) {
    // Read file
  }
} catch (IOException x) {
  // Handle error
}

Programs with elevated privileges may need to write files to directories owned by unprivileged users. One example is a mail daemon that reads a mail message from one user and places it in a directory owned by another user. In such cases, the mail daemon should assume the privileges of a user when reading or writing files on behalf of that user, in which case all file access should occur in secure directories relative to that user. When a program with elevated privileges must write files on its own behalf, these files should be in secure directories relative to the privileges of the program (such as directories accessible only by the system administrator).

Exceptions

FIO00-J-EX0: Programs that operate on single-user systems or on systems that have no shared directories or no possibility of file system vulnerabilities do not need to ensure that files are maintained in secure directories before operating on them.

Risk Assessment

Performing operations on files in shared directories can result in DoS attacks. If the program has elevated privileges, privilege escalation exploits are possible.

Rule

Severity

Likelihood

Remediation Cost

Priority

Level

FIO00-J

Medium

Unlikely

Medium

P4

L3

Related Guidelines

Android Implementation Details

On Android, the SD card ( /sdcard or /mnt/sdcard ) is shared among multiple applications, so sensitive files should not be stored on the SD card (see DRD00-J. Do not store sensitive information on external storage (SD card)).

Bibliography

[API 2014]

Class FileLock
   Methods createTempFile
   Method delete
   Method deleteOnExit

[Darwin 2004]

Section 11.5, "Creating a Transient File"

[Garfinkel 1996]

Section 5.6, "Device Files"

[Howard 2002]

Chapter 11, "Canonical Representation Issues"

[J2SE 2011]

"The try-with-resources Statement"

[JDK Bug 2015]

Bug JDK-4171239
Bug JDK-4405521 
Bug JDK-4631820

[Open Group 2004]

open()

[Secunia 2008]

Secunia Advisory 20132

 


20 Comments

  1. There is a pretty good description of why Compliant Solution (Java 1.7: Check-Use-Check) is still insecure. Why then do we consider it a compliant solution? My first reaction is that this solution is only compliant if performed within a secure directory. Perhaps that was the intent?

      • First, the vul you reference exists because I can't find a Java analogue to fstat(), which operates on the file descriptor and not the path name. I've asked Hawtin about this problem and if it can be rectified, no response.
      • The check-use-check is therefore not perfect, but the best we can do in Java.
      • 'the best we can do' is still damn good. A successful attack requires successful timing of not one but two race windows. Highly unlikely in realtime code.
      • While C has fstat() and therefore we can do better, check-use-check isn't perfect in C either. A malicious file that causes daverse affects when opening is not fixed by check-use-check. eg /dev/mouse
      • Finally, a secure directory mitigates these (and we may want another CS that says so), but it is not always feasible.
  2. Current features & liabilities of the isInSecureDir() method:

    Liabilities:

    • No awareness of sticky bit
    • race condition on path is still exploitable to falsely accept insecure dir, unfixable in Java 7

    Features:

    • Handles relative paths
    • Handles relative & absolute symlinks
    • No memory mgmt problems (Java is good at something (smile)
    • Detects symlink cycles
    • Proper cleanup
  3. Is the check of the symlink level right? It is:

    if (symlinkDepth <=0)

    but looks like that should be just <0 ?

    Also, wouldn't an exception be more appropriate (e.g. IllegalArgumentException or somesuch)?

  4. This rule is extreme, even with the exception. Do we have strong justification for it? It seems overly broad. For example, I'm working with Android now and there are many valid uses of shared directories in Android. A simple example is when you want your photo app to import data from the sdcard. The sdcard is a public shared directory. But, this kind of scenario is typical. For that matter, any interaction with a distributed file system necessitates breaking this rule as the root account on the remote file system (if such  even exists) is necessarily different than the application account and the root account on the current system.

    I vote to remove this rule and downgrade it to a guideline, or substantially specialize it to particular circumstances, but I'm open to any counter-arguments.

    1. First off, this, and all the Java rules are officially 'guidelines'. IIRC we demoted them out of an inability to distniguish rules from recommendations.

      Second, most Android machines are single-user, right? So they would fall under the exception. That is, the user has access to all the files on the system, and need not 'share resources' with another user. Your photo example illustrates the issue. A single user manages both the photo app and the sdcard, and no other user has immediate access. So your user can ensure that the photos are properly shared, and doesn't have to worry about protecting the files from hostile users while doing the transfer.

      1. That's a matter of perspective. Each app is its own user account and I think its helpful to regard each app as its own "user" from a security perspective. So, I'd consider an Android phone a multi-user system.

        1. Well, when we wrote "single-user systems" in the exception, we meant systems that could be used by only one person.. Systems with virtual users (eg the 'www' user most Unix webservers have) or sandboxed systems have a different definition of 'user' than this exception meant.

          An Android system (like any other) can have multiple apps running simultaneously. You might argue that a malicious app could exploit TOCTOU vulnerabilities described in this rule, the same way a malicious person could. But one important difference is that the user (person) still controls what apps live on the device, whereas on a traditional multi-user system, one user has no control over the actions of another user (outside of administrators).

          You can consider each Android app to have its own user, in which case this guideline becomes 'extreme' as you initially suggested. Or you can consider an Android device to be single-user, in which case all the file ops are covered by the exception.

          1. Agreed, but I meant Android only as an example.  The same notion applies to traditional Unix systems. Just because you're accessing a file from an insecure directory doesn't mean that file has any impact in security.  As an extreme example, imagine the cp utility written in Java. Wouldn't it be non-conforming? Or cat. Or head. etc.

            One distinction which might be helpful is whether the file has any influence on control flow or whether it is "parametric" in the contents of the file (so to speak). That is, it is simply moving bits. However, even that distinction is probably not enough because if it is a video player, the same logic applies – who cares if the video is trustworthy.

            1. Agreed, but I meant Android only as an example.  The same notion applies to traditional Unix systems. Just because you're accessing a file from an insecure directory doesn't mean that file has any impact in security.  As an extreme example, imagine the cp utility written in Java. Wouldn't it be non-conforming? Or cat. Or head. etc.

              Grrr...you're taking this rule to a logical extreme. First of all, this rule is fundamentally about POSIX filesystems, not Java, so this rule could apply to C or any other language that operates on POSIX files. Second, the Unix cp command could conform to this rule by refusing to work with files in an insecure directory. But that's still rather draconian, as it precludes cp from working in /tmp. Do we really mean to be so extreme?

              There would be a whole bunch of practicality issues to work out. For instance the check-use-check NCCE is very difficult to exploit...I don't know that it's been done in the wild, it is a theoretical possibility. (Historical note, that code used to be a compliant solution, because it was the best we could do, even in C. We realized that secure dirs were better, and so recommend them.)

              I'd say it is a worthwhile stance to take...forbid new code from working with shared directories, and deprecate their use. I think this is slowly happening in the POSIX world anyway. Perhaps we need an additional rule saying "if you must work with shared directories, use check-use-check and watch out for the following...". Or an exception to this rule.

              I'm really not sure when you must work with shared directories...perhaps Android has examples? Having applications share data does NOT require sharing directories, b/c the permissions system allows for two users to share data; with only one user being able to modify it.

              1. Grrr...you're taking this rule to a logical extreme. First of all, this rule is fundamentally about POSIX filesystems, not Java, so this rule could apply to C or any other language that operates on POSIX files. Second, the Unix cp command could conform to this rule by refusing to work with files in an insecure directory. But that's still rather draconian, as it precludes cp from working in /tmp. Do we really mean to be so extreme?

                David, that's exactly my point. The rule as written is extreme. I'm not by any means suggesting that we should limit use of cp and others, but that the rule does and that's untenable. I can't write a checker for this rule as is. The question, then, is what to do about it. If, as you mentioned earlier all of these rules are just guidelines, then they all implicitly come with the exception of "don't apply when unreasonable" and the rule is fine (although the checker is hopeless). If not, then we should think change the rule or add an appropriate exception, which you seem to be saying as well.

                I think we're in agreement here?

            2. cp would only be nonconforming if it were run as a setuid process.  we probably have to retool this rule to make it clear that the problem is allowing unprivileged users to access privileged files.  I think "Programs must adhere to a well-specified file-usage policy. The default policy is that files may not be accessed from shared directories" is not a very useful rule, not enforceable, and maybe wrong.   The generic guideline should be "every program should have a well-specified security policy".

              always happy to discuss this further.  i've given way too much thought to file vulnerabilities.

              1. Right. The rule is internally consistent, we just need to decide if it is still tenable given that it pretty much forbids insecure directories.

                The TOCTOU exploit works fine even with a non-setuid cp. Suppose I type 'cp /tmp/a /tmp/b', and you substitute a malicious file for /tmp/a at just the right moment. That scenario does not involve setuid or root, but it does involve 'elevated' privileges; in that you cause your data to be used by a process with my privileges. The cp program cannot prevent this; it could only refuse to work with files in /tmp (which is the most common insecure directory).

                I'd like to resolve this soon, as I'm going to be making a Perl version of this rule.

                1. I realized this morning when I got up that there was a problem with non-setuid programs as well, so no argument there.

                  OK, you guys should resolve this and let me know what you come up with. 8^)

                  I think the first step is a more careful characterization of the terms "shared directory" and "secure directory".  I don't believe these are the only two types of directories, as your comment above suggests.

                  I think we have a good definition of "secure directory".  We probably need a better definition of "shared directory".

                  There was a paper from some guys that did a really extensive evaluation of file i/o vulnerabilities.  If you look on the C rules, I think they commented about us not having this quite right.  Maybe we should see what they had to say.

                  1. There was a paper from some guys that did a really extensive evaluation of file i/o vulnerabilities.  If you look on the C rules, I think they commented about us not having this quite right.  Maybe we should see what they had to say.

                    Sounds interesting – do you remember the authors and/or title?

              2. Robert,

                Agreed about the uselessness of my proposed fix, and with David's response to your first point.

  5. Let me try to clarify the rule / guidelines thing.

    When we did the C standard, we thought it was important to separate rules from recommendations.  We used guideline as a term to mean "rule or recommendation".

    We had some problems with this approach, although we still retain it.  First, it is a bit hard to differentiate between the two.  In the end it become a fairly subjective assessment of "would you make someone repair this problem in legacy code, you know, at great cost".  If you could answer yes, it was a rule.  Second, a conforming system only had to conform to the rules, but analyzers would develop analysis for the recommendations as well, if they were analyzable.  This caused much confusion.

    The Java standard had way too many guidelines.  We decided to only keep the rules, and we moved all the recommendations to a separate document.  So these are all rules, and a conforming system must conform to them all.

     

     

  6. So first off, the device stuff should be separated out and put in a separate rule.

  7. in correctly -> incorrectly