Skip to end of metadata
Go to start of metadata

Serialization can extend the lifetime of objects, preventing their garbage collection. The ObjectOutputStream ensures that each object is written to the stream only once by retaining a reference (or handle) to each object written to the stream. When a previously written object is subsequently written to the stream again, it is replaced with a reference to the originally written data in the stream. Note that this substitution takes place regardless whether the object's contents have changed in the interim. It requires a table of references to be maintained to keep track of previously serialized objects. This table of references prevents garbage collection of the previously serialized objects because the garbage collector cannot collect object instances referred to by live references.

This behavior is both desirable and correct for data that may contain arbitrary object graphs, especially when the graphs are fully allocated and constructed prior to serialization. However, it can lead to memory exhaustion when serializing data that lacks references to other objects being serialized and can be allocated in part or in full after serialization has begun. One such example is serializing a data stream from an external sensor. In such cases, programs must take additional action to avoid memory exhaustion. That is, programs reading in independent serialized data must reset the table of references between reads to prevent memory exhaustion.

This rule is a specific instance of the more general MSC05-J. Do not exhaust heap space.

Noncompliant Code Example

This noncompliant code example reads and serializes data from an external sensor. Each invocation of the readSensorData() method returns a newly created SensorData instance, each containing one megabyte of data. SensorData instances are pure data streams, containing data and arrays but lacking references to other SensorData objects.

As already described, the ObjectOutputStream maintains a cache of previously written objects. Consequently, all SensorData objects remain alive until the cache itself becomes garbage-collected. An OutOfMemoryError can occure because the stream remains open while new objects are being written to it.

class SensorData implements Serializable {
  // 1 MB of data per instance!
  ... 
  public static SensorData readSensorData() {...}
  public static boolean isAvailable() {...}
}

class SerializeSensorData {
  public static void main(String[] args) throws IOException {
    ObjectOutputStream out = null;
    try {
      out = new ObjectOutputStream(
          new BufferedOutputStream(new FileOutputStream("ser.dat")));
      while (SensorData.isAvailable()) {
        // Note that each SensorData object is 1 MB in size
        SensorData sd = SensorData.readSensorData();
        out.writeObject(sd);
      }
    } finally {
      if (out != null) {
        out.close();
      }
    }
  }
}

Compliant Solution

This compliant solution takes advantage of the known properties of the sensor data by resetting the output stream after each write. The reset clears the output stream's internal object cache; consequently, the cache no longer maintains references to previously written SensorData objects. The garbage collector can collect SensorData instances that are no longer needed.

class SerializeSensorData {
  public static void main(String[] args) throws IOException {
    ObjectOutputStream out = null;
    try {
      out = new ObjectOutputStream(
          new BufferedOutputStream(new FileOutputStream("ser.dat")));
      while (SensorData.isAvailable()) {
        // Note that each SensorData object is 1 MB in size
        SensorData sd = SensorData.readSensorData();
        out.writeObject(sd);
        out.reset(); // Reset the stream
      }
    } finally {
      if (out != null) {
        out.close();
      }
    }
  }
}

Risk Assessment

Memory and resource leaks during serialization can result in a resource exhaustion attack or can crash the Java Virtual Machine.

Rule

Severity

Likelihood

Remediation Cost

Priority

Level

SER10-J

Low

Unlikely

Low

P3

L3

Related Guidelines

MITRE CWE

CWE-400, Uncontrolled Resource Consumption (aka "Resource Exhaustion")
CWE-770, Allocation of Resources without Limits or Throttling

Bibliography

[API 2014]

 

[Harold 2006]

Section 13.4, "Performance"

[Sun 2006]

Serialization Specification

 


5 Comments

    • Add a try-finally. out.close() should occur in a finally
    1. fixed (but prob need to fix this in other rules)

      1. Resource acquisition written ignoring the conventional way of doing it almost always results in bugs. Here we have the very common NPE, and also some people (including me) object to exceptions in construction of decorators causing resource leaks. Write resource acquisition as release as:

        final Resource rawResource = resources.acquire();
        try {
            DecoratedResource resource = new DecoratedResource(rawResource);
            use(resource);
            resource.flush();
        } finally {
            rawResource.close();
        }
        
        

        Don't try to ram in a catch into the same try. If the hole is the wrong shape for the catch clause, don't force it in.

        Also, the SensorData class implies mutable statics, which are evil.

        1. I changed the SensorData.continue() method to an isAvailable() method. I suppose it might look like there is a static boolean, but its also possible there is no such var, and the class merely queries the sensor.

          1. There's going to be a mutable static there somewhere, even if the mutability were outside of the machine.

            Mutable statics are poor design. That poor design has problems including but not limited to:

            • Mobile code security: It's no longer a capability. Perhaps could be hacked with a security manager check.
            • Testing: Sensor cannot be replaced with a mock. Therefore, testing always requires: the external hardware to be present, repeatable stimulus applied to the sensor and prevents testing at the unit level.
            • Dependency: This code always requires the sensor code. The sensor and code are not switchable.
            • Initialisation: There are implicit constraints added to initialisation order.