should Class.getResourceAsStream be closed?

You are assuming that Class.getResourceAsStream() will always return a stream that points to a file inside your class' JAR file. This is incorrect. Your classpath may also contains folders, in which case Class.getResourceAsStream() will return a FileInputStream. Some other class loaders might also return other type of resources, such as remote files (in the case of a URLClassLoader).

Even in the case of a JAR file, it is possible that the implementation maintain, by whatever mean, a persistant view inside the JAR file to the compressed bytes of the file you are accessing. Maybe it is holding upon a memory mapped ByteBuffer...

Why take the chance? You should always close streams (and any other Closeable, actually), no matter how they were given to you.

Note that since Java 7, the preferred method to handle closing any resource is definitely the try-with-resources construct. It correctly handles several corner cases that are very hard to manage in hand written code, and yet it is almost as easy to write for you as if you had simply forgot about closing the resource. For example, you may use code like this:

    try (InputStream in = Class.getResourceAsStream("someresource.txt")) {
        // Use the stream as you need to...
    }

    // Then forget about it... and yet, it has been closed properly.

As for detecting leaks, the best strategy is to obtain a memory dump at the time the VM is shut down, then analyze it with some tool. Two popular tools are VisualVM and Eclipse mat.

As a rule of thumb you should close all streams (and ay other types that provide close functionality). It can lead ro resource leaks (memory is one type of resource).

Although there is automatic garbage collection that reclaims memory eventually when an object is no longer referred to, it might still hold native resources that might not be released. Also, if some other long-lived object (e.g. the classloader) keeps a hold on the resource, it will not be claimed by the garbage collector.

If your program is short lived, or f you only open a small number of resources, you might not run into problems, but resource leaks are hard to detect in long-running applications.