Do I need to close() both FileReader and BufferedReader?

As others have pointed out, you only need to close the outer wrapper.

BufferedReader reader = new BufferedReader(new FileReader(fileName));

There is a very slim chance that this could leak a file handle if the BufferedReader constructor threw an exception (e.g. OutOfMemoryError). If your app is in this state, how careful your clean up needs to be might depend on how critical it is that you don't deprive the OS of resources it might want to allocate to other programs.

The Closeable interface can be used if a wrapper constructor is likely to fail in Java 5 or 6:

Reader reader = new FileReader(fileName);
Closeable resource = reader;
try {
  BufferedReader buffered = new BufferedReader(reader);
  resource = buffered;
  // TODO: input
} finally {
  resource.close();
}

Java 7 code should use the try-with-resources pattern:

try (Reader reader = new FileReader(fileName);
    BufferedReader buffered = new BufferedReader(reader)) {
  // TODO: input
}

no.

BufferedReader.close()

closes the stream according to javadoc for BufferedReader and InputStreamReader

as well as

FileReader.close()

does.