Do I need to Dispose() or Close() an EventWaitHandle?

Class definitions from MSDN:

public class EventWaitHandle : WaitHandle
public abstract class WaitHandle : MarshalByRefObject, IDisposable

So yes you must as WaitHandle is IDisposable. FxCop would find this as a rule violation if you didn't.

You need to dispose them explicitly. Close() is more appropriate for them as it does call Dispose().

The disposable resource of an EventWaitHandle is actually a SafeHandle (wrapped in a SafeWaitHandle). SafeHandle implements a finalizer, which eventually makes sure the necessary resource is release, so it should be safe to let the garbage collector / finalizer thread handle it in this case.

However, it is always a good idea to explicitly call Dispose() when the resource is no longer needed.

The threading chapter in C# 3.0 in a Nutshell states

This practice is (arguably) acceptable with wait handles because they have a light OS burden (asynchronous delegates rely on exactly this mechanism to release their IAsyncResult's wait handle).