How do you 'cancel' a UdpClient::BeginReceive?
You need to jerk the floor mat, call the UdpClient.Close() method.
That will complete the BeginReceive() call and your callback method will run. When you call EndReceive(), the class lets you know the floor mat is gone and will throw an ObjectDisposedException. So be prepared to handle this exception, wrap the EndReceive() call with try/catch so you catch the ODE and just exit the callback method.
Do note that this tends to get .NET programmers a bit upset, using exceptions for flow control is in general strongly discouraged. You however have to call EndInvoke(), even if you already know that this going to throw an exception. Failure to do so will cause a resource leak that can last a while.