lsof runs very slow in my server

Pass the -n option to skip resolving DNS names of IP connections. This will almost certainly be the biggestion portion of any slowdowns.

If you are certain that you can skip the open TCP or UDP connections, since you mentioned there are too many of them, you can use the dialect-specific option of lsof -X.

lsof -X

Read the man page of lsof and search for '-X' for detailed information.