Do HTML comments (<!-- -->) affect SEO?

They are completely ignored so no, it doesn't affect rankings. The only problem might be if you have such a massive amount of text in comments, that it increases the load time significantly. Users might go elsewhere.

If you're using a server side language, you could always put some comments inside comment tags in that language. I do this for comments I don't want outputted, or for situations where I'm taking down a section and don't want people to be able to view it like so...

<?php // php comment example
/*

<strong>DON'T WANT ANYONE TO SEE THIS</strong>

*/
?>

Your HTML page will be smaller, but your PHP process will be interpretting more... not sure which is more efficient, so for me it comes down to need.