How do I see if a domain uses DNSSEC

TL;DR

It is very difficult to diagnose, do not do it yourself, do not think that a single DNS query or whois output can really fully answer the question, it is more complicated.

If you trust them, the following tools are useful and make life simpler:

• https://dnssec-debugger.verisignlabs.com/ as reported long ago in another answer
• https://dnsviz.net/ a very nice display of a DNS hierarchy based on a specific query, with
• https://zonemaster.net/ a generic zone checker

At least the last two are also software you can download and install locally to do the same tests; you also have dig, or its successor for DNSSEC which is delv (see below), and unbound provides the drill utility for equivalent feature set.

"I need to figure out is if the domain hare marked as using DNSSEC at the NIC."

This is not what is relevant or is poorly phrased based on the following of your question.

What is written in whois output is not useful: it can indeed show sometimes DNSSEC: Yes or some equivalent, but Whois and DNS are two separate things, and if you want to work on DNS issues, you should stay in DNS land, so let us just ignore whois for now.

Going back to dig +dnssec <domain> dnskey, this is the good direction but starts with two big problems:

1. You are using dig without specifying with @ what nameserver your query. Hence the reply you will get will come from some recursive nameserver that you may or may not control, that may or may not lie to you, and whose path to you may or may not control and in the latter case the answers can be modified in transit; to resolve this point you need in fact to query one of the authoritative nameserver of the domain, hence you will first need to find them; this may get complicated because you need to use DNSSEC to ensure you get valid replies in all your queries and at the same time any error in DNSSEC will give you SERVFAIL as reply
2. Second big problem is that basically your query will show if some DNSKEY is published with the zone data, and any signature but:
   1. it does not ensure that the recursive resolver you asked has validated anything (hence the signatures can be all bogus) because to do that you need the +nocdflag not the +dnssec (that one triggers display of the signatures, aka the RRSIG records); +cdflag is a flag to disable validation in fact, so you need to invert it (as a resolver may validate by default, and in that case, comparing dig with dig +cd results can help explain if an observed error is related to DNSSEC or not (all DNSSEC failures are currently just returning SERVFAIL which is a generic error code that can happen for a myriad of other cases not related to DNSSEC; there are works ongoing to add richer error reports to DNS exchanges)
   2. and finally, even if everything clicks right there, the fact that the end domain has published a DNSKEY does not mean at all that it is DNSSEC enabled because for that to work it must have a DS record matching that specific key but published on the parent authoritative nameservers, that is those of the registry; without such record (and its signature with a key itself published, and itself correspoding to some other DS record one level higher and so on recursively up to the DNS root), even if a DNSKEY is published it will never be trusted, so the domain does not really have DNSSEC.

Validating like a resolver does

So to start everything from scratch in fact and doing it right, what you need to do is to do what a recursive validating nameserver will do: it will DNSSEC validate your domain (or fail). This is the only test that proves the domain is DNSSEC enabled because that means it has published what is needed, the parent is also publishing what it needs, and so on.

Of course, redoing all of this manually on your side is a bad idea because DNSSEC validation is complicated.

What you will do instead is, either install a local validating resolver like unbound or use a library like getdns which will take care of all of this for you, or you use a remote recursive nameserver that validates DNSSEC for you if and only if you fully trust both the nameserver in question and all the network path between you and it (or you use DoH or DoT nowadays to wrap your DNS exchanges into a TLS stream). Because if you use a remote server that you can not trust it can lie to you about DNSSEC validation results, and if you trust the nameserer but not the network path then an active attacker can modify the results before they come to you from the recursive nameserver.

Note that recent versions of bind provide delv which is kind of the successor to dig specifically for DNSSEC related troubleshooting: https://kb.isc.org/docs/aa-01152

BIND 9.10 contains a new debugging tool that is a successor to dig. So, of course, we had to name it delv. It works very much like dig, but it understands DNSSEC better.

delv +vtrace clearly shows all the validation and the DS/DNSKEY records retrieval at each step.

DNSSEC displayed in whois

And finally to go back to this point, and discuss what it really means.

If a registry shows a one point in some whois output some indication that the domain is "signed", that DNSSEC is active, it means only one very narrow thing: at some point in the past (which could be long ago), the registrar sponsoring this domain name, on behalf of its client, sent cryptographic material (the equivalent of either a DNSKEY or a DS record, depending on registry policies; if this is DS the registrar expects the registry to publish it as is in the registry authoritative nameserver; if this is a DNSKEY instead, the registry will compute itself the DS value to publish; sometimes the registrar has to send both so tht the registry can double check if the DS was computed correctly from the DNSKEY) to the registry, typically through EPP, and some moments later (few hours/days) the DS record appeared in registry authoritative nameservers.

But:

1. Few registries nowadays do check at update time, so a registrar can send a request to add a DS record when there is absolutely no DNSKEY published by the child zone. This will result in DNSSEC: yes in whois output but the domain will fail for any resolving nameserver
2. Even if everything was set up correctly at the moment this update happened, the nameservers could have been changed (change of nameservers for a domain that is signed is a difficult problem, especially if old provider is not cooperative, and has no good univeral foolproof solution, except stopping to sign the domain for some time, then resign it after the nameserver changes)
3. Even without changing the nameservers themselves, the content of the zone could be changed, by error or voluntarily, hence dropping the DNSKEY while the DS is still published, doing the same effect as in first point. This happens far more often than expected/hoped for.

Note that some registries do asynchronous DNSSEC checks on all domains they publish, and warn registrars and/or end clients, if their domain is not setup correctly anymore.

Use the dnssec debugger from verisign: http://dnssec-debugger.verisignlabs.com/.