Is there a single UAC binary?

UAC is a multi-component architechture implemented by multiple binaries

User Account Control (UAC) refers to several components that together form the UAC architecture. I'll briefly review some of them along with the binaries responsible for their implementation, but first here's an overview of the UAC architecture from the Microsoft Docs article How User Account Control Works:

Local Security Authority (LSA) / Filtered Token

Conceptually the "first" component of UAC is implemented by the Local Security Authority subsystem which handles creation of a user's Access Token during the logon process. Starting with Windows Vista, the logon process was modified so that when an Administrator logs on with UAC enabled, the LSA subsystem generates two separate access tokens for the user:

1. One with full administrator access, and
2. A second "filtered token" with standard user access

As shown here this process is different than that of a standard user logon:

The LSA subsystem service lives in the lsass.exe process.

Virtualization

Added in Windows 7, File and Registry Virtualization is a component of UAC that shims older applications that are not-UAC compliant but only require administrative rights in order to access certain protected areas of the filesystem or Registry:

When an administrative application that is not UAC compliant attempts to write to a protected directory, such as Program Files, UAC gives the application its own virtualized view of the resource it is attempting to change. The virtualized copy is maintained in the user's profile.

Source

By redirection these access attempts to areas that don't require admin permissions, these applications continue to function despite UAC being enabled on the system.

This virtualization is implemented in the Kernel.

Application Information Service

The Application Information Service (AIS) reads an application's manifest and works with the UAC Consent Prompt to determine if an application is allowed to execute with elevated rights (i.e. start in the context of the non-filtered administrative-level access token created at logon). This blog post provides a good overview of its role in the UAC process:

AIS Facilitates the running of interactive applications with additional administrative privileges. If this service is stopped, users will be unable to launch applications with the additional administrative privileges they may require....The shell checks with this service when it launches an application. AIS is the one that reads the manifest and the ‘trustInfo’ xml section that has the requirements for the ‘requestedExecutionLevel’...

Here's a graphic that follows the above quote detailing AIS' role in the UAC Consent Prompt process:

The AIS is implemented in the DLL appinfo.dll which is executed by svchost.exe.

Consent Prompt

@BenN's answer explains the key role of the (in)famous UAC Consent Prompt. This is implemented in consent.exe and is responsible for getting the user's consent or an administrative user's credentials in order to allow launching of an application requiring admin rights.

Secure Desktop

The Secure Desktop is where the UAC Consent Prompt is displayed by default. Microsoft's UACBlog tells us what's unique about this Desktop compared to the User Desktop:

You most commonly interact with [the Secure Desktop] when you log on to Windows since the Logon UI runs on the Secure Desktop. The Secure Desktop’s primary difference from the User Desktop is that only trusted processes running as SYSTEM are allowed to run here (i.e. nothing running as the User’s privilege level) and the path to get to the Secure Desktop from the User Desktop must also be trusted through the entire chain.

The idea behind using it when asking the user's consent to run an application with elevated permissions is that malware cannot imitate the Secure Desktop unless it already has administrative rights, in which case tricking a user into granting them is moot.

---

Conclusion: UAC is not just one binary. It's a fabric of interwoven subsystems.

There are yet other aspects of the UAC architecture not covered here, but this should provide enough evidence for the facts that:

1. UAC is not implemented in a single binary.
2. If enabled, it's an integral part of performing administrative tasks.

Since its introduction in Windows Vista it has been deeply integrated into key portions of the operating system, making it infeasible delete all the code responsible for UAC without breaking other things (such as your ability to logon!)

I think it's safe to say that if you "forcefully deleted" UAC you would break Windows.

As Twisty excellently explained, there are a lot of components that help implement UAC. The part of UAC that people are most familiar with is the elevation/consent dialog:

This is provided by consent.exe, "Consent UI for administrative applications." I tried renaming it in a VM and seeing what happens. As expected, no elevation prompts appear when using "run as administrator" — instead, you get a file-not-found error that blames the thing you're trying to elevate:

Trying to use any Control Panel UI element that requires elevation (i.e. has the shield icon), even if logged in as an administrator, fails with similar errors. Trying to launch administrative things from the Start menu produces a slightly different error:

Depending on the ACL set when doing the rename that broke everything, it could be impossible to fix this from within the OS, since file operations can require elevation (even if they doesn't usually produce the consent dialog). Normal-user-like activities don't seem to be degraded, though.