How to undo a "Get Latest Version" on TFS
I think the problem is that you do not agree with TFS on what "latest" mean. "Latest" in TFS mean that nothing has changed on the server since you did get latest. It do not mean that what is on the hard drive is equal to the latest version on the server. So TFS shows what it is supposed to, see this question for more: Why doesn't TFS get latest get the latest?
The intended solution for folders on the server that you do not want to have on your local hard drive is "Cloak", as MBulava mentioned. (Right click folder -> Cloak). If you do not wish to have a folder on your hard drive I recommend this solution as it will never be downloaded until you uncloak it, and will show as greyed out and "not downloaded".
If you want to look at the differences between the contents on the hard drive and the server version you can use the "compare folders" feature. It will show that the folder is deleted on the hard drive.
If you want to get the folder mapped and grey (as the other folders you did not download) you can cloak it, and then uncloak it but answer "No" to downloading it now. This is equivalent to not downloading the folder in the first place.
Martin Woodward has a cool TFS Top Tip #11 - Removing source control files from your local file system blog post that answers this problem without resorting to cloaking.
You need to Get Specific Version, change the Version Type to Changeset and specify 1 for Changeset number. His blog post goes into detail about why this works. I've verified this behavior in Team Explorer for Visual Studio 2013, 2015 and 2017.
Update:
If you have got a bunch of folders to process you can use the command line as follows replacing folderName with the relative folder as the Client itemspec or the equivalent Server itempec:
tf.exe get folderName /v:1 /recursive