Duplicate & switch case
Python, 56 54 bytes
lambda s:''.join(c+c.swapcase()*c.isalpha()for c in s)
Test it on Ideone.
JavaScript ES6, 70 68 66 64 bytes
Saved 2 bytes thanks to @Kevin Lau - not Kenny
Saved 2 bytes thanks to @Cᴏɴᴏʀ O'Bʀɪᴇɴ
s=>s.replace(/[A-Z]/gi,l=>l+l[`to${l<"a"?"Low":"Upp"}erCase`]())
Explanation
This uses a really hacky:
l[`to${l<"a"?"Low":"Upp"}erCase`]()
which ungolfed is:
l[`to${
l < "a" ?
"Low" :
"Upp"
}erCase`]()
Basically l < "a"
checks if the code point of the letter is less then the code point of a
(therefore being an uppercase letter). If it is it'll do to + Low + erCase
which becomed l['toLowerCase']()
and makes the character lowercase. `
quotes allow string formatting so essentially you can think of:
`to${l < "a" ?"Low" : "Upp"}erCase`
as: "to" + (l<"a" ? "Low" : "Upp") + "erCase"
which generates the function to call (make the string upper or lower case). We put this in square brackets [ ... ]
which lets us access a property given its name as a string. This returns the appropriate function and then we just call it.
Jelly, 5 bytes
żŒsQ€
Try it online!
How it works
żŒsQ€ Main link. Argument: s (string)
Œs Yield s with swapped case.
ż Zip s with the result.
Q€ Unique each; deduplicate each pair of characters.