Bi-directional translator

JavaScript (ES6),  118 113  109 bytes

s=>s.toUpperCase(A=[':)',...'NF1G]H5S9T[X=Z','{N}','M','--*']).replace(/[{-]..|:?./g,c=>A[A.indexOf(c)^1]||c)

Try it online!

How?

Because the input is guaranteed to contain exclusively [a-zA-Z] and/or the special patterns, we can use the following minimal regular expression to match them all:

/[{-]..|:?./g
 [{-]..        // either "{" or "-" followed by 2 other characters
               // -> to match "{N}" or "--*"
        :?.    // or any character, preceded by an optional ":"
               // -> to match ":)" or any single character

Working on the uppercase version of the input, we look for the position of each matched pattern in the lookup array A[] and replace it with its counterpart, or leave it unchanged if it is not found (which may only happen for ABCDEIJKLOPQRUVWY).


Retina 0.8.2, 70 68 57 bytes

.[}*]|\)

T`l`L
T`-{=519[]:NGTSF\HXZM`Ro
{
{N}
-
--*
:
:)

Try it online! Link includes test cases. Uppercases the result. Explanation:

.[}*]|\)

Remove the suffixes from {N}, --* and :) leaving their first character.

T`l`L

Uppercase everything.

T`-{=519[]:NGTSF\HXZM`Ro

Translate the first characters. The string is mapped to its reverse, so that - maps to M etc. and vice versa. The H is a special character and needs to be quoted. The - is normally a special character but placing it at the beginning avoids this.

{
{N}
-
--*
:
:)

Expand the {, - and : to the full strings again.


05AB1E, 65 63 bytes

u"{N}"¬.:"--*"¬.:":)"¬.:"SFHXZMNGT[]:-{=519"‡"{-:N-)}*"3ιD€нs‡

Uses the same approach as Neil's answer. Probably not going to be too competitive among the golf answers, but there's definitely improvements to be made here (I'm still a novice with 05AB1E, especially with string manipulation).

Explanation:

u            uppercase the input

"{N}"        push '{N}'
     ¬       extract and push head ('{')
      .:     replace all {N} => {

"--*"        push '--*'
     ¬       extract and push head ('-')
      .:     replace all --* => -

":)"         push ':)'
    ¬        extract and push head (':')
     .:      replace all :) => :

"SFH...519"  push lookup string

            bifurcate (push a, reversed(a))
 ‡           transliterate input with lookup

             set up stack for reverse process
             to do { -> {N} ... etc

"{-:N-)}*"   push literal string
3ι           uninterleave by 3
  D          duplicate
   €         apply single function map 
    н        extract first element
     s       swap
      ‡      transliterate

The bi-directional bit is achieved by transliterating with a bifurcated lookup string; each character that should be swapped is in the opposite position with the one it should be swapped with in the lookup string. The main problem (and potential savings) is that I repeat a lot of logic for the "{N}" -> "{" ... part at the beginning as in the "{" -> "N" ... part at the end. Originally, I thought transliterate could go both ways, so I was going to set up the stack carefully at the beginning, but I don't think transliterate likes it when there's a multi-character string in the list (it probably goes by character, rather than doing a replace all?).