What are the rules for valid identifiers (e.g. functions, vars, etc) in Bash?

Command identifiers and variable names have different syntaxes. A variable name is restricted to alphanumeric characters and underscore, not starting with a digit. A command name, on the other hand, can be just about anything which doesn't contain bash metacharacters (and even then, they can be quoted).

In bash, function names can be command names, as long as they would be parsed as a WORD without quotes. (Except that, for some reason, they cannot be integers.) However, that is a bash extension. If the target machine is using some other shell (such as dash), it might not work, since the Posix standard shell grammar only allows "NAME" in the function definition form (and also prohibits the use of reserved words).

The question was about "the rules", which has been answered two different ways, each correct in some sense, depending on what you want to call "the rules". Just to flesh out @rici's point that you can shove about any character in a function name, I wrote a small bash script to try to check every possible (0-255) character as a function name, as well as as the second character of a function name:

#!/bin/bash
ASCII=( nul soh stx etx eot enq ack bel bs tab nl vt np cr so si dle \
            dc1 dc2 dc3 dc4 nak syn etb can em sub esc fs gs rs us sp )

for((i=33; i < 127; ++i)); do
    printf -v Hex "%x" $i

    printf -v Chr "\x$Hex"
    ASCII[$i]="$Chr"
done
ASCII[127]=del
for((i=128; i < 256; ++i)); do
    ASCII[$i]=$(printf "0X%x" $i)
done

# ASCII table is now defined

function Test(){
    Illegal=""
    for((i=1; i <= 255; ++i)); do
        Name="$(printf \\$(printf '%03o' $i))"
        eval "function $1$Name(){ return 0; }; $1$Name ;" 2>/dev/null
        if [[ $? -ne 0 ]]; then
            Illegal+=" ${ASCII[$i]}"
            #        echo Illegal: "${ASCII[$i]}"
        fi
    done
    printf "Illegal: %s\n" "$Illegal"
}
echo "$BASH_VERSION"
Test
Test "x"

# can we really do funky crap like this?
function [}{(){
   echo "Let me take you to, funkytown!"
}
[}{    # why yes, we can!
# though editor auto-indent modes may punish us

I actually skip NUL (0x00), as that's the one character bash may object to finding in the input stream. The output from this script was:

4.4.0(1)-release
Illegal:  soh tab nl sp ! " # $ % & ' ( ) * 0 1 2 3 4 5 6 7 8 9 ; < > \ ` { | } ~ del
Illegal:  soh " $ & ' ( ) ; < > [ \ ` | del
Let me take you to, funkytown!

Note that bash happily lets me name my function "[}{". Probably my code is not quite rigorous enough to provide the exact rules for legality-in-practice, but it should give a flavor of what manner of abuse is possible. I wish I could mark this answer "For mature audiences only."

From the manual:

   Shell Function Definitions
       ...
       name () compound-command [redirection]
       function name [()] compound-command [redirection]

name is defined elsewhere:

       name   A  word  consisting  only  of alphanumeric characters and under‐
              scores, and beginning with an alphabetic character or an  under‐
              score.  Also referred to as an identifier.

So hyphens are not valid. And yet, on my system, they do work...

$ bash --version
GNU bash, version 4.2.25(1)-release (x86_64-pc-linux-gnu)