ShellCheck is carping that my expression is not in double quotes when it really is; why?

It is a false positive, but it's not the one you think it is. It has nothing to do with the *, and didn't point there for me. It's upset about `Name` being inside of single quotes. For example, echo '`Name`' produces the same warning, because it thinks that you want the backticks to be evaluated, so it's warning you that they won't be.

Not an answer, but a formatted comment:

Pedantically, you shouldn't be using a for loop, but a while read loop:

while IFS= read -r server; do
    : do stuff here
done < <(
    "$aws" ec2 describe-instances \
        --query 'Reservations[].Instances[][].{Name: Tags[?Key==`Name`].Value[] | [0]}' \
        --filters "Name=tag:Name,Values=${server_name}*" \
        --output text
)

for loops read whitespace-separated words, while loops read lines -- see http://mywiki.wooledge.org/BashFAQ/001

Alternately, use readarray to capture the output

readaray -t servers < <(
    "$aws" ec2 describe-instances \
        --query 'Reservations[].Instances[][].{Name: Tags[?Key==`Name`].Value[] | [0]}' \
        --filters "Name=tag:Name,Values=${server_name}*" \
        --output text
)

for server in "${servers}"; do ...; done

Lastly, for long and unreadable commands, storing the options in an array can improve readability:

opts=(
    --query 'Reservations[].Instances[][].{Name: Tags[?Key==`Name`].Value[] | [0]}' 
    --filters "Name=tag:Name,Values=${server_name}*"
    --output text
)

readarray -t servers < <("$aws" ec2 describe-instances "${opts[@]}")