Is there a way to escape single-quotes in the shell?

Dennis points out the usual alternatives in his answer (single-in-double, double-in-single, single quoted strings concatenated with escaped single quotes; you already mentioned Perl's customizable quote operators), but maybe this is the direct answer you were looking for.

In Bourne-like shells (sh, ksh, ash, dash, bash, zsh, etc.) single quoted strings are completely literal. This makes it easy to include characters that the shell would otherwise treat specially without having to escape them, but it does mean that the single quote character itself can not be included in single quoted strings.

I refuse to even think about the csh-type shells since they are utterly broken in other regards (see Csh Programming Considered Harmful).

For me, all your examples produce:

Hello, world!

And so does this:

perl -e 'print "Hello, world!", "\n";'

And this:

perl -e 'print '\''Hello, world!'\'', "\n";'

Can you clarify what it is you're trying to do?

In the shell, such as Bash, if you want to print quotation marks as part of the string, one type of quotes escapes the other.

$ echo '"Hello, world!"'
"Hello, world!"
$ echo "'Hello, world!'"
'Hello, world!'

Edit:

Here's another way:

$ perl -e 'print "\047Hello, world!\047", "\n";'
'Hello, world!'
$ echo -e "\047Hello, world! \047"
'Hello, world! '

The space after the exclamation point avoids a history expansion error. I could have done set +H instead.