Is it possible to serve multiple Jekyll sites locally?

Yes it is possible to serve multiple Jekyll sites locally by setting different port numbers for each site using the Local Server Port serve command option.

See https://jekyllrb.com/docs/configuration/options/#serve-command-options.

Set a port number other than Jekyll's default port (4000) in the site's _config.yml file or when serving the site via the command line.

For multiple Jekyll sites, I just run this

bundle exec jekyll serve --port <your_port_number>

e.g

bundle exec jekyll serve --port 4001

You can also start the server with an additional argument --port 1234 or --host hostname. For example:

$ jekyll serve --port 4001 --host my_hostname_or_ip

You can view all the possible CLI flags from The official Jekyll documentation