Apple - Can I add a "space" to my Dock to separate apps into groups?

Yes you can! Follow these steps:

1. Within Finder launch the Terminal app (by default it’s in the Utilities folder within your Applications folder)

2. Copy and paste the following command into Terminal:

   defaults write com.apple.dock persistent-apps -array-add '{tile-data={}; tile-type="spacer-tile";}'
   

3. Press enter

4. Rstart the Dock by entering the killall Dock command into Terminal and pressing enter

Now you’ll see a space in your Dock. You can click and drag this around as you would any icon!

Keep in mind that any time you launch an app (be it within macOS or your Windows VM), you’ll get a new icon at the right end of your Dock (unless it’s been permanently added to your Dock previously). So if you’re wanting to group your Windows app icons together you’re going to need to create a second space so that one is to the left and the other to the right of your Windows apps. In this way it’ll be a distinct group, separated also from icons at the rightmost end of your Dock.

Of course, even with this approach, the next time you open a new Windows app you’ll have to drag it within that group and also choose to keep it in your Dock (assuming that’s what you’re wanting to do).

---

Additional configuration options

Since originally posting this answer I've discovered that the spacer tile can be configured in terms of size and location. So, at Step 2 of my instructions above, you actually have the following options:

Large space

The original command at Step 2 of my answer (see below) creates a large space in your Dock.

    defaults write com.apple.dock persistent-apps -array-add '{tile-data={}; tile-type="spacer-tile";}'

Small space

Instead of the original command, you can add small within the command to create a smaller space in your Dock (see below):

    defaults write com.apple.dock persistent-apps -array-add '{tile-data={}; tile-type="small-spacer-tile";}'

Documents space (as opposed to an Applications space)

Both of the commands above create a space in your Dock where your Application icons appear. However, you can change the command so that the space appears where the Document icons are. To do this, use the key persistent-others instead of persistent-apps in the command you're going to use.

For example, to create a small space on your Dock within your Document icons you would use the following command:

    defaults write com.apple.dock persistent-others -array-add '{tile-data={}; tile-type="small-spacer-tile";}'

NOTE: Regardless of the command you use, it still needs to be followed with the killall Dock command to take effect.

@Monnomeeth's answers adds spacers while this one will add dividers:

From this GitHub Repository containing an app called VerticleBar:

Add as many Vertical bars to the macOS dock as you want.

1. Download

2. Unzip

3. Open > Right click the icon in your dock > Options > "Keep in Dock"

4. Rightclick > Quit

It's basically an application with the divider as its icon that you can use to add fake dividers.