Apple - enabling virtual machine Guest Additions on a Mac host and Mac guest
I'll quote a previous answer of mine:
From Virtualbox Manual : Chapter 14:
14.2. Known Issues
Mac OS X guests:
VirtualBox does not provide Guest Additions for Mac OS X at this time[snip]
From the content of this, I wouldn't hold out for such support in VirtualBox any time soon.
The situation really has not changed since then.
There is some discussion of a workaround involving the built in OS X sharing functionality and configuring your network setup to work, but nothing well defined.
Guest Additions are available for macOS starting with VirtualBox 6.0
Per the change log, macOS/OSX Guest Additions are at "initial support" meaning features/functionality could be limited.
Resolving this question involves having guest additions on a MacOS guest. If, instead, you want to install guest additions from a MacOS host onto a non-MacOS guest:
You can copy the guest additions iso from your machine.
First locate VirtualBox in the Application folder, right-click it and select show package contents. Then browse to Contents/MacOS/VBoxGuestAdditions.iso. Copy it to a location that accessible when browsing for files.
After that, you can insert the device by clicking Devices > Optical Drives > Choose disk image... and then choose your copied iso file.
Then you can run your iso from within your vm and install the guest additions.