Why are subnet-masks relevant for the individual computer on the network?
Your original assumptions are not entirely correct. What you call a "router" is two devices in one – a two-port router internally connected to a multiple-port Ethernet switch. (Here's an example diagram.)
This means that the computers are directly connected at layer 2, and can send packets to each other without going through the router core – they're simply relayed between ports by the switch chip. (The router has its own "port" in the switch.)
So if you look at the packets using Wireshark, you'll see that they directly use each other's MAC addresses, while "outside" packets always have the router's MAC as the destination.
(I'm assuming you're talking about the typical "wireless routers" found in most homes, which are the usual cause of this kind of question. A bigger network would have a separate router with one port per subnet, and a few separate switches (perhaps a master one plus one per floor/room), and several dozens of computers connected to those switches.)
It's roughly the same with Wi-Fi networks, except "switch" is replaced with "wireless bridge" aka "access point". In both cases, connected computers can send packets directly to each other at layer 2, without going through the router.
Comments:
When I stated router, I did actually mean switch. My mistake. My point beeing, that each computer in a subnetwork is not connected to each other, but rather to a switch, which then can pass on packages to the correct destination. An ethernet-frame does not contain the subnet-mask, as the switch already has this knowledge, and hence does not need it to do the correct switching.
That's again incorrect. Switches do not have this knowledge; their switching core works at layer 2 and does not know anything about IP – it forwards Ethernet frames purely based on the 'destination MAC address' field.
Therefore, hosts need the subnet mask to figure out what MAC address to use as the destination:
If the peer is within the same subnet, it's assumed to be on-link by definition – so the Ethernet frame will have peer's MAC as destination.
For peers outside the subnet, the Ethernet frame will have the gateway's MAC as destination.
(This applies to the default configuration. Some special-snowflake networks alter this – e.g. most operating systems allow adding extra "on-link" routes for additional subnets; conversely, some switches may be configured spoof ARP responses such that even "on-link" traffic is forced through the gateway.)
How does a computer know if a destination address is in the same subnet on in another?
Checking the local adddress and the subnet mask.
Let's check a couple examples:
If my computer has the IP 192.168.0.1
and the mask is 255.0.0.0
it means that any address from 192.0.0.0
to 192.255.255.255
is in the same subnet. The packets to all those other computers don't need to go through the router, they can be send directly. Send an ARP packet to get the MAC adddress of the destination computer and then send the packet.
But, if my computer has the IP 192.168.0.1
and the mask is 255.255.255.128
then the computers in the same subnet are from IP adddress 192.168.0.0
to 192.168.0.127
only. They can be reached directly (send ARP, find MAC address,etc.). Any other address, for example 192.168.0.200
must be reached passing through the router.
Something non-obvious about IP is that every IP device is itself a router.
This can be seen on a normal PC with the command "route print". You are connected to two networks: your local Ethernet or wifi segment, and the localhost network. Every packet needs to be subject to a decision as to which network to put it on.
This becomes more apparent if you put your computer on two networks, say a "public" and "private" one. Now you definitely need the subnet mask in order to decide which network to send the packet on.
Many people will accidentally discover that a PC with a single network connection may work with a wrongly configured submask: they just end up sending everything to the gateway.