How does covalent bonding actually work?
In physical chemistry, this problem is usually treated in MO-LCAO theory.
What you do is to assume that you can create the molecular orbitals of the molecule as a linear combination the atomic orbitals of the atoms in the molecule (MO-LCAO stands for Molecular Orbitals - Linear Combination of Atomic Orbitals). Therefore, your atomic orbitals are a mathematical basis set on which you project (using some coefficients) your molecular orbitals. The problem is further simplified if you consider that the atomic orbitals that will combine together should have the same character for the symmetry operations possible for that molecule (it means that every atomic orbital combining should belong to the same point group, in order for their linear combinations to belong to that group). You can therefore create the SALC (Symmetry Adapted Linear Combinations), linear combinations of atomic orbitals of the same point group, and use them as a more powerful mathematical basis set for the molecular orbitals.
Stated this, you can calculate the coefficients of the linear combination and the energy of each molecular orbital. What you get is a certain number of levels (same number of the atomic orbitals considered in your basis set) ordered by their energy. You can now distinguish between three types of molecular orbitals:
bonding, the atomic orbitals constructively interfere in the region between the two atoms;
antibonding, the atomic orbitals destructively interfere in the region between the two atoms;
non bonding, the molecular orbital is almost identical to one atomic orbital (the coefficient of a certain atomic orbital is way greater than the others).
You can distinguish (at a very basic level) between them by representing the atomic orbitals involved and their sign in the region between the atoms: if they have the same sign, they are bonding, else they are antibonding. (Please note that by doing this I forget about the magnitude of the coefficient, that should be relevant in most cases.)
Now you have a sort of "ladder" of molecular orbitals and you know if each step is bonding or not. You can now put the electrons (same number as the sum of the electrons that where in the atomic orbitals you used in your basis set) as you did for isolated atoms: from bottom to top, two electrons in each level, antiparallel spin, and so on (the same rules also if you have more levels at the same energy).
You can now go back to a classical chemistry framework using the so called bond order: $$ BO =1/2( n-n^*)$$ where $n$ is the number of electrons in bonding orbitals and $n^*$ is the number of electrons in antibonding orbitals (non bonding orbitals just doesn't count). The bond order tells (if it is an integer) how many bonds we represent in a classical picture, thus going back to the concept of octet rule.
In fact, consider the valence shell of oxygen. It is made by the atomic orbitals $2s$, $2p_x$, $2p_y$, $2p_z$ and it contains six electrons. By combining these (and ignoring the interaction between $2s$ and $2p_z$, that could be possible and that only modifies the energy of these molecular orbitals) you get $4\times 2$ molecular orbitals (the apex * means that they are antibonding).
The electrons for oxygen are black (red ones are added when considering the F$_2$ molecule).
The bonding molecular orbitals from a shell of this type are four, therefore the total of the bonding electrons are eight. Here comes the octet rule, but this kind of reasoning is trying to fit an empirical and wrong way of reasoning into a more powerful and quantum framework.
Please note that my answer is from a really introductory and basic point of view; things, starting from this, can become a lot more complicated.
The octet rule is old and is not accurate (has nothing to do with quantum mechanics and is backed by 'empirical' evidence only)
The octet rule was proposed much before the foundations of quantum mechanics were established.
Here is an excerpt from Wikipedia:
The octet rule is a chemical rule of thumb that reflects observation that atoms of main-group elements tend to combine in such a way that each atom has eight electrons in its valence shell, giving it the same electronic configuration as a noble gas. The rule is especially applicable to carbon, nitrogen, oxygen, and the halogens, but also to metals such as sodium or magnesium.
Source: https://en.wikipedia.org/wiki/Octet_rule
Important points to note here are:
- "a chemical rule of thumb that reflects observation": established based on observations only
- The rule is especially applicable to carbon, nitrogen, oxygen, and the halogens, but also to metals such as sodium or magnesium: works for most of the compounds formed by the elements of the first few periods of the periodic table only.
Not only there are several exceptions to the rule when atoms above the atomic number 20 are considered, there are exceptions to the rule when some of the elements from the lower periods are considered as well (not a surprise):
- there are stable atoms which have incompletely filled valence shell but are still stable ($BCl_3$, a phenomenon called back bonding plays a role here which ensures momentary octet for the Boron atom)
- there are stable atoms with odd number of electrons (nitric oxide, $NO$; nitrogen dioxide, $NO_2$; chlorine dioxide, $ClO_2$)
- there are stable atoms with more than 8 valence electrons ($SF_6$ has 12 electrons surrounding the central atom, i.e: sulphur)
To put everything in nutshell, the octet rule is not correct.
How does octet rule work?
In chemistry class, we get around this by "double counting" covalently bonded electrons -- somehow, they can count as valence electrons on two atoms at once. But how can a single electron be in two quantum states at once?
The octet rule states that the atoms tend to form molecules such that they have 8 electrons in their valence shell. It does not matter if the electron is a lone pair (or a radical electron) or if it is a bonded electron; whatever type the electron maybe, it is still a part of the atom.
You don't double count, you count all the shared electrons because they are part of the atom. As the name says, the electrons are being shared; therefore, shared electrons are included while counting.
Why do we still use the octet rule today?
We still use the octet rule today as it is easier to understand and describes the behavior of most of the common compounds (the compounds formed by the first few elements). You wouldn't want Molecular Orbital Theory in a $10^{th}$ grade textbook, would you?
Molecular Orbital Theory
This is the latest theory which explains bond formations. JackI has given a concise and neat explanation of the Molecular Orbital Theory.