How do magnets affect the ATMega328?

Magnets cause problems with magnetic storage (hard disks, tapes, etc). The AVR chips use flash, so won't be a problem.

Recently, I did some experiments with AVRs and hall effect sensors. I never saw problems from being near magnets.

Joby hit it on the head as far as storage goes.

I would caution people experimenting with magnets and electronics (like reed switches) to be very wary of the filings that collect on powerful magnets. You'll get nearly invisible shorts when an iron filing that collected on the magnet falls onto your chip, embeds itself in some leftover flux, and takes down your system. Industry is worried about tin whiskers 10um in diameter and less than 1mm long - iron filings are just as dangerous.

Magnets will not usually affect the chip. Magnets cause damaged to hard disks and floppy disks (remember those?) because they misalign the domains and destroy the data.

However, if you happen to have long traces on your PCB and strong (especially pulsating) magnetic fields, you could get induced voltage in these traces. This could range from a tiny bit of additional noise causing no harm to damaging the microcontroller from a voltage spike. This is only theoretical. You'd probably need MRI-level magnetic fields to induce any significant voltage.

As you are using a fridge magnet I do not see any of the above being a problem.