Student caught cheating when leaving class after handing me the exam

Exact rules will depend on where you are. But normally the student has due process rights: to be confronted with the evidence of her cheating, and to appeal to some higher administrator or body if she desires. Two of your options (grade more harshly / secretly lower grade) are things that you would do unilaterally, without the student's knowledge, and without giving her the right to challenge. In my view, these options are therefore totally inappropriate and unethical.

The evidence against her is your testimony about what you observed. If you think this would be sufficient evidence under your university's rules, then it is appropriate to pursue formal punishment as the rules dictate. If you don't think it's sufficient, then do nothing, and let the student complete the rest of the course without prejudice. (Though of course you can watch her more closely in the future, and perhaps institute procedures to deter this kind of behavior in general.)

For this particular incidence, I am afraid that the window of opportunity for acting on it has closed. Be prepared for the next time. Have a camera ready. Have stuff to say ready. Make sure to watch this particular student on the next exam: she's not going to fly under your radar next time round, and over the length of her coursework, that brag might cost more than the one cheat bought her.

If she no longer has courses with you, she likely will with others. You can tell them of your goof informally so that they keep an eye open.

The easiest way out longer term is to create exams where cheating (short of communication) does not help. I remember exams where people were allowed to bring one hand-written sheet of A4 paper (in the U.S., you could declare one sheet of "legal" legal): condensing the course contents like that was so much of a learning experience that you usually could then forego the sheet anyway. There were others where you were allowed to bring anything except electronic devices: the time on those exams just was far too short for applying significant amounts of knowledge not suitably internalized.

Remember: in their job, they will be allowed to look up things, too. So try teaching and checking for skills that go beyond dictionary lookups.

It's not proven that she cheated.

I have known several students who would write their answers on their hand during the exam, so they can later compare their results with others. I don't know in what country you are in, but if you should go after the alleged cheating, the student might carry this situation to court.

It's then up to you to prove that she had the notes before she entered the exam, which I doubt is possible for you. Therefore, my suggestion is to let it go and be more careful next time, should you see the student again.