Is it appropriate to attend a class one is not registered for?
In the American system, this is called "auditing" a course. Auditing means that you are attending the class but are technically only listening (auditing => auditory => listening) to the material and not sitting for exams or handing in homework assignments. You can audit a course formally (which often means that this is listed on your transcript) or informally in which you just sit in on the lectures/discussions. For the former/formal, see your local university guidelines. This answer deals with the latter.
Permission of the instructor in all but the larger lectures is always a good idea unless your university has an open classroom policy (as with many European universities). The instructor may have different rules. For example:
- In larger lectures, I do not mind auditors (of any ilk) in lecture but they may not attend discussion sections since those are for the paying students.
- I allow graduate students to audit my graduate seminars but with three rules: a) They must attend all of the sessions; b) they must do all of the readings; c) they must not be a 'dead body' but must participate in the seminar discussion.
Note that the university itself may have rules on auditing which the faculty member may ignore (for example, I do not mind if local community residents audit my lectures even if they are not registered as auditors through the university).
This is different from PASS/FAIL (also known as CREDIT/D/FAIL) which is another alternative in which you take the class but are allowed a lower level of participation.
Should I ask the professor's permission first?
Yes, this is the polite thing to do. (It's a perfectly reasonable and common request, so no need to be embarrassed about asking.)
As an instructor, it's useful to be able to understand any apparent discrepancy between the number of "official" students and the number of students I see in class. (For example, for determining whether I need to reserve a bigger classroom for an exam, or if I need extra TA support.) So even though I've never turned down a request like this (given that it's permitted in my university), I still like to be asked.
Some schools have an official policy about this kind of thing. If you couldn't find anything relevant, maybe the professor will know and will clue you in when you ask him.
Other, more qualified, people have answered your direct question, so I will add to it without detracting from what they have to say.
There are some schools with 'alumni audit' policies, in which - as the name suggests - alumni can formally audit courses. Check with the registrar on policies about this. If your school has such an option, find out the rules. Some classes may be auditable and others may not. Some may be auditable online if you are far away from your university...etc.
While you will likely only be able to do this for one course at a time for courses offered at night, assuming you work a day job, it could be a good way to take those elective courses, create & maintain relationships with professors and with your university.
Good luck with your studies, and have fun.