Use a trigger to stop an insert or update

Try the SIGNAL syntax - https://dev.mysql.com/doc/refman/5.5/en/signal.html

create trigger agency_check
before insert on foo
for each row
begin
  if (new.agency < 1 or new.agency >5) then
    SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT = 'your error message';
  end if 
end

EDIT

Updated based on popular comment below by Bill Karwin.

If your version of MySQL is older than 5.5, try setting a non-null field of the table to NULL. It is a hack, but it does prevent the update or insert from completing.

The SIGNAL command which Naveen suggests looks great, and I'm looking forward to using it after we upgrade.