Buffering with physical barrier using ArcGIS Desktop?

If I understand the question correctly, I would perform the buffer as normal, then intersect your buffer result with your physical barrier features (rivers?). Then do a select by location to select the features in the intersect results that do not touch your points. Delete these, and you are done. It is essentially what you are already doing manually, but in an automated fashion.