Is it rude to call a professor by their last name with no prefix in a non-academic setting?
I wouldn’t find it rude, but I would find it weird. I can’t think of any situation where you wouldn’t be better off using a first name instead of a last name without a title. If you’re going to be formal do it right, and if you’re going to be informal use first name. We’re not on a football team or in the military.
(It’s of course totally fine and normal students talking to each other without the professor present, to just use last name and no title.)
Would it be rude if I called you by your surname only?
It depends.
How well do I know you? What is our relationship? Is there a significant age gap between us? Are we in a fraternity or the military?
As a student, I never once called a professor by only their surname when speaking to their face, formally or otherwise. They were always older than me (even if just by a few years) and we never had such an informal relationship that surname alone seemed appropriate.
I did call some professors by just their given name. This was usually when they specifically requested it or when I knew them quite well. As I rose higher through the ranks in academia
(freshman -> sophomore -> ..... -> graduate student -> PhD candidate....)
calling professors by their given name became more common.
I'm sure there could be some professors that would be fine with being called by their surname alone. Most would not be, however. This is not unique to academia. Most people in their 40s do not like being called by just their surname by 18 year old kids. Even when I was a 25 year old graduate student, I'll admit that it would sort of bother me if students called me by only my surname (even outside of class). We weren't in the army or something. Just call me by my first name.
Obviously saying something like "I had combinatorics from Levenworth and topology from Kostanza" when speaking to fellow students is a different story. There's no need to worry about offending someone when referring to them by surname alone when they are not there.
In the U.S., in this year, at my age, I would be surprised if anyone addressed me by my surname. But that is a very different thing from reference to my work, where I'd mostly expect people to say "Garrett, [year]", or something similar. Still, in an in-house situation, to hear a speaker say "Paul's work..." would not be jarring, and might be more congenial than "Garrett's work...".
... although the times that a speaker has said "Garrett's work" and waved to me in the audience were perfectly fine.
I guess some nod of respect is the only substantive point, and this is dependant on the local culture... for which there is no clean algorithm.