How do professors learn how to write letters of recommendation?

As with most professorial duties, it's a sink-or-swim process. There is no formal or official training they receive for this task. They're just thrown into the situation with no experience and are expected to perform well at it immediately. If they've seen other recommendations during their other duties (that they likely also weren't trained for) they can use that, otherwise they can try asking colleagues or do google searches. So for the most part it's a bunch of untrained people relying on other untrained people and crossing their fingers that the other guys were able to figure out a good approach.

Ideally, you'll discuss the first few letters you write with a mentor, who can offer feedback and keep you from writing an ineffective letter (getting advice certainly helped me tremendously). The same situation can come up again later in your career, for example the first time you write a letter for a liberal arts college if you are used to recommending people for jobs at research universities. If you aren't confident that you can write an effective letter in certain circumstances, then you should seek advice from someone with more experience, even if you are used to writing other sorts of letters.

Serving on hiring and admissions committees is extremely valuable, both in figuring out what people reading letters want to know, and in calibrating how strong the competition is and how much enthusiasm one should express in any given case.

Trial and error does not seem useful, at least in my experience. There's just too much randomness in the decision-making progress and too many unknown factors that differ between students.

Writing good letters also gets easier with age, as you develop more experience and have a broader basis for comparison. As an incoming faculty member, you haven't seen how careers progress over time, and you've never had the experience of finding out which of the smart young people you know will live up to or exceed their early promise and which will not. Of course you'll never be able to predict this with certainty, but over time you'll start to see patterns, and you'll be able to make more nuanced and illuminating comparisons in letters.

I've seen a few good ones that were written about me. I use similar thoughts and structures without copying and by supplying my own knowledge about the subject of the letter. In time as a professor, you will see letters written about others during your service on hiring, student selection, and tenure committees. You can borrow (without plagiarising) ideas and structures from the ones that make strong impressions on you.