Will the admission committee try to get in touch with the referees who wrote the recommendation letters? [Graduate admission]

I have written many letters of reference.

I have also been contacted, over the phone, by HR or admissions who wish to discuss the candidates, just to expand on experience or whatever.

So, if they wish to contact the people who you put as referees, then they will do so.

This isn't something you need to worry about. It is unlikely that they will do so, but possible. If they do, the language issue will not reflect badly on you. The people are peers and will understand that language can be an issue and will compensate for it. If your letter writers are positive about you in writing, it will be clear to the caller that they are positive about you when speaking.

You have not stated where you are applying to. I can imagine that practices vary by geography.

However, in 25+ years of industry, academic, and related work, primarily in North America, I have found that it is exceedingly rare for hiring or admissions committees to reach out to providers of written references (this is in contrast to a habit in industry of asking applicants for the contact information of references to speak with by phone instead of asking for written references).

I recall only 3 cases where I have been a bystander or participant to such a reachout. In two cases it was to confirm the reference provided was valid in the first place, and in one case to factually confirm a presumed typo that made one sentence in the reference letter have an anomalously negative tone.

Normal admissions or even jobs committees do not have the time to seek a supplementary "deep and meaningful" discussion about applicants. In addition, there would likely be concerns about procedural fairness in who and how would lead such a discussion, who would participate, and the possibility of bias if one person was reporting the outcome of a discussion to the committee. (As someone who has also hired in the private sector, I think those concerns are overblown, but so be it.)

Finally, it's not applicable in your situation I presume, but I would add that of course for senior, prestigious posts (deans, named professorships, etc.) there is sometimes back-channel discussion with people familiar with a candidate who may or may not be providers of formal reference letters, but that is in the context of a committee forming its own dossier about 1-2 leading candidates, a different story.