How is an experimental particle physicist evaluated?

I happened to be a member of my college's executive committee when an experimental particle physicist was being considered for promotion to tenure. She was, indeed, a member of a huge collaboration, so a typical paper of hers had perhaps 100 co-authors. The papers provided no information about what, specifically, she had contributed (and we know better than to pay much attention to citation counts and impact factors). The information we needed, about the quality and quantity of her work, was provided by numerous letters from external reviewers. ("External" means not in our university.) Ordinarily, the college requires that most external letters come from senior researchers who are not co-authors of the candidate's papers, but an exception is allowed in areas, like experimental particle physics, where only co-authors really know about the candidate's work. My recollection (about 15 years old and therefore somewhat vague) is that the letters gave us very detailed information, not only about what the candidate had contributed but also about how efficiently she had done her part of the work.

HEP PhD here.

What matters for a permanent position?

• By far the most important factor is the "physics." You need to have innovative ideas about how to discover particles. On the small experiments, this can mean new approaches for detectors, analysis, etc.; on the larger experiments, this usually means new analysis techniques.
• Another important factor is "leadership" -- you need to attend lots of meetings and make substantial contributions at all of them.
• Though it's understandable that universities want to hire the highest-impact people possible, this does lead to a slightly toxic environment. There are endless meetings where everyone argues over factors of epsilon, and everyone wants to be involved in dozens of things but not put time into any of them. Skills, teaching, and knowledge of other fields/subfields don't count at all.

How is this measured?

• The most important way is the letters of recommendation. You will need 3 for post-docs and up to a dozen for a faculty position.
• "Official" positions such as analysis lead, sub-convenor, etc. help a lot.
• Awards and grants also help, though there are not many of these for grad students.
• As you say, publications don't count at all, as there's no way to tell who worked on what (which is really stupid, IMO).

Finally, let me point out that here are ~10 US HEP faculty jobs per year and hundreds of HEP PhDs produced each year (and the lack of new particles showing up at CERN means that hiring is likely to decelerate). So, there is some element of randomness.

H-index is a simple metric - and as it goes for simple metrics, they are mostly used by people who are not in the field, or funding agencies. As you say, h-index just does not work in particle physics, where h-index naturally grows over time, just because you are a member of a collaboration. To give you a feeling, the average PhD graduates with an h-index around 40. I met a young researcher that changed experiment twice in his PhD career and because of bizantine rules continued to sign papers from all three big collaborations. His h-index was around 70 before his graduation. One particle physics collaboration puts the main authors at the top of author list. Some had the main authors only listed on a publicly accessible webpage that explains the measurement. Most of the time, a great deal of effort is spent masking who the lead authors actually are. While experimental particle physics represents the extreme scenario, other fields share similar issues.

How are particle researchers assessed in reality, So, h-index is mostly meant for outsiders. People in your field - any field - judges by reputation. Reputation is built by being consistently successfull in leading original work, being a good disseminator, a respected leader, etc. How to measure things without looking at papers, you might ask?

1. The institution that awarded your PhD. Having a PhD at MIT or similar institution is the earliest sign of outstanding skills, and it is probably the one that sticks with you the most.
2. Your PhD supervisor. If you graduated with someone who is already famous in the field, and he/she highly admires you, that is the best walking letter of reccomendation you will ever have.
3. Talks/seminars. Great research work leads not only to highly cited papers, but to invitation to present your results at seminars at key institutions, or talks at key conferences. A hiring committee member will be able to map the (selected) papers you display on your CV to the talks you gave. While talks/seminars are important per se, it is probably the networking the follows that matters the most.
4. Breadth of research. In particle physics, that means working both in "hardware" - building/operating the detectors - and in "data analysis" - producing the measurements themselves. If you collaborate on theoretical papers, that's a plus. By the way, "hardware" and theoretical papers have "normal" authors list, i.e. list only the main authors.
5. Prizes/awards. Admirers will nominate you for prizes. You will feel you have a shot at winning awards, and will apply to all of them. Prizes and awards lead to more prizes and awards.
6. Reputation prompts invitations to lead scientific efforts, that produce higher returns than working solo. Leadership in turn prompts additional reputation boost.
7. Luck. As pointed out, the number of High Energy Physics opening is so limited, and the number of PhD awarded so high, that you are most likely to get invited to interviews if somebody in the hiring committee already had a chance to encounter your work directly, and/or was impressed by a public appearance of yours.

How does university ranking play into this? With this question you actually meant how to rank established professors. Established professors reputation works as for younger researcher - basically the points 1) to 6) above. You can replace Luck with Group size. Group size is basically a proxy of how a professor reputation gains her grants to hire postdocs, and aura to attract students.

As you can see, most of the above is actually pretty general, and holds for most field. What characterizes experimental high energy physics is the complete lack of interest in h-index (which would anyway just be a poor-man proxy for the points highlighted) and the extremely competitive scenario, that puts additional emphasis on visibility, networking, and a bit of luck.