Bell polytopes with nontrivial symmetries

Any symmetry of the local hidden variable polytope must map a vertex of the polytope to another other vertex (or trivially to itself). This is true in general by convexity. By the duality between vertex representation and facet representation we only need consider vertices. I have modified the way you write vertices to obtain $p(r_{1} r_{2} ... r_{N}|s_{1} s_{2} ... s_{N})=\delta^{r_{1}}_{f_{1}(s_{1})}\delta^{r_{2}}_{f_{1}(s_{2})}...\delta^{r_{N}}_{f_{N}(s_{N})}$ where $f_{j}(s_{j})$ is the image of $s_{j}$ under a single-site function $f_{j}:\mathbb{Z}_{m_{j}}\rightarrow\mathbb{Z}_{v_{j}}$.

Therefore, a symmetry will map from the product of single-site maps $\delta^{r_{1}}_{f_{1}(s_{1})}\delta^{r_{2}}_{f_{1}(s_{2})}...\delta^{r_{N}}_{f_{N}(s_{N})}$ to other products of single-site maps $\delta^{r_{1}}_{f'_{1}(s_{1})}\delta^{r_{2}}_{f'_{1}(s_{2})}...\delta^{r_{N}}_{f'_{N}(s_{N})}$ with $f_{j}$ not necessarily equal to $f'_{j}$. Of course, one can reorder the products by permuting the parties and still produce a product of delta functions. Locality prevents us from allowing delta functions of the form $\delta^{r_{j}}_{f_{j'}(s_{j'})}$ with $j\neq j'$. Therefore, other than permutations the only symmetry transformations that are allowed will be transformations on the maps $f_{j}(s_{j})\rightarrow f'_{j}(s_{j})$.

We only need to consider each site's marginal probability distribution $p(r_{j}|s_{j})$ which can be written as a $m_{j}v_{j}$ length real vector. The vertices have $m_{j}$ non-zero elements which have unity value for each value of $s_{j}$. In order to conserve these two conditions of the vertex probability distributions, the only allowed transformations on the real vectors that are allowed are a restricted class of permutations of row elements. The restricted class of permutations of row elements is naturally generated by relabelling a measurement outcome for each value of $s_{j}$ and relabelling values of $s_{j}$.

This applies for the full probability distribution polytope. However, for other forms of correlations such as joint outcome statistics, e.g. $p(\sum_{j}^{n}r_{j}|s_{1} s_{2} ... s_{N})$ there are other subtle forms of symmetry outside of the 'trivial' classes. If you want me to elaborate, I can.

This is my first post to the TP.SE. I'm sorry if it is not detailed enough.

Matty Hoban pointed me to a paper (PDF here) by Itamar Pitowsky from 1991 which looks the geometry of correlation polytopes and their symmetries. I haven't read the paper in full, but glancing through it, on page 400 (page 6 of the actual paper) under the statement of results the author seems to say that the cardinality of the symmetry group is $n! 2^n$ which would be consistent with just the bit flips and permutations, and with the existence of only the trivial symmetries you mention.