Matematical way to compare a pair of 3 variables

TL;DR

Compare the sum of each triplet, the product of each triplet, and the sum of the products of all possible combinations of each triplet.

The Nitty Gritty

By the Fundamental Theorem of Algebra, for a polynomial of degree N, we must have N roots.

Using this fact we let our zeros be a1, a2, and a3. Now, we find the coefficients of this polynomial.

(x - a1) * (x - a2) * (x - a3)
(x^2 - (a1 + a2) * x + a1a2) * (x - a3) 
x^3 - (a1 + a2) * x^2 + (a1a2) * x - a3 * x^2 + (a1a3 + a2a3) * x - a1a2a3

x^3 + (-1 * (a1 + a2 + a3)) * x^2 + (a1a2 + a1a3 + a2a3) * x + (-1 * a1a2a3)

If two polynomials are equivalent, they must have the same roots (again by the FTA). Thus all we need to do is compare the coefficients of the generated polynomials.

So, if,

(-1 * (a1 + a2 + a3) == (-1 * (b1 + b2 + b3))
      ---equivalently---
a1 + a2 + a3 == b1 + b2 + b3

And

(a1a2 + a1a3 + a2a3) == (b1b2 + b1b3 + b2b3)

And

-1 * a1a2a3 == -1 * b1b2b3
      ---equivalently---
a1a2a3 == b1b2b3

Then we can conclude the triplets a1, a2, a3 and b1, b2, b3 are equivalent.

Is it worth it?

From a practical standpoint, let's see if this is indeed more efficient than brute force checking as illustrated by the OP.

First check: Sum and Compare. This requires 4 total additions and 1 check for equality.

Check total = 5; Running total = 5

Second check: Product, Sum, and Compare. This requires 6 total multiplications, 4 total additions, and 1 check for equality.

Check total = 11; Running total = 16

Third check: Product and Compare. This requires 4 total multiplications and 1 check for equality.

Check total = 5; Running total = 21

Adding the two logical AND operations, the total number of binary operations for the "coefficients of the generated polynomial approach" only requires:

23 binary operations

The brute force check requires 18 total equality checks, 12 logical AND comparisons, and 5 logical OR comparisons for a total of:

35 binary operations

So, strictly speaking, the answer is yes, the "coefficients of the generated polynomial approach" is indeed more efficient. However, as @WJS points out, the brute force approach has many more opportunities for short circuiting and thus execute as/more efficiently than the mathematical approach.

For Complete Thoroughness

We cannot skip checking the sum of the products of all possible combinations of each triplet. If we leave this out, there are countless examples where this fails. Consider (23, 32, 45) and (24, 30, 46)^*:

23 + 32 + 45 = 100
24 + 30 + 46 = 100

23 * 32 * 45 = 33120
24 * 30 * 46 = 33120

They are not equivalent but give the same sum and product. They don't however give the same sum of the products of all possible combinations:

23 * 32 + 23 * 45 + 32 * 45 = 3211
24 * 30 + 24 * 46 + 30 * 46 = 3204

^*In case one is curious how to derive an example similar to the one above, first generate all integer partitions of an integer M of length 3, take their product, find the duplicates, and pick a pair.