Is there a way to make CSS calc() never be a negative value?

In the latest browser versions there is max() function in CSS:

.my-margin-top {
    margin-top: max(50vh - 325px, 0);
}

Browser compatibility: Firefox 75+, Chrome/Edge: 79+, Safari: 11.1+

There is also clamp(): clamp(0, 50vh - 325px, 100vh), where 100vh is an upper bound. But this is not in Safari yet, although can be simulated with min() and max() calls.

Premising that there's no way to bound the values computed by calc() you could use a mediaquery when max-height is at most 650px

.my-margin-top {
    margin-top: calc(50vh - 325px);
}

@media all and (max-height: 650px) {
   .my-margin-top {
       margin-top: 0;
   }
}

or you might also revert the logic by wrapping the existing rule into a min-height mediaquery

@media all and (min-height: 650px) {
   .my-margin-top {
       margin-top: calc(50vh - 325px);
   }
}

To answer the general question:

Is there a way to make CSS calc() never be a negative value?

The answer is no, because CSS does not provide a way to manually constrain (or clamp) a quantity to some minimum or maximum value.

Whether or not a number or a length may be natively restricted to a certain range is dependent on the property that has this calc() value. From the spec:

The value resulting from an expression must be clamped to the range allowed in the target context.

These two are equivalent to 'width: 0px' since widths smaller than 0px are not allowed.

width: calc(5px - 10px);
width: 0px;

For example, while negative values for margin and z-index are valid, negative values for border-width, padding, height, width and so on are not. The specifics of these are almost always defined in the specification for the respective property.

As you happen to be using viewport units in your example, you can employ media queries in conjunction with them as demonstrated by fcalderan.