How to convert a custom type to an Integer in Haskell?

You can unwrap the Integer out of the Prime data constructor:

genPhi :: Prime -> Prime -> Integer
genPhi (Prime p) (Prime q) = (p-1) * (q-1)

If you have a simple newtype wrapper over an existing type, a good trick is to use the DerivingVia extension:

{-# LANGUAGE DerivingVia #-}

newtype Prime = Prime { unPrime :: Integer }
   deriving Num via Integer

phi :: Prime -> Prime -> Integer
phi p q = unPrime $ (p-1)*(q-1)

Given this, you can say:

*Main> phi (Prime 3) (Prime 5)
8

And furthermore all other numeric operations will automatically work on your Prime type, simply using their Integer equivalents.

See deriving via for details.

NB As noted in the comments, this doesn't mean GHC will make sure the result of these operations is Prime by any means; it just lets you lift the underlying operations. (But you could've made the same mistake without the deriving mechanism anyway.) If your program maintains the invariant that the Prime constructor always means the argument is a prime number, then do not use this trick. This is often not an issue as the invariant is either not clearly defined or easily enforceable, and the constructor simply acts as a reminder. But it's best to be clear on that and not use the deriving trick if you crucially depend on it.