Is it a turtle prime?

Haskell, 104 102 99 98 97 95 91 bytes

p x=product[2..x-1]^2`mod`x>0
f[]=1>0
f x|y<-zip[0..]x=or[f[snd b|b<-y,b/=a]|a<-y,p$read x]

Try it online!

Explanation

First we set up a primality test

p x=product[2..x-1]^2`mod`x>0

This uses Wilson's Theorem to determine the primality of an input.

We then declare a base case, which will assert that the empty string is truthy.

f[]=1>0

Now we define the actual function

f x|y<-zip[0..]x=or[f[snd b|b<-y,b/=a]|a<-y,p$read x]

We use a pattern guard to bind zip[0..]x to y, because we need to use it twice later. We then assert the answer is

or[f[snd b|b<-y,b/=a]|a<-y,p$read x]

[[snd b|b<-y,b/=a]|a<-y] is all of the numbers that are a digit removed from our input. So we are asserting that at least one of these numbers is truthy for f. In order to ensure that composite numbers are falsy we add in prime$read x. If the number is not prime the list will become empty and the any of a empty list is false.

R, 124 122 120 113 95 93 106 105 bytes

 g=pryr::f(`if`(gmp::isprime(sum(x*10^((l<-sum(x|1)-1):0))),any(!l,sapply(0:l+1,function(z)g(x[-z]))),!1))

Which evaluates to the function:

function (x) 
if (gmp::isprime(sum(x * 10^((l <- sum(x | 1) - 1):0)))) any(!l, 
    sapply(0:l + 1, function(z) g(x[-z]))) else !1

Recursive solution. Takes input as a list of digits.

Has 2 logical statements:

1. Is x prime when concatenated?

2. Is any of the following TRUE:

   1. Is the length of x nonzero? This is our final terminating condition.

   2. Is f TRUE for any subset of x?

The first statement ensures we keep working with primes only. The second does the actual recursion.

Saved two bytes thanks to @Giuseppe.

I had to revert some of my golfs because of a bug, where I was testing with a previous function definition by accident.

R, 98 bytes, non-competing

Like I mentioned in the comments, I made a package. Since the challenge predates that, this is non-competing, but I wanted to showcase it a bit. It's not much so far, but we'll get there.

g=pryr::f(`if`(gmp::isprime(RG::C(x)),any(!(l<-sum(x|1)-1),sapply(0:l+1,function(z)g(x[-z]))),!1))

C() is the first function in the package, and takes care of concatenating digits into a numeric.

Jelly, 16 bytes

DŒPḊṖLÐṀḌ߀¬Ȧ<ÆP

Try it online!

How it works

DŒPḊṖLÐṀḌ߀¬Ȧ<ÆP  Main link. Argument: n

D                 Decimal; convert n to base 10.
 ŒP               Powerset; get all sub-arrays of n's decimal digits.
   Ḋ              Dequeue; remove the first sub-array (empty array).
    Ṗ             Pop; remove the last sub-array (all of n's digits).
     LÐṀ          Maximal by length; keep those of the remaining subarrays that
                  have maximal length. This keep exactly those sub-arrays that have
                  one (and only one) digit removed. If n < 10, this yields an empty
                  array. Without Ḋ, it would yield [[]] instead.
        Ḍ         Undecimal; turn the generated digit arrays into integers.
         ߀       Recursively map the main link over the generated integers.
           ¬      Negate; map 1 to 0 and 0 to 1.
            Ȧ     Any and all; yield 0 if the array is empty (n < 10) or any of the
                  recursive calls returned 1 (mapped to 0). If all calls returned
                  0, this will yield 1.
              ÆP  Test n for primality, yielding 1 for primes, 0 otherwise.
             <    Test if the result to the left is less than the result to the
                  right. This is possible only if the left result is 0 (n < 10 or
                  removing a digit results in a turtle prime) and the right result
                  is 1 (n itself is prime).