What is the best way to check whether lead belongs to Queue or User?

The easiest way to do so is to not rely on the key prefix. Instead, just ask salesforce what the type is:

if(lead.OwnerId != null) { // User is currently known
    if(lead.OwnerId.getSObjectType() == User.SObjectType) {
        // Owner is a user
    } else if(lead.OwnerId.getSobjectType() == Group.SObjectType) {
        // Owner is a group
    }
}

To use Owner.Type, you need a query first:

Lead[] records = [SELECT Owner.Type FROM Lead WHERE Id IN :Trigger.new];

Note: Owner ID isn't always available in trigger context (e.g. Before Insert), because the owner isn't assigned until later.

The reason why you get the error Method does not exist or incorrect signature is due to Substring being a method of the String class. You need to cast the owner id into a string and then check.

Have a look at the string methods here

On the documentation, there are also examples of how to convert other data types into strings or assign the id to a string variable. You also need to specify the startIndex and the endIndex in the Substring method, information that is available in the link I've provided.