How to send an Email Template via Apex?

In your code you need to bring in a slight change. Where you setting AccountId in setWhatId() method, change it to

mail.setWhatId(ContactId);

A brief explanation from the docs.

setWhatId(whatId) If you specify a contact for the targetObjectId field, you can specify an optional whatId as well. This helps to further ensure that merge fields in the template contain the correct data.

Replace the setSubject method parameter with the value you are fetching from the EmailTemplate object. This way you can change the email content by changing it in the template.

EmailTemplate et = [SELECT Id,Subject, Body FROM EmailTemplate WHERE DeveloperName =:emailTemplateName];
List<string> toAddress = new List<string>();
toAddress.add(primaryEmail);
Messaging.SingleEmailMessage mail = new Messaging.SingleEmailMessage();
    mail.setTemplateId(et.Id);
    mail.setToAddresses(toAddress);
    mail.setSubject(et.subject);
    mail.setHTMLBody(et.Body);
    mail.setTargetObjectId(primaryContact);
    mail.setWhatId(primaryAccount);
    mail.setSaveAsActivity(false);
    mail.setUseSignature(false);
List<Messaging.SingleEmailMessage> allmsg = new List<Messaging.SingleEmailMessage>();
allmsg.add(mail);

try {
    Messaging.sendEmail(allmsg,false);
    return;
} catch (Exception e) {
    System.debug(e.getMessage());
}