Drupal - Custom Twig template for confirmation webform page being cached too hard

Seems like there are two different issues here:

(a) Loading a twig template from an arbitrary folder.

Could you try setting the path theme property and see if that allows your twig template to be loaded? For example:

'webform_confirmation__application_form' => [
  'template' => 'application-form-confirmation',
  'base hook' => 'webform_confirmation'
  'path' => \Drupal::moduleHandler()->getModule('application')->getPath() . '/templates',
],

(b) Modifying the cache settings for a part of your render variables.

You should check that the cache tags bubble all the way up from the message array. Apparently:

You have to render the content variable to ensure that its cache tags bubble up and end up in the page cache.

as mentioned here => (https://www.previousnext.com.au/blog/ensuring-drupal-8-block-cache-tags-bubble-up-page)

So, one place to start would be trying to render the whole content variable in your twig template, provided you got past (a)

Update: You could also try calling \Drupal::service(''page_cache_kill_switch'')->trigger(); in order to avoid Internal Page Cache to interfere with the request and page loading.

A simple way to do this inside a twig template would be using an approach similar to this module => https://github.com/stefanospetrakis/twig_killswitch_trigger

Good luck!

P.S.: Sample code is also available here => https://www.drupal.org/sandbox/stefanospetrakis/3011610, I may promote this to a full project if there is any potential interest.

Summary of the final answer to my original questions.

hook_theme was still required in the module to state that it can be used for suggested Twig templates.

function application_theme($existing, $type, $theme, $path) {
  $info = [
    'webform_confirmation__application_form' => [
      'base hook' => 'webform_confirmation'
    ]
  ];
  return $info;
}

hook_theme_suggestions_webform_confirmation_alter was not required in the end once hook_theme was setup correctly.

hook_theme needed to have the suggested name as proposed by the Twig debugger. Twig suggested these files:

<!-- FILE NAME SUGGESTIONS:
   * webform-confirmation--application-form.html.twig
   x webform-confirmation.html.twig
-->

The dashes get converted to underscores i.e. webform-confirmation--application-form is translated to webform_confirmation__application_form.

The only variable associated with that part of the array was base hook. This states to use the initial hook functionality rather than setting it all up again in my module - read more here.

base hook: Used for theme suggestions only: the base theme hook name. Instead of this suggestion's implementation being used directly, the base hook will be invoked with this implementation as its first suggestion. The base hook's files will be included and the base hook's preprocess functions will be called in addition to any suggestion's preprocess functions. If an implementation of hook_theme_suggestions_HOOK() (where HOOK is the base hook) changes the suggestion order, a different suggestion may be used in place of this suggestion. If after hook_theme_suggestions_HOOK() this suggestion remains the first suggestion, then this suggestion's function or template will be used to generate the rendered output.

template was not required so long as the filename of my Twig file matched the one Twig was suggesting. For my site it was webform-confirmation--application-form.html.twig.

With the above tweaks my Twig file was loading correctly while also still using all of the original hooks from the webform module.

The other issue was the confirmation page being cached for anonymous users. I was able to resolve this by using \Drupal::service('page_cache_kill_switch')->trigger(); inside hook_preprocess_webform_confirmation. After clearing Drupal's cache the confirmation page would then never get cached again.

function application_preprocess_webform_confirmation(array &$variables) {
  \Drupal::service('page_cache_kill_switch')->trigger();
}

This same logic will apply for updating Twig templates in other modules as well - this is not a webform module only solution.

Thanks to 4k4 and Stefanos for the guidance.