Why does React discard the entire DOM subtree and recreate it from scratch?

This will depend in which scope DisplayA is defined. Functional components should usually be defined at the top level of a file. In your demo DisplayA is a component that is created inside the render of App, so every time App renders a new functional component is created, not a new invocation of the same component.

To resolve this make DisplayA top level in the file and pass props through to it.

const DisplayA = ({handleToggle, toggled}) => <div className={'containerA'}>
  <button onClick={handleToggle}>{"A toggled: " + toggled.toString()} </button>
</div>

const App = () => {
  ...
  return <>
    <DisplayA handleToggle={() => {...}} toggle={...} />
  </>
}

The second approach does not create a component which is passed to react for reconciliation, but is a function which is invoked while rendering and puts the contained elements into the rendering of this component.

In the first case you are not calling DisplayA. Instead, you are letting react decide when to render it. Notice how when transpiled, React.createElement(DisplayA) does not invoke this function. When react renders the subtree, it decides what needs to be re-rendered.

The process of handling tree changes/updates is called reconciliation. In react docs it says that same types will try to maintain state, whereas different component types will always perform a tear down on the DOM tree.

The latter happens with your DisplayA component because it's a different value on every render. Although the component renders the same view, React can't be sure that this is the same component because the value DisplayA points to a different component reference each time. In this case you are using function components, therefore the value is a reference to a new function. The variable name just happens to be the same - it does not have any importance at run time.

In the second case with displayB you are explicitly calling the function and rendering its result. Because the function is pure, this is equivalent to pulling its return value and inlining it inside the parent component:

return (
  <>
    <DisplayA />
    { 
      <div className={'containerB'}>
        <button onClick={handleToggleB}>{"B toggled: " + toggledB.toString()}</button>
      </div>
    }
  </>
)

Notice how the second child of this fragment is now a div. Because divs are primitive elements, they are represented by the literal string 'div' and not a reference to a component. React knows between renders that this is the same tree and as such does not destroy it. This would also work if you had any external component with a stable reference - it will be considered an element of the same type.