Use new cellgroup opener button for closing too?

Automatic closer is not there but it is on a todo list:

An inline closer is coming in the next version per popular request. The thinking was that it might be ugly/annoying to have closers everywhere, but the current plan is to only show closers on groups where you opened them with the openers. That plan may or may not survive the fire of testing.

- John Fultz in chat

Meanwhile you can create a custom one:

How to change the appearance of a Cell GroupOpener?

or use the old one, called "OutsideFrame" now:

ShowGroupOpener's mess in 11.3