Is copying paragraph structure plagarism?

It's not "plagiarism" because you are copying a standard format. That is known as scenes a faire in fiction, but can apply to Academia. This is not only allowed, but encouraged, because you are conforming to a "formula."

You should be worried if you were copying an unusual or original format. That's not the case here.