Difference between <?php ?> and <script language="php"></script>

I think, in terms of semantics there is no advantages or disadvantages. According to PHP manual:

There are four different pairs of opening and closing tags which can be used in PHP. Two of those, <?php ?> and <script language="php"></script>, are always available.

Also, note that:

If you are embedding PHP within XML or XHTML you will need to use the <?php ?> tags to remain compliant with standards.

Anyway, the <?php ?> is the most commonly used and recommended to use it.

NOTE: Script tags <script language="php"></script> and ASP tags <% %> are removed from PHP 7.

They both work _{(much to my surprise)}.

However, <?php ?> is by far the most common construction. You're much more likely to confuse fellow coders if you use the very rare deprecated <script> method.

As a result, you should use <?php ?> merely to avoid confusion or potential compatibility problems across implementations.