Deploy custom labels with Ant

I got the same error and tried the code below. It worked. The metadata type is CustomLabel not CustomLabels. It's very strange.

<types>
    <members>*</members>
    <name>CustomLabel</name>
</types>

Reference:
Salesforce Metadata Types