How can I validate for only CSV file uploads using the pattern attribute using HTML5?

Now you can use the new HTML5 input validation attribute:

pattern="^.+\.(xlsx|xls|csv)$"

Accept type for other files (Reference: HTML5 Documentation):

For CSV:

<input type="file" accept=".csv" />

For Excel files, 2003-2007 (.xls):

<input type="file" accept="application/vnd.ms-excel" />

For Excel files, 2010 (.xlsx):

<input type="file" accept="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" />

For text files (.txt):

<input type="file" accept="text/plain" />

For image files (.png, .jpg, etc.):

<input type="file" accept="image/*" />

For HTML files (.htm, .html):

<input type="file" accept="text/html" />

For video files (.avi, .mpg, .mpeg, .mp4):

<input type="file" accept="video/*" />

For audio files (.mp3, .wav, etc.):

<input type="file" accept="audio/*" />

For PDF files, use:

<input type="file" accept=".pdf" /> 

Use this:

<input type="file" name="txtFile" id="txtFile" class="required" accept=".csv,text/csv" />

Mentioning the MIME type is good practice.

It is not easy to accept just one of the file types. It depends on different OS and installed text reading programmes. In Unix type systems if you pass .csv file you will get .csv file type, but if you will pass the same file on Windows you will ger different file types, such as text/comma-separated-values, text/csv, application/csv, application/excel, application/vnd.ms-excel, application/vnd.msexcel, text/anytext.

It calls mime-types and can be different for any of text type files. Here is the link where you can read more.