Where can I find country borders data in plain text format? (XML, JSON, CSV, etc)

I would suggest downloading and installing QGIS. You can then open the file that you link to and export the data in variety of formats.

The most common format for spatial data of this type is a shapefile which is one of the files that you linked to (TM_WORLD_BORDERS_SIMPL-0.2.zip). The reason that it is a zipfile is that there are several associated files with a shapefile and zipping them into a folder keeps them together.

When you download and install QGIS click on this icon. Then navigate to the folder where you unzipped TM_WORLD_BORDERS_SIMPL-0.2.zip and add the file TM_WORLD_BORDERS_SIMPL-0.2.shp. You should then see the country boundaries on your screen.

Then, right-click on the file name on the left part of the screen. There will be an option save-as and you can choose a variety of formats (in my screenshot I choose geojson).

I like @celenius' answer; however, another option would be to export the the World Borders Dataset to CSV using ogr2ogr. I just downloaded your preferred dataset and ran this command with ogr2ogr --note that your script should be a single string without any line breaks. I find them easiest to write in notepad with wordwrap turned on, then I copy them into my ogr2ogr terminal:

ogr2ogr -f "CSV" "E:\4_GIS\01_tutorials\worldborders\World_wkt"
   "E:\4_GIS\01_tutorials\worldborders\TM_WORLD_BORDERS-0.3.shp" -lco 
   "GEOMETRY=AS_WKT" -lco "LINEFORMAT=CRLF" -lco "SEPARATOR=SEMICOLON"

(CAVEAT: Copying/pasting my ogr script from the web and into your terminal won't work because the website will introduce linebreaks and ASCII character substitutions for the double-quotes. OGR will have trouble reading the script as a result. For this reason I recommend you type your adjusted script into notepad first, or directly into the terminal.)

OGR will create the World_wkt directory before it performs the file format translation, so don't make that directory before running your script or you'll get an error.

Here's a snippet of the results I got:

WKT;FIPS;ISO2;ISO3;UN;NAME;AREA;POP2005;REGION;SUBREGION;LON;LAT
"MULTIPOLYGON (((-61.686668 17.024441000000138,-61.73806 16.98971
"POLYGON ((2.96361 36.802216,2.981389 36.806938,3.001111 36.80971
"MULTIPOLYGON (((45.083321 39.768044000000145,45.266388 39.611107
"POLYGON ((19.436214 41.021065,19.450554 41.059998,19.513611 41.2
"MULTIPOLYGON (((45.573051000000135 40.632488,45.528881 40.606098
"MULTIPOLYGON (((11.750832 -16.75528,11.775 -16.804726,11.77 -16.
"MULTIPOLYGON (((-170.542511 -14.2975,-170.546112 -14.29861,-170.

That's a snippet of Well Known Text (WKT). Very human-readable, but those strings continue to the right for a long, long way, so I didn't want to copy-in the complete records. :)

If you just want to download the CSV dataset I created, I posted it here, but I would encourage you to try this technique for yourself. If you're interested and want more info, I wrote a closely-themed blog post a few weeks ago, which you're welcome to snipe for the more salient details.