Does it take longer to download a zipped file than an unzipped file?

When the connection is using compression, then of course.

You cannot efficiently compress data 2 times. So when compression is turned on, a 1 MB zip file will be transferred slower then a 1 MB txt file.

NB: This is dependent on the transfer protocol. FTP or other protocols don't have built-in compression. HTTP has.

It's not true if you are downloading via standard FTP or HTTP. For other connection types see Christopher's answer.

Assuming the same connection the speed of download is determined by the size of the file.

There might be a delay at the end of the download if you have automatic virus checking enabled as it will have to open and unpack the zip file to check the contents rather than being able to check the file directly.

If you use a PPP (dial-up or VPN) connection with compression, zipped files may download with lower speed than text files due to their nature (the former are already compressed and the latter will be compressed by the protocol thus increasing measured speed).

But if you compare amounts of information you receive, downloading zipped files will still be more efficient because any file archiver is usually superior to link-layer compression. So a zipped text file will be downloaded faster than the same text file verbatim, even if compression increases download speed a little.