Trouble using SOQL to filter results by a range of Dates - What is the correct syntax?

CreatedDate is of DateTime type so your query must use the format from the very top of the article you've linked to: YYYY-MM-DDThh:mm:ssZ (optionally with timezones)

SELECT Id 
FROM Case  
WHERE CreatedDate >= 2013-12-21T00:00:00Z
AND CreatedDate <= 2013-12-23T23:59:59Z

Another example:

SELECT Id
FROM Account
WHERE CreatedDate > 2005-10-08T01:02:03Z

Salesforce filtering by date examples:

All these SOQL queries are working for me on Salesforce API version 29

Greater than a certain time:

SELECT count(Id) 
FROM Lead
WHERE createdDate > 2015-02-01T00:00:00Z

Greater than yesterday:

SELECT count(Id)
FROM Lead
WHERE createdDate > YESTERDAY

Greater than 62 days ago:

SELECT id
FROM Lead
WHERE CreatedDate = LAST_N_DAYS:62
LIMIT 100

Greater than last week:

SELECT Id
FROM Lead
WHERE createdDate > LAST_WEEK

This year:

SELECT Id
FROM Lead
WHERE CALENDAR_YEAR(CreatedDate) = 2015

Get the current Year returned:

SELECT CALENDAR_YEAR(CreatedDate)
FROM Opportunity
group by CALENDAR_YEAR(CreatedDate)