Android - Where does Google Now get its weather data?
Google displays Weather conditions using Weather Underground as per Google Web Search Features page.
All weather conditions and forecasts are provided by Weather Underground, Inc.
So, Google Now also might be picking from the same source.
Though this question is almost 5 years old but I think an updated answer can help users looking for updated info. Nowadays Google gets its weather data from weather.com as mentioned in bottom left corner of Google weather search results when you look for temperature of any location (highlighted in red rectangle).
But as the OP says a lot of times Google's weather data is way off the actual current weather. For example as of this writing it shows current temperature in Shimla, India to be -4 C at 9 pm, but according to official website of Indian weather department or IMD even whole day's minimum temperature was 1.4 C for 6 Jan 2018 and minimum temperature of whole day usually happens in early morning hours.
Similarly I have matched data for other cities and locations in India and current weather is not what actual current weather is. I think from wherever Google takes its weather data they just provide an average of previous years and not the actual current temperature. And various weather websites showing wildly different current temperatures for same location.