Do non-academic extra curricular activities matter for grad school applications?

In general. non-academic activities are not considered for admissions for schools and most fellowship programs in the US. The primary reason for this is that graduate programs are looking for people who will become excellent scholars and researchers (and sometimes even good teachers!). Success in sports and other highly non-academic activities is less likely to have significant influence in assessments of one's ability to do research, and therefore is not strongly considered.

However, there are exceptions. People can choose to fund activities however they choose, and can place whatever restrictions on the use of the money they donate. For instance, an alumnus could fund a fellowship for PhD students who play the tuba in a college marching band. While this doesn't happen often, it can be done. But typically admissions committees aren't concerned with such issues, and don't take them into account in making admissions decisions.

In my country (Asia region), people are fanatical about non-academic extra curricular activities that are totally not related to their topic (sports, debate team, run charities etc). ... I think the situation is different in the USA.

Its not actually that different. Universities are generally looking for engaged graduate students, so showing involvement outside of classes is generally a positive thing. Extra-curriculars should be noted in any application.

Holding a leadership role in a student organization is seen as positive, and should be especially noted in any graduate school application. This includes leadership positions in Fraternities or Sororities. See below for the one exception to this rule.

Think carefully before listing religious, political or fan clubs

Be careful listing activities that involve either religion or politics. Its a fact of life that some people are prejudice, and listing a republican, democratic, LGBT, or any other club that would imply a strong religious or social viewpoint opens you to attack. Do not give someone on the admittance committee reason to hate you.

Also be careful listing fan club activities like your Quiddich team, or presidency of the Star Trek club. These activities may make you appear immature to the admittance committee.