How do i call a controller function in ajax?

Instead of $this->getUrl(), you can try

 Mage::getUrl('adminhtml/attributes/Ajax');

Please note the method is Ajax and not ajax. This is because your controller consists of a method AjaxAction() and it is not ajaxAction().

Generally this is what Mage::getUrl('module/controller/method') looks like. The fields are self explanatory I hope.

Why your code didnt work?

This is because you are calling getUrl() on $this which is wrong. getUrl() function comes inside Mage and it is a static function. So you need to call that method like this Mage::getUrl()

Wierd that, Mage::getUrl() is not working for you. I will investigate the reason when I get some free time. Now I will answer why Mage::helper('adminhtml')->getUrl() worked.

Mage::helper('adminhtml') will return admin helper class. More specifically it return the class Mage_Adminhtml_Helper_Data which is located at app/code/core/Mage/Adminhtml/Helper/Data.php. If you look in this file, you can find the method getUrl()

public static function getUrl($route='', $params=array())
{
    return Mage::getModel('adminhtml/url')->getUrl($route, $params);
}

You can see that in case of admin section, magento uses Mage_Adminhtml_Model_Url model class in order to generate the url. This is the exact reason why Mage::getUrl() didn't work in this case I suspect. This is because Mage::getUrl() uses Mage_Core_Model_Url model class in order to generate the url, which is I think generally used for frontend url generation.

So the big point is Magento uses a special model class Mage_Adminhtml_Model_Url class for processing magento admin urls

Your request is not reaching your controller because of the way you have defined your controller resource in your config.xml. You are using building a URL with the call:

$this->getUrl('attributes/attibutes_ajax'); // also a typo here, missing 'r', underscore should also be '/'

but you have defined your controller with:

<attin_exportdb before="Mage_Adminhtml">Attin_Exportdb_Adminhtml</attin_exportdb>

The problem lies in the fact that you have not declared a new controller with a frontName, rather used the before syntax to tell Magento to look for a controller match in your module before reverting to Mage_Adminhtml. This is a good way of doing things for admin, but in order to make it work you must target the admin module in your request not your own. This will trigger the logic to check your module for a match first, then revert to Mage_Adminhtml.

So when you are generating the URL you need to be using:

Mage::helper('adminhtml')->getUrl('adminhtml/attributes/ajax');

which will target the admin module, your attributes controller, and the ajax action within that controller. Using the adminhtml helper getUrl() method will include the necessary key in the generated URL (otherwise a request to admin will be rejected).

By using:

Mage::helper('adminhtml')->getUrl('attributes/attributes/ajax');

you would have needed to have delcared your own controller rather than using before and set a frontName similar to the following in your config.xml:

<?xml version="1.0"?>
<config>
    ....
    <admin>
        <routers>
            <exportdb>
                <use>admin</use>
                <args>
                    <module>Attin_Exportdb</module>
                    <frontName>attributes</frontName>
                </args>
            </exportdb>
        </routers>
    </admin>
    ....
</config>

Initiate js in any.phtml file!

jQuery.ajax({
    url: "/module/index/getdata"
    type: "POST",
    data: {key: 'value'},
    success: function(response){
        console.log(response);
    }
});

Write controller app/code/local/Package/Module/controllers/IndexController.php

    Package_Module_IndexContoller extends Mage_Core_Controller_Front_Action{
        public function getdataAction(){
            if($data = $this->getRequest()->getPost("key")){
                echo "data received !";
            }else{
                echo "unable to receive data !";
            }
        }

    }