Can I return custom error from JsonResult to jQuery ajax error method?

You could write a custom error filter:

public class JsonExceptionFilterAttribute : FilterAttribute, IExceptionFilter
{
    public void OnException(ExceptionContext filterContext)
    {
        if (filterContext.RequestContext.HttpContext.Request.IsAjaxRequest())
        {
            filterContext.HttpContext.Response.StatusCode = 500;
            filterContext.ExceptionHandled = true;
            filterContext.Result = new JsonResult
            {
                Data = new
                {
                    // obviously here you could include whatever information you want about the exception
                    // for example if you have some custom exceptions you could test
                    // the type of the actual exception and extract additional data
                    // For the sake of simplicity let's suppose that we want to
                    // send only the exception message to the client
                    errorMessage = filterContext.Exception.Message
                },
                JsonRequestBehavior = JsonRequestBehavior.AllowGet
            };
        }
    }
}

and then register it either as a global filter or only apply to particular controllers/actions that you intend to invoke with AJAX.

And on the client:

$.ajax({
    type: "POST",
    url: "@Url.Action("DoStuff", "My")",
    data: { argString: "arg string" },
    dataType: "json",
    traditional: true,
    success: function(data) {
        //Success handling
    },
    error: function(xhr) {
        try {
            // a try/catch is recommended as the error handler
            // could occur in many events and there might not be
            // a JSON response from the server
            var json = $.parseJSON(xhr.responseText);
            alert(json.errorMessage);
        } catch(e) { 
            alert('something bad happened');
        }
    }
});

Obviously you could be quickly bored to write repetitive error handling code for each AJAX request so it would be better to write it once for all AJAX requests on your page:

$(document).ajaxError(function (evt, xhr) {
    try {
        var json = $.parseJSON(xhr.responseText);
        alert(json.errorMessage);
    } catch (e) { 
        alert('something bad happened');
    }
});

and then:

$.ajax({
    type: "POST",
    url: "@Url.Action("DoStuff", "My")",
    data: { argString: "arg string" },
    dataType: "json",
    traditional: true,
    success: function(data) {
        //Success handling
    }
});

---

Another possibility is to adapt a global exception handler I presented so that inside the ErrorController you check if it was an AJAX request and simply return the exception details as JSON.

The advice above wouldn't work on IIS for remote clients. They will receive a standard error page like 500.htm instead of a response with a message. You have to use customError mode in web.config, or add

<system.webServer>
        <httpErrors existingResponse="PassThrough" />
    </system.webServer>

or

"You can also go into IIS manager --> Error Pages then click on the right on "Edit feature settings..." And set the option to "Detailed errors" then it will be your application that process the error and not IIS."