Math.min.apply returns 0 for null

Well, the numerical value for null is 0. If you don't want null values to be be considered, you have to filter them out:

var values = arrayObject.map(function(o){
    return o.y;
}).filter(function(val) {
    return val !== null
});

Reference: Array#filter

Alternative to Felix's solution: treat null as + or - infinity for min and max calls, respectively.

var max = Math.max.apply(Math, arrayObject.map(function(o) {
    return o.y == null ? -Infinity : o.y;
}));
var min = Math.min.apply(Math, arrayObject.map(function(o) {
    return o.y == null ? Infinity : o.y;
}));