how to add values to an array of objects dynamically in javascript?

You have to instantiate the object first. The simplest way is:

var lab =["1","2","3"];
var val = [42,55,51,22];
var data = [];
for(var i=0; i<4; i++)  {
    data.push({label: lab[i], value: val[i]});
}

Or an other, less concise way, but closer to your original code:

for(var i=0; i<4; i++)  {
   data[i] = {};              // creates a new object
   data[i].label = lab[i];
   data[i].value = val[i];    
}

array() will not create a new array (unless you defined that function). Either Array() or new Array() or just [].

I recommend to read the MDN JavaScript Guide.

In Year 2019, we can use Javascript's ES6 Spread syntax to do it concisely and efficiently

data = [...data, {"label": 2, "value": 13}]

Examples

var data = [
      {"label" : "1", "value" : 12},
      {"label" : "1", "value" : 12},
      {"label" : "1", "value" : 12},
    ];
    
data = [...data, {"label" : "2", "value" : 14}] 
console.log(data)

For your case (i know it was in 2011), we can do it with map() & forEach() like below

var lab = ["1","2","3","4"];
var val = [42,55,51,22];

//Using forEach()
var data = [];
val.forEach((v,i) => 
   data= [...data, {"label": lab[i], "value":v}]
)

//Using map()
var dataMap = val.map((v,i) => 
 ({"label": lab[i], "value":v})
)

console.log('data: ', data);
console.log('dataMap : ', dataMap);