How to overwrite setTimeout before it reach the time set?

First, you need to save your timeout id in a global variable, or in a variable that can be accessed later when the function is called again.

Now, whenever your function is called, first you clear that timeout if it exists. Thus you clear any pre-existing timeouts and set a new one every time the function is called.

var myTimeout;

function searchVendor() {
  clearTimeout(myTimeout);
  myTimeout = setTimeout(searchVendor2, 5000);
}

function searchVendor2() {
  var search = $('#inputVendor').val();
  $.ajax({
    type: 'POST',
    url: '/getVendors',
    data: {search: search},
    dataType: 'json',
    success: function(s) {
      $('#inputVendor').autocomplete({source: s});
    }
  });
}

The other answers involving setTimeout() are simple and will work, but this is one of the few times when I would recommend using a utility library as they can go a few steps further in a way that is noticeable to the user.

It is also important that we avoid re-inventing the wheel. And in this case, what you want is a debounce or throttle function to limit the number of times your handler gets executed within a given time span. The good libraries also accept options to tweak when exactly your handler gets run, which can affect the responsiveness of your app.

Read more about debouncing and throttling.

For your use case, I would recommend Lodash's _.throttle() with both leading and trailing options set to true. This will ensure that long entries of text will still get some intermediate results, while also getting results as fast as possible (not having to wait for a timer the first time around) and still guaranteeing that the final keystroke will trigger a new result, which not all debounce settings would do.

const handler = (evt) => {
    console.log('I will talk to the server.');
};

const throttled = _.throttle(handler, 500, {
    leading  : true,
    trailing : true
});

Then register the throttled function as the event listener.

<input type="text" class="form-control input-sm" id="inputVendor" onkeyup="throttled()">