How to filter a very large bootstrap table using pure Javascript
Your best option is to not render all those things and store object versions of them and only show a max of 50 rows at a time via pagination. Storing that many objects in memory, in JS is no problem. Storing all of those in DOM on the other hand will bring browsers to their knees. 5000 is at around the upper bound of what a browser can do on a good machine while maintaining decent performance. If you start modifying some of those rows and tweaking things ('hiding', 'showing') things definitely will get even slower.
The steps would look something like:
- Organize the data into an array of objects, your hash map is great for supplementary and quick access purposes.
- Write some sorting and filtering functions that will give you the subsets of data you need.
- Write a paginator so you can grab sets of data and then get the next set based on some modified params
- Replace your "draw/render" or "update" method with something that displays the current set of 50 that meets the criteria entered.
The following code should be considered pseudo code that probably works:
// Represents each row in our table
function MyModelKlass(attributes) {
this.attributes = attributes;
}
// Represents our table
function CollectionKlass() {
this.children = [];
this.visibleChildren = [];
this.limit = 50;
}
CollectionKlass.prototype = {
// accepts a callback to determine if things are in or out
filter: function(callback) {
// filter doesn't work in every browser
// you can loop manually or user underscorejs
var filteredObjects = this.children.filter(callback);
this.visibleChildren = filteredObjects;
this.filteredChildren = filteredObjects;
this.showPage(0);
},
showPage: function(pageNumber) {
// TODO: account for index out of bounds
this.visibleChildren = this.filteredChildren.slice(
pageNumber * this.limit,
(pageNumber + 1) * this.limit
);
},
// Another example mechanism, comparator is a function
// sort is standard array sorting in JS
sort: function(comparator) {
this.children.sort(comparator);
}
}
function render(el, collection, templateContent) {
// this part is hard due to XSS
// you need to sanitize all data being written or
// use a templating language. I'll opt for
// handlebars style templating for this example.
//
// If you opt for no template then you need to do a few things.
// Write then read all your text to a detached DOM element to sanitize
// Create a detached table element and append new elements to it
// with the sanitized data. Once you're done assembling attach the
// element into the DOM. By attach I mean 'appendChild'.
// That turns out to be mostly safe but pretty slow.
//
// I'll leave the decisions up to you.
var template = Handlebars.compile(templateContent);
el.innerHTML(template(collection));
}
// Lets init now, create a collection and some rows
var myCollection = new CollectionKlass();
myCollection.children.push(new MyModelKlass({ 'a': 1 }));
myCollection.children.push(new MyModelKlass({ 'a': 2 }));
// filter on something...
myCollection.filter(function(child) {
if (child.attributes.a === 1) {
return false;
}
return true;
});
// this will throw an out of bounds error right now
// myCollection.showPage(2);
// render myCollection in some element for some template
render(
document.getElementById('some-container-for-the-table'),
myCollection,
document.getElementById('my-template').innerHTML()
);
// In the HTML:
<script type="text/x-handlebars-template" id="my-template">
<ul>
{{#each visibleChildren}}
<li>{{a}}</li>
{{/each}}
</ul>
</script>
Using AngularJS can indeed be a good idea, which lets us render your rows as simple as
<tr ng-repeat="row in rowArray">
<td>{{row.id}}</td>
<td>{{row.attr}}</td>
</tr>
where you only need to supply your rowArray as array of objects like {id: 1, attr: 'X'}, see the documentation for ng-repeat directive. One of Angular's big powers lies in its extremely compact code.
Among other things, Angular also has powerful filter building library to filter and sort your rows on the fly right inside your HTML:
<tr ng-repeat="row in rowArray | yourCustomFilter:parameters">
<td>{{row.id}}</td>
<td>{{row.attr}}</td>
</tr>
Having said that, it'll clearly be a performance drag to throw 5K rows into your array. That would create a huge HTML in your browser memory that, however, will not fit into your viewport. Then there is no point to have it in the memory if you can't show it anyway. Instead you only want to have the viewable part in your memory plus possibly a few more rows around.
Have a look at the directive "Scroll till you drop" provided by Angular UI Utils - it does exactly that!
Pagination as mentioned in another answer is surely a valid alternative to the infinite scroll. There is lot written on the web about strengths and weaknesses of pagination vs infinite scroll if you want to dig into that.
Speaking of your code specifically, it has other performance drags. For instance, on each call, this function
document.getElementById(id).style.display="none"
will look up the DOM for the element by its id, and then will look up its property .style (which can be a drag if the JavaScript needs to go high up in the Prototype chain). You could do much better performance wise by caching direct reference links to the display properties, which are the ones you really need.
EDIT.
By caching here I mean pre-compiling a hash linking id with the interesting properties:
hash[id] = document.getElementById(id).style.display
Then you switch the style by simple setting:
hash[id] = 'none'
hash[id] = 'block'
This way of calculating hash assumes that your elements are all inside the DOM, which is bad for performance, but there are better ways!
Libraries like jQuery and, of course, Angular :) will let you create your HTML elements with their complete style properties but without attaching them to the DOM. That way you are not overloading your browser's capacity. But you can still cache them! So you will cache your HTML (but not DOM) Elements and their Display like that:
elem[id] = $('<tr>' +
'<td>' + id + '</td>' +
'<td>' + attr + '</td>' +
</tr>');
display[id] = elem[id].style.display;
and then attach/ detach your elements to the DOM as you go and update their display properties using the display cache.
Finally note that for better performance, you want to concatenate your rows in a bundle first, and only then attach in a single jump (instead of attaching one-by-one). The reason is, every time your change the DOM, the browser has to do a lot of recalculation to adjust all other DOM elements correctly. There is a lot going on there, so you want to minimize those re-calculations as much as possible.
POST EDIT.
To illustrate by an example, if parentElement is already in your DOM, and you want to attach an array of new elements
elementArray = [rowElement1, ..., rowElementN]
the way you want to do it is:
var htmlToAppend = elementArray.join('');
parentElement.append(htmlToAppend);
as opposed to running a loop attaching one rowElement at a time.
Another good practice is to hide your parentElement before attaching, then only show when everything is ready.
I would ask
- Why you want to write this code for yourself? From personal experience, trying to filter efficiently and on all browsers is a non-trivial task.
- If you are doing this as a learning experience, then look at source of the packages listed below as examples.
- With 5000 rows, it would be more efficient to do server side filtering and sorting. Then use ajax to update the displayed table.
I would suggest that you look at using one of the several JavaScript packages that already do this. There are many more packages that the two below. I'm showing these two as examples of what is available.
- http://datatables.net/ - This is a very full featured package that handles both client and server side filtering and sorting.
- http://www.listjs.com/ - is a lightweight client side filtering and sorting package.