Is there a way to detect horizontal scroll only without triggering a browser reflow

I think your code is right, because at the end of the day you need to read one of those properties to find out the scroll direction, but the key thing here to avoid performance problems is to throttle the event, because otherwise the scroll event fires too often and that is the root cause for performance problems.

So your example code adapted to throttle the scroll event would be like this:

var ticking = false;
var lastScrollLeft = 0;
$(window).scroll(function() {
    if (!ticking) {
        window.requestAnimationFrame(function() {

            var documentScrollLeft = $(document).scrollLeft();
            if (lastScrollLeft != documentScrollLeft) {
                console.log('scroll x');
                lastScrollLeft = documentScrollLeft;
            }

            ticking = false;
        });
        ticking = true;
    }
});

Source: https://developer.mozilla.org/en-US/docs/Web/Events/scroll#Example

Elements that use position: absolute; are taken out of the document flow (see source).

With this in mind, you could use CSS to make your element absolutely-positioned within its parent...

#parent{
    width: 500px;
    height: 100px; 
    // ...or whatever dimensions you need the scroll area to be
    position: relative;
}
#element{
    position: absolute;
    top: 0px;
    left: 0px;
    bottom: 0px;
    right: 0px;
}

... and then you can feel free to read the element.scrollLeft and element.scrollTop attributes as you did in your original question without fear of bottlenecks from reflowing the entire DOM.

This approach is also recommended in Google's developer guidelines.