Why do Vim experts prefer buffers over tabs?

As ZyX said on #vim, this question sounds like "Why do Vim experts prefer tasty over warm?".

"Vim experts" don't prefer buffers over tabs: they use buffers as the file proxies they are and tab pages as the workspaces they are. Buffers and tab pages have different purposes so preferring one to the other makes no sense whatsoever.

The issue with buffers and tabs is one of confusion, caused by a combination of independent facts.

Most "modern" text editors and IDEs use a tab metaphor to represent loaded files. That metaphor acts as an information system — it shows the user what files are opened and their state — and as an interactive device — it allows the user to manipulate (reorder, select, close…) those opened files. Despite their many limitations, tabs are everywhere and people are used to them and expect them everywhere.
Vim introduced tab pages in 7.0 as a way for its users to create ad-hoc "workspaces". Nothing in their features, their specific options, their specific commands or their :help sections suggests that tab pages could or should be used as file proxies.

Nothing except the name and the appearance of "tab pages", of course, which leads to much confusion.
Without :set hidden, which is disabled by default and not very easy to find, Vim makes it impossible to switch to another buffer without writing the current one or abandoning its changes. New users, unaware of that option, have no choice but to turn to heavy windows use or to the closest "tab-like" feature they can find: tab pages.

"Tab page" is an unfortunate name choice for that feature, especially in an era dominated by the idea that reading documentation is a waste of time.

In Vim, tab pages are an abstraction built on top of windows, themselves an abstraction built on top of buffers. Each new level adds useful features but restricts your workflow.

The "buffer way"

With a buffer-based workflow, the files you are working with are distributed along a single dimension. You can cycle through your buffers, you can access a specific buffer by typing part of its name (with completion) or its number, you can alternate between buffers, you can target them pretty easily. There's basically no friction.

Eight buffers open, only one visible:
Switching by number:
Switching by name:

Buffers are Vim's file-proxies. If you think in terms of files, you think in terms of buffers.

The "window way"

With a window-based workflow, your "files" are both distributed along the same single "virtual" dimension as they would if you only used buffers and along two other "physical" dimensions. But the cartesian spaces in which those dimensions are found are almost completely separated: moving to another buffer still means "moving to another file" but moving to another window doesn't. The buffer that corresponds to the desired file may be displayed in that window but it could also be displayed in another one, maybe in another tab page, or not at all.

With windows, navigating between open files either becomes too complex or too simplistic, even with 'switchbuf' and :sb. Mostly because you are forced to use two sets of commands for what is essentially the same thing: accessing a buffer.

Windows have their use, as described below, but they don't have what it takes to replace buffers in anybody's workflow.

Here I am working on a Vim colorscheme. The two windows are different views of the same buffer: the top one serves as reference, with a table of the color codes used in the colorscheme, and the bottom one is where I work:

Working on a colorscheme

Windows are not designed as file-proxies and can't be made into ones: they are "containers" or "viewports" designed to offer you a view into a buffer. No more, no less.

The "tab way"

With a tab-based workflow, you essentially try to mimic the user experience you are used to from your previous editor while completely ignoring the very nature of Vim's tab pages. If we forget for a moment that this strategy is generally very unproductive, it is also impossible, just like with windows, to force Vim to adhere to that "one file = one tab" paradigm without losing a lot of flexibility.

Still working with the same files as above, the tabline takes up a significant space for virtually no benefit. All my files and all my tabs are called javascript*.vim so I can't do 3gt and be confident I'll end up at the right place and it is impossible to reach a specific tab by name. Add to that the fact that its label can very well be the very unhelpful but perfectly logical [Quickfix List]… Since there is no practical way to tie a file/buffer to a tab page, you are basically left with only one practical way to navigate between tab pages/buffers/files: cycling.

And yeah, my tabline is clobbered with only 8 tabs, imagine if I had 20!

Eight buffers open in eight tab pages (wrong)
Two tabs for two specific tasks (right)

Tab pages are "containers" or "viewports" designed to contain one or more windows, themselves also "containers" designed to contain buffers.

In conclusion

"Vim experts" (let's assume I can speak as if I was one) don't prefer buffers over tabs: they just use Vim as it was designed and are perfectly comfortable with that design:

"Vim experts" have 2, 30 or 97 buffers loaded and are very happy they don't have to deal with spatial distribution;
when they need to compare two files or work in one part of the current buffer while keeping another as a reference, "Vim experts" use windows because that's how they are meant to be used;
when they need to work for a while on a separate part of the project without messing with their current view, "Vim experts" load a brand new tab page.

Tosses 2c into the pile.

TLDR; :b *part-of-filename* is the best way to find a file you need in the buffer list i.e. it is FASTER and has LESS cognitive load than buffer numbers, tabs, or windows for tracking files.

It's nothing for me to have 30 buffers open (I.e. I haven't been housekeeping), and the beauty of buffers used-well is that it doesn't slow me down at all. In fact, it speeds things up when four days after I opened the file I need it, call :b *part-of-filename* and it magically appears, impressing co-workers and toady collectivists alike.

Buffers are for files.

To be effective:

Open an important first file from a devilishly well chosen root directory
Open subsequent files with :e
Use ls ALL the time when you first start to get a good mental model (you can't grok what you can't see, mentally or literally)
Never :q, it blows
Enter :b into your muscle memory
:b1 is good for the first file you know you opened, otherwise numbers and letters get clumsy quick
:b# is good for switching to your last file, which is a common need
:bd# is good for when you've switched to a temp file, done what you needed to do, switched back with :b#, and now want to close that temp file
:b *part-of-filename* is otherwise the best way to find a file you need in the list i.e. it is FASTER and has LESS cognitive load than buffer numbers, tabs, or windows for tracking files.

The only annoyance of :b *part-of-filename* is that sometimes you haven't opened the file yet, and you need to go back and :e path/to/full-filename first.

Tabs are for differentiating truly unrelated files.

Or keeping a particular windows layout handy (disclaimer: I've never used it for this myself).

Or for files rarely used, but predictably needed. For me, that's usually a commitMessage file that I annotate as I work so I don't have to do too much thinking when it comes time to make a commit. gt is faster than :b com<enter> (if you're feeling lucky, otherwise :b com<tab><enter>)

:tabe commitMessage
gt or gT also a muscle memory favorite

Window splits are for visually comparing information

Or having immediate access to important information (truthfully, unless that info is somehow something I need to live update with :e i.e. a log file, I usually just pull the content into the current file and deal with it there).

:vsp or C-w v opens a vertical split i.e. left | right, then use :b or :e to get the file you want
:sp or C-w s open a horizontal split i.e. top / bottom
C-w C-w i.e. double Ctrl-w, rotates you around the available windows
C-w c close current window
C-w o close all other windows, keep current ONLY

The downside of tabs is that you can only see the contents of one at a time. So if you use them like in a browser, you're losing out on viewing multiple buffers side by side, or even viewing separate parts of the same file in splits. Therefore, many recommend to use tabs only to segregate different workspaces (e.g. have one for a Java project, another for a todo list, a third to hack on a script on the side).

The problems you describe make it appear that you're using Vim wrong. Either have (mostly) a single, dedicated instance. Then, buffers that become hidden will simply "reappear" if you re-edit them (and you can now use the buffer list to recall them), and there won't be swap file messages. Or, use separate Vim instances per project / file / edit session, but then make it a habit to fully :quit each instance when you're done with the file.

I used to keep every buffer in a separate tab, but I grew tired of constantly gt and gT-ing around everywhere.

I also felt that buffers were too difficult to manage.

Here are some techniques that totally changed my earlier opinion:

Buffer management: :b. You get surprisingly fast at this. See vim, switching between files rapidly using vanilla Vim (no plugins)
Jump/Change lists (ctrl o/i and g;)
Alternate file (^)
tpope's Unimpaired plugin. Convenient mappings for flying through your buffers (among others).

Here is my typical workflow:

Open Vim, and use :e (usually with a regex like :e src/**/F*Bar.js) to open a buffer
Realize I need to open another file. Use :e for that as well. If I want to toggle between this buffer and the currently open buffer I will use :sp or :vsp to open it in a separate window.
Repeat until I've got the 3-5 files that I will be switching between using the techniques in the above bulleted list to fly between your buffers.
If I want to "start over" with my buffers, just close Vim and re-open.

I felt that after a week or so of forcing these new patterns, it became much easier to visualize which buffers I had open, and how to get to any one of them in only a few automatic strokes.