Why do we need .so.1 file in Linux?

Let's say we are talking about libtest. If you look around, you'll see libtest.so, which is a link to libtest.so.1, which in turn links to libtest.so.1.5.

An executable using libtest will link against libtest.so which is libtest.so.1 in this case (this is written into the executable, see ldd(1)). If your distribution changes libtest to fix bugs, the new version might give libtest.so.1.6 (and after update libtest.so.1 links to it, running programs will still use libtest.so.1.5 until they exit). As long as no ABI changes are made, everything works fine. And the fact that there are no API changes is signalled by the unchanged 1 version number.

Let's say the busy libtest beavers come up with a new, all shiny, rewritten from scratch library, with changed ABI. As the ABI changed, they change the major version number to 2. You install that one, and now you have the chain libtest.so --> libtest.so.2 --> libtest.so.2.1. Note you have now both versions 1 and 2 installed. Your earlier programs still work fine, using libtest.so.1, but if you compile a new program the compiler (linker, really) will pick up libtest.so and thus point the executable at the new libtest.so.2.1 (unless specifically asked to use the old version, that is).

Note that the so version numbers don't need to have any relationship to the source code version numbers; the major number is ABI version, the minor number is optional and can be used to track revisions. So here (Fedora 20) I'm using systemd-libs-208-15.fc20.x86_64, which provides libsystemd-daemon.so.0.0.10.

These are different versions of the same library.

Normally, you want the most recent stable version of the library, so you link to x.so, which is linked to that most recent version. When a new version becomes available, say x.so.2, you can make your whole system use it, by linking x.so to x.so.2

Sometimes you want to link to an old version - for example if your program relies on a quirk that's been fixed, or the latest version has introduced a bug. Sometimes you want to link to a new experimental version - for example if you're testing it, or if it fixes a bug in the current version. In that case you would link directly to the numbered version - hopefully as a temporary measure.