Is it possible to change password database file(/etc/passwd) in linux?
You're right: /etc/passwd and /etc/shadow are consulted by pam_unix.so, which are part of PAM. At least on modern Linuxes. You could change this by patching pam_unix.so. If the manpage is to be believed, you can't change the location of the system databases.
And you really don't want to. /etc/passwd isn't just used for authentication, it's also used for (reverse) name resolution and to look up things like the user's full name, shell, et cetera. The name and location are so standardised that moving them would almost definitely break things outside of just PAM. You'd have to patch a lot more than you bargained for.
Update: If you're trying to hide the /etc/{passwd,shadow,group} files for security, don't worry about it. Security by obscurity rarely helps as a policy. Leave them where they are and tighten the rest of your policies.
Update: a possible solution
If you have some custom software you need to access a different set of user/group databases, you could make a copy of the relevant PAM and NSS modules and patch them to use your custom databases. The original unix databases stay where they are so software isn't confused, but you can set PAM and NSS to use your custom modules wherever you need to and using whatever policy makes sense to you.
Keep the unix databases essentially pristine and you have what you asked for. This is pretty much what the radius and ldap PAM/NSS modules do: they provide an additional (not replacement) source of credentials and user/group information.
Going one step further: you can go back to PAM and NSS and disable the unix database lookups altogether. Leave the files there for old software (naturally, their views of the user/group databases will be inaccurate, but at least they won't break).
What you're looking for is the pam_pwdfile module. On Debian/Ubuntu the package is libpam-pwdfile (not sure about RedHat derived distros).
The README included with the package explains how to use it.
I'm not sure if it's possible out of the box, but It's definitely possible with some hacking. In order to do this you'll need to:
- Tune/hack nsswitch library (/lib/libnss_files.so or libnss_db.so - check your /etc/nsswitch.conf) so it would read some other file.
- Tune/hack pam_unix module so it would read some other files.