Can we do web push notifications in chrome without using GCM/FCM?

No, it is not possible to use another push service.

In Firefox, you can do it by modifying the dom.push.serverURL preference, but obviously you'd need privileged access to alter the value of the pref.

There are third-party services that you can use to implement push notifications, but they will use the Web Push API under the hood (so Autopush on Firefox, GCM/FCM on Chrome).

Yes. Using VAPID spec and service worker you can use web push notifications without FCM/GCM. For more information please look into below google docs.

https://developers.google.com/web/fundamentals/engage-and-retain/push-notifications/how-push-works