Is It possible to use Passport.js without Expressjs?

Passport Strategies expect inputs on certain channels (e.g. 'Local' looks for username and password fields on <OBJ>.body), but if you provide for these Strategy requirements directly you can use Passport just fine without Express/Connect.

Take a look directly at comments lifted from the relevant Passport code:

https://github.com/jaredhanson/passport/blob/master/lib/authenticator.js#L146

passport.authenticate('local', function(err, user) {
  if (!user) { return res.redirect('/login'); }
  res.end('Authenticated!');
})(req, res);

If you provide req and res as your own objects (paying attention to the requirements of the Strategy you're using and whatever you're using of req/res in authenticate), you have total control.

(Or just write a simple custom framework wrapper).