How are the facebook chat windows implemented?

Probobly just an absolutely positioned div, containing a scrolling div for the content. Ajax would provide the content.

If you install FireBug and enable the net monitor for Facebook, you'll see that when you click most links inside the application, you're not doing a full page refresh, but rather an AJAX call which updates the page with the new content.

It looks like a new page, but in reality you're on the same page with just about everything but the chat-bar replaced.