Developing casual relationship with advisor
You stated a key point
Perhaps this is just my personal preference, but I'd like to have a more casual relationship with someone that I see everyday and talk to everyday.
It could very well be that his personal preference to take time to foster the kind of relationship that you describe - it could also be a case of him not feeling comfortable have a casual relationship with his employees/advisees. There is no way to really know how long he has known the other professors that he speaks and laughs with.
You can not and should not force it. This is very important, some people recoil when people try and force these kinds of things.
To be honest, based on the conversation snippet, it sounds like he is not being rude at all, but just is focused on the task and on maintaining positive progress.
It is not a bad thing to have a strong working relationship - a polite and productive relationship is beneficial to many working relationships. I would suggest to focus on that aspect.
My advisor was rather driven. I suppose if he hadn't been, he wouldn't have accomplished nearly as much as he did.
There were times when he was ready to open up, and times when he wasn't. Once, for example, he asked me to drive him to the airport. I went to the wrong airport because he was telling me how he made it through food shortages in the immediate postwar period and it was fascinating. (It was okay, I realized my mistake and got him to the correct one in time for his flight.)
In contrast, I remember running into him on a Sunday when we were walking up the steps of our building, heading to our respective offices. I was elated about a concert I had been to the night before, but I could see that he hadn't even taken in what I had said. He was completely focused on getting to his office and progressing with whatever he had in his head that day.
My spouse is like that too.
Don't take it personally.
Although it is certainly true that many faculty are socially maladroit, it is also true that there is a definite delicacy in the mentor/mentoree relationship. It is not a peer relationship, and it is far from equal in terms of "power". Thus, even with the best of intentions, it cannot truly be a "peer" relationship, even if "friendly". And I think that it is important, just as in good parenting, that as mentor/superior, one be absolutely sure at all times to be squeaky-clean, and adamantly assume the persona that one knows one should.
Sure, this does not directly preclude "more genuine" personal interactions, but, actually, it is completely mandatory that one stay within certain boundaries.
Some faculty have a subliminal understanding of this, but/and through nerdiness cannot behave in a way that accommodates this complication-to-life. Others clumsily ignore it, and may "hit on" their students and postdocs...
Seriously, there truly are (in my opinion) several delicacies in personal relationships with people over whom one has great power. That enormous power differential is inevitably an elephant-in-the-room, and to pretend that it's not is, for the more-powerful person, violently irresponsible, in my opinion.
So, there is a range of ways your supervisor may be "impersonal", and they are not at all necessarily negative.
(Also, as I have gradually come to understand, substantial age differences do entail substantially different perceptions, and different implicit interpretations of situations... who knew?)