There is a language/cultural barrier between me and my supervisor, what can I do to improve this situation?

It will be much easier for you to learn German to conversational level than all of your co-workers learn English. Even if they are some-what able to talk in English, conversational English is actually pretty tough, whilst German is pretty structured and easy once you know the rules (and the many exceptions ;) )

More importantly, you moved to Germany. Whilst English is expected of all Scientists, there is probably more of an expectation in your lab that you will, eventually, learn German.

Personally, I would tell everyone you are going to German classes on Monday mornings, etc, and that should not only fix this issue, but also give you a positive perception among your co-workers as someone who is a fixer.

Also, speaking from experience as someone who didn't take their own advice when doing a PhD in Germany - if you dumb-down your English for non-native ears, you very quickly find yourself not being able to speak English with natives. Whenever i go back home my parents think I have some kind of brain damage...

You said that you understand more German than you can speak. I conjecture that the Germans on your team can understand more English than they can speak (or are wiling to speak). If so, then you can have conversations in which you speak English and they speak German. A better approach, since they're speaking German at lunch and you're just keeping quiet, is for them to keep speaking German while you speak a mixture of German and English --- German as much as you can, and falling back to English when you don't know the German word or phrase for what you want to express. Chances are they'll understand you, the conversation can keep going, and they might even tell you some of those German words or phrases that you didn't know.

My hunch is that this problem is more cultural than linguistic.

Do you feel left out of the conversation when your colleagues speak German? Could it be you are projecting that onto your colleagues? Normal behaviour for Germans is sometimes misinterpreted as cold and dismissive by people from Anglo-Saxon cultures. And in Germany, it's not generally considered rude to carry on a conversation in a group without actively involving everyone present.

It is natural, when conversation in one's second language is difficult to follow, to "fall off the train" -- that is, you miss a few words, and suddenly the conversation has left you behind and then it all seems like too much effort. That's when the "looking off into the distance" starts.

That's something people might notice.

Consider trying the following strategy: make an effort to follow the conversation. If you "fall off the train", get back on. Don't be afraid to ask questions. Maybe set a goal for yourself -- "I'm going to ask four questions in German about the conversation during this lunch break."

This will accomplish two things: first, you will at least appear interested. That will make you feel like more a participant, and it will encourage your colleagues to involve you. And second, it will help you learn the conversational language.

Finally, don't be afraid to talk to your colleagues about this. Judging by what you wrote about their English, they understand how challenging this is.

You are only four months in. Don't be so hard on yourself. What you are experiencing is normal. It will take time. But you are also a scientist; you can do this!