Why is Physics so hard?

The problem is: There are so many formulas which you can combine, rearrange and manipulate in oh so many ways that just knowing the formulas doesn't get you that far. What you need is intuition and understanding as to which formulas relate to your problem and how you should relate them to get what you want. This intuition can only be built through experience, i.e. numerous hours of problem solving.

That's why it's so important you do all the problem sets you can find, because only then will you become really familiar with the formulas.

Think of it like learning to play chess: You can learn the rules in an afternoon, they aren't that hard. But this teaches you nothing about how to be a good chess player. That takes years of practice.

There is an answer to your question in the form of a physics paper : Determining dynamical equations is hard by Toby Cubitt, Jens Eisert and Michael Wolf. Basically, of you define physics as determining dynamical equations —the laws of physics— from some observations —the experiments—, they prove that this problem is NP-complete.

In other words, physics is not only hard, it is NP-hard !

I dont know what else to do at this point :(

Party less? Just kidding. Well @fprime, the first thing to realize is that physics is really friggin' hard and it seems to me that once you're over that hump you might actually end up enjoying it. However, loving the subject does not imply that you will never have night terrors where you wake up screaming "Its a real job I tell you, its a real job".

So if you think you have what it takes then hold your nose and dive in. Six months and no social life later you might start finding physics a lot easier and also a lot more different than what you expect.

One good place for beginners seeking inspiration is the set of lectures by Feynman. Good luck !