Why is calculus considered to be difficult?
Part of it is that a lot of adults now never took calculus, because calculus wasn't as commonly offered in high schools as it is now. In their era, it was a college level subject. So they see it as super high level.
People fail in calculus courses because it is at a slightly higher conceptual level than pre-calculus and (high school) algebra. Calculus requires that you put in a lot of work doing practice problems, which is something a lot of people aren't willing to do.
Ultimately though, calculus is a bogeyman of sorts. It's really not the devil it's made out to be. For someone who did well in pre-calculus, calculus would just be the next progression, and it wouldn't seem like some huge jump up in difficulty. The real abrupt change in mathematics is from computation to proofs. I'm assuming your class isn't proof-based.
So, to not fail, just read your textbook, pay attention in class, and do practice problems. Normal behavior. You'll be fine, really.
In my opinion, Calculus is peppered with nonintuitive facts. These facts usually relate to infinity or thinking in the limit; calculus is often the first time you confront either of those two.
In support of my opinion is the fact that calculus was a massive human undertaking, spanning decades during which many super smart people "stood on each other's shoulders" and ignored complaints from the scientific community. There were doubts taking limits or using infinitesimals was valid and repeatedly, things people thought were true turned out not to be. Continuity implies differentiability? Nope: The Weirstrass function fails this, and badly. Think there can only be 1 "size" of infinity? Cue the continuum.
A pair of examples that I still don't find intuitive, one which you will see this year, the other you will see later in calculus:
Why does $\sum_{n=1}^{\infty}\frac{1}{n}=\infty$ but $\sum_{n=1}^{\infty}\frac{1}{n^2}$ a finite number (not to mention why it is actually equal to $\frac{\pi^2}{6}$).
Why is $\sqrt{x}$ uniformly continuous on the interval $[0,1]$, while $1/x$ on $(0,1)$ is not? They both have unbounded slope for some time on intervals of the same length.
One of the difficulties with calculus is the amount of time spent solving one problem. Most of that time is simplifying, evaluating, and the like, before applying the calculus-level concept at the end. If you can still do precalculus-level work fluently and without many mistakes, then the exercises won't take too long; it's when students get bogged down by loss of fluency that they get frustrated with how long it takes for one exercise (or exam question).
One other piece of advice that I haven't seen in the answers posted before this one: go to your instructor's office hours, even if you have no specific questions to ask. Just watch as others ask for help (and receive it). Doing that was a huge help for me (though I got a median-level score on the midterm, I got the highest score on the final), so I recommend it for all college math courses, but especially for calculus.