What are "instantaneous" rates of change, really?

In math, there's intuition and there's rigor. Saying $$ f'(x) = \lim_{h\to 0} \frac{f(x+h)-f(x)}{h} $$ is a rigorous statement. It's very formal. Saying "the derivative is the instantaneous rate of change" is intuitive. It has no formal meaning whatsovever. Many people find it helpful for informing their gut feelings about derivatives. Edit I should not understate the importance of gut feelings. You'll need to trust your gut if you ever want to prove hard things.

That being said, here's no reason why you should find it helpful. If it's too fluffy to be useful for you that's fine. But you'll need some intuition on what derivatives are supposed to be describing. I like to think of it as "if I squinted my eyes so hard that $f$ became linear near some point, then $f$ would look like $f'$ near that point." Find something that works for you.

The idea behind $$\lim_{h \to 0} \frac{f(x+h) - f(x)}{h}$$ is the slope of the graph $y=f(x)$ .

Now for a moment forget about your instantaneous velocity and think about your average velocity. What is average velocity? I think average velocity is $$\frac{x_f-x_i}{t_f-t_i}$$. Now if look at this carefully this is my slope but in a given interval of time.

So you might question what is the difference between instantaneous velocity and average velocity , Both of them talk about intervals .

No that not the idea over here. Now if it had been a linear graph it would have been very easy to calculate your instantaneous velocity , but in your you graph that's not the case. Here you see the particle (or object) is changing its velocity every moment of time and that becomes impossible to deal with . So to remove this element of doubt what we do is try to take the limit and try to get close to the frame of time and try finding the velocity and name it instantaneous velocity.

Think of a mountain: far away it's nearly flat; near the top it's more steep.

If you ask "How steep is it where I am standing right now?", you're asking exactly the same thing as "What is the rate of change of my elevation with respect to my position right here?"

I feel that should be pretty intuitive; that's what instantaneous rate of change means.

(Side note: in this case, you can move in two independent directions: north/south and east/west. That means there is a separate rate of change of elevation for each of those directions at every possible position on the ground. But in your case, you're only dealing with time, which can only go forward/backward, and hence there's only one number to worry about at each instant, not two.)