When we refer to topology on a metric space $S$ , do we mean the topology generated by open ball?

Yes, a metric space $(S, d)$ induces a topology $\mathcal{T}$ which is generated by the basis $$\mathcal{B} = \{B(x, r) \ | \ x \in S \ \text{ and } \ r > 0\}$$

When authors refer to the topology on this metric space $(S, d)$, they usually mean the topology $\mathcal{T}$ above, which you can think of as the topology generated by open balls.

Yes. So you can say that a set $U$ is open iff for each $x\in U$, there exists an open ball $B(x,r)$ centered at $x$ with $B(x,r)\subset U$.