What is the difference between probability and statistics?
Your first sentence pretty much covers it. If you toss a coin 100 times and get the same outcome every time, the probabilist can tell you that that's just as likely as any other outcome, whereas the statistician will suspect the coin is biased. The probabilist can tell you what to infer about the data, given the model; the statistician can tell you what to infer about the model, given the data.
Probability theory is the study of theoretical random processes in which everything is known in principle. Problems in probability theory involve being given some of the attributes of a theoretical random process and being asked to solve for others that are related.
Statistics is two completely different subjects.
Descriptive statistics is the study of the descriptions and properties of probability distributions themselves, theoretical or empirical.
Inferential statistics is the study of what can be told about a random process of which all or part is unknown, by examining one or more samples.