Is Hartshorne's Remark III.2.9.1 actually a valid argument?

The key is to get more inventive with your diagram. Instead of taking the colimit all at once, build it up in pieces: if $I$ is an infinite set, then we can consider the diagram made up of finite subsets $J\subset I$ where the arrows are inclusions. Then $\bigoplus_{i\in I} A_i$ is the colimit of $\bigoplus_{j\in J} A_j$ as $J$ varies over that diagram, and as cohomology commutes with finite direct sums and direct limits, we have a proof.

An infinite direct sum is naturally the direct limit of its finite subsums (that is, $\bigoplus_{i\in I}A_i$ is the direct limit of $\bigoplus_{i\in F}A_i$ where $F$ ranges over the directed set of finite subsets of $I$). Since cohomology preserves finite direct sums and direct limits, it follows that it preserves infinite direct sums.