Who first defined _simply connected_, reference?

It seems indeed pretty clearly to have been Riemann, in §6 of his Inaugural Dissertation (1851). There he defines zusammenhängend as well as einfach, zweifach and mehrfach zusammenhängend without citing any prior sources.

Yet a possible antecedent is Gauss, of whom J.-C. Pont's book (cited in the comments above and reviewed here) singles out a text, written around 1840, "which one can consider as a sketch of the theory of the order of connectivity". It would be interesting to dig Pont's exact reference, which ought to be available in Gauss's Werke.

(On Gauss and Riemann see also a letter of Betti published by A. Weil, Riemann, Betti, and the Birth of Topology, Archive for History of Exact Science 20 (1979) 91-96.)

The reference mentioned in Francois Ziegler's answer, that is given on page 37 in Pont's book is p.407-410 in volume 10.1 in Gauss's Werke:

"Bestimmung der Convergenz der Reihen, in welche die periodischen Functionen einer veränderlichen Größe entwickelt werden"

(It's a bit tricky to find the reference, since there is a typo where Pont gives the reference: 33 d should be exchanged with 33 c)