Alternative axiom to induction
A crucial difference between non-Euclidean geometries and "non-inductive" models of PA- is that any model of PA- contains a canonical copy of the true natural numbers, and in this copy of $N$, the induction schema is true. In other words, PA is part of the complete theory of a very canonical model of PA-, and as such it seems much more natural (so to speak) to study fragments of PA rather than extensions of PA- which contradict induction.
To make this a little more precise (and sketch a proof), the axioms of PA- say that any model $M$ has a unique member $0_M$ which is not the successor of anything, and that the successor function $S_M: M \to M$ is injective; so by letting $k_M$ (for any $k \in N$) be the $k$-th successor of $0_M$, the set $\{k_M : k \in N\}$ forms a submodel of $M$ which is isomorphic to the usual natural numbers, $N$. Any extra elements of $M$ not lying in this submodel lie in various "Z-chains," that is, infinite orbits of the model $M$'s successor function $S_M$.
(In the language of categories: the "usual natural numbers" are an initial object in the category of all models of PA-, where morphisms are injective homomorphisms in the sense of model theory.)
So, while PA seems natural, I'm not sure why there would be any more motivation to study PA- plus "non-induction" than there is to study any of the other countless consistent theories you could cook up, unless you find that one of these non-inductive extensions of PA- has a particularly nice class of models.
The literature on this question is large!
First recommendation: take a look at George Boolos (1984/1998)'s "The Justification of Mathematical Induction", in Logic, Logic, And Logic, pp. 370—375, Harvard University Press.
Boolos talks there about the general idea of proving induction, and discusses several principles whose correctness might be seen as more immediately amenable to intuition. He then discusses an idea of Dana Scott's, that induction can be justified in terms of an extended language of set theory with a relation "<" expressing which of two sets was formed at the earlier stage in the cumulative hierarchy. He argues that the principles of this theory are much less inductive that the principle of induction. This is Good Stuff, easy to follow, and an antidote to inappropriate worries about the logical strength of the principles involved.
Then, I recommend thinking about Getzen's proof of the consistency of arithmetic. The tricky issue with induction is that it expresses a metatheoretic property of the structure $\mathbb N$. Gentzen's proof transforms the issue, showing finitarily that the consistency of the whole system is the same thing as the well-foundedness of XP (the extended polynomials, i.e. the polynomials with addition, multiplication, and exponentiation). So the metatheoretical basis for the induction principle gets transformed, proof-theoretically, into a combinatorial thesis.
I found the citation below here
89e:03084 03E20 Swanson, Leonard G. (1-PRLS); Hansen, Rodney T. The equivalence of the multiplication, pigeonhole, induction, and well ordering principles. Internat. J. Math. Ed. Sci. Tech. 19 (1988), no. 1, 129--131.
Informal set-theoretic arguments are given for the equivalence of the principles mentioned in the title, all of which are stated for the natural numbers. The authors work in Zermelo-Fraenkel set theory, but such arguments should be given in a weaker system of set theory or arithmetic in which the principles in question are not theorems. The strength of several forms of the pigeonhole principle was studied by T. von der Twer [Arch. Math. Logik Grundlag. 21 (1981), no. 1-2, 69--76; MR 84e:03072].