What is the point in publishing a paper in a journal rather than arXiv?
Papers published in (reputable) journals are reviewed by other scientists (peer review), which usually makes it considerably more difficult to publish a paper there. By contrast papers on the ArXiv only receive a brief inspection to keep out utter crap. Thus, most academic evaluations consider only peer-reviewed publications or value them considerably higher.
Moreover, often only peer-reviewed papers are considered citable, in particular for purposes of backing up your claims. This may extend to papers that are considered certain to be eventually peer-reviewed¹, but this is unlikely to apply to your paper. As being cited (by peer-reviewed papers) is another important academic evaluation metric, this is another advantageous aspect of journal publications.
As a sidenote: There are a lot of physics journals where you can publish without a fee.
¹ in particular in fast-moving fields or fields with long peer review such as parts of mathematics
Publishing in a journal typically means that your paper has been peer-reviewed. As far as I know arXiv does not review any papers.
Reminder: publishing is about making (a work) public. From a link on your webpage, to a highly considered journal, the ways you make it public provide a stamp on the level of "evaluation" your work are undergone. Going through peer-view and being published in a journal means that a board of editors, generally with the help of reviewers, consider that your paper possesses sufficient potential value to stand among the other papers.
Nobody knows the actual value of a paper just published. It takes time to reveal. Journals and archives (like arxiv) serve different purposes:
- journals: they are "official", and produce indices (the infamous "impact factor") of average "values" of papers published in the journal.
- arxiv: can store preprints, set a public "first date" for a submitted paper, since peer review can take long time.
If you stick to the publishing system, you can submit your paper to a journal, and perform an arxiv upload (in accordance with the journal's rules). When it is published, you can add this mention to the arxiv page. Having a paper under review, under revision or even better published would be quite good for a PhD admission. Having a paper on arxiv only shows that you are confident enough to share your work, but does not give the same impression.
If you do not stick to the system, you can put your preprints online, and hope that the people you talk to will effectively read your papers, instead of relying on peer-review judgement done by others.
For the second question, one might consider that a huge proportion of arxiv preprints that have not been published (say, a few years later) may raise questions about the publishable quality of the work. Unless one reads them to evaluate their content (and somehow do the peer-review work that has not been done before), this is likely to cast shades on an academia career. Unless one is a genius who does not care about evaluation, think about a recent Fields medal.