I am starting out learning Magento with no PHP experience, what are my options?
Unfortunately you won't be able to work on Magento without knowing at least some PHP so I would recommend that you at least learn the basics of the language (including how to use classes).
If you are intending to focus on just styling of the page (no new functionality that would require some code changes) then you should look at how magento layouts and themes work. All the content can be found in app/design folder and this is the place where the least PHP can be found.
Layout files define (if not otherwise speccified in code) which blocks, css and javascript files will be added where (this is the place where your javascript inclusion should go).
Templates are the actual block HTML content files (it is there where most of HTML is written and dynamicaly changed with PHP).
Note that layout and template files can be overwritten by placing them on different locations so you should never rewrite default layout and template files.
Resources
The best resource would be to look into already existing default layout and template files.
Google :)
Stack Overflow - This link is on how to add your own JS file to theme
Alan Storm tutorials - They are more technical so you should learn basic PHP first (he also wrote a book that extensively covers the Magento layout system but requires you to know PHP).
Magento For Designers - I didn't read the tutorials but they seem to go through Magento without expecting too much PHP knowledge
There is also an 40h online video training Fundamentals of Magento Development on Magento U, which you can take for free if you create an account. The Magento Knowledge Base is also a good place to start.
Before you start coding, you should learn how Magento actually works, about Zend's MVC Framework, how layout XMLs and blocks work together, etc.
But there is really no 100% answer to your question. It depends on where you want to go with Magento. Like, you asked to add your analytics js to your pages. This could be done via local.xml or even the simplest way with no file modifications at all in Magentos Backend under System > Configuration > General > Design > HTML Head > Miscellaneous Scripts by just adding your <script>... there.
I suggest you follow these steps to learn magento from you background
Learn HTML/JS/CSS: this will be used for front end development or any UI customization in client side so that you must expertise these things before doing anything
Learn PHP: at least you build your own PHP simple app from beginning to understand some basic feature. For example: you can build a simple CMS with authentication
Learn OOP & MVC: these design pattern are very important to get started with magento so that you can't miss it.
Install and configure magento: it is not too complicate to do this. At least you can set up a magento site from beginning to get it work and act as administrator of the site.
Install and configure some module: one step checkout, smtp, theme, language ... so you understand how the system work
Do the first theme customization
Do the first module customization
Do the full new module/theme
So these step can take you 3 to 6 months before you can do a good job.