What typically is the expiration date of a session cookie?
I'm guessing it's the browsing session, so if I don't set an expiration date this will be used as the default, right?
Yes. Unless you have a particular need for sessions to survive a browser restart, omit the expires parameter so that the cookie is browser-session-only and not persisted to disc.
does it depend on how long I want users to stay logged in before automatically logging them off
That is governed by your actual session expiry time, which should be implemented on the server-side alone. If you do use an expires time you would generally want it to be at least as long as the server-side timeout, but you shouldn't rely on the browser honouring that expires as your method of ensuring old sessions are unreachable.
Generally, session-only (no-expires) cookies are used for session-tracking, with timeout happening on the server side. If a request is made with an unrecognised or missing cookie, then likely the session has expired at the server side, the browser has been closed at the client side, or both, and you should direct the user to start a new session.
Typically there will be a session management tool included in whatever your web framework is on the server-side that will work this out for you by sending the appropriate Set-Cookie headers on an HTTP response (either the initial HTML page, or an XMLHttpRequest response). Whilst you could reimplement session management yourself using only JavaScript, passed parameters and, say, localStorage as an alternative to cookies, there doesn't seem to be that much to win by reinventing that wheel.
My recommendation would be: "Don't create the authentication cookie using JavaScript." A cookie identifying an authenticated session should be marked wit the HttpOnly flag to help mitigate XSS attacks, and so must be created by the server and sent with the response, not created on the client.
That bit of advice aside, your assumption is mostly correct. If there is no expiry set on the cookie, then it is a session cookie and will live as long as the browser is open, and the sessionid is valid. If the server expires the authenticated sessions periodically, then the cookie will no longer be attached to a session on the server and will therefore be essentially null.
To your second question, if you wish to specify a maximum amount of time a user is logged in before needing to re-authenticate, it's usually done with a rolling expiry, where the expiration time is updated with each request to be x minutes from now, so active user sessions aren't forcibly expired, only idle sessions where a user hasn't made a new request in the last x minutes. The most secure way to do this is to tie the value of the cookie to a session on the server that expires on time, which can't be interfered with by the user. The expiry on the cookie is not sufficient, as it can be changed by the client. If you need to store a session expiration client side, it needs to be encrypted in the value of the cookie, so again needs to be created server-side, not by JavaScript, because the server must be the only place the value can be decrypted in order for it to be secure.
And lastly to your third question, what is an appropriate amount of time before expiring a session? It depends entirely on your application. Financial applications often have very short timeouts of five or ten minutes. Many applications have a more traditional default time out of 20 or 30 minutes. If the workflow of your app requires extensive amount of time on a page without refreshing, even longer may be in order. I don't know that it's terribly important in any case, unless your application has specific security needs.
I'm not a web developer so this could be wrong but I would expect that you could just use the Set-Cookie: header in the HTTP response to the AJAX query to set the session cookie.
You should not need to pass the session ID inside the AJAX response and then use Javascript to set that cookie. The standard PHP session_*() functions should handle setting the expiry time correctly for you.
On a security note, HTTPOnly, Secure and SSL. You should be doing all of these.
You should also be expiring sessions on the server both when the user logs out and after a certain period of inactivity from the user. The period you choose is a tradeoff between security and usability. The default in PHP is 1440 minutes (24 hours).