How to set full calendar to a specific start date when it's initialized for the 1st time?

You should use the options 'year', 'month', and 'date' when initializing to specify the initial date value used by fullcalendar:

$('#calendar').fullCalendar({
 year: 2012,
 month: 4,
 date: 25
});  // This will initialize for May 25th, 2012.

See the function setYMD(date,y,m,d) in the fullcalendar.js file; note that the JavaScript setMonth, setDate, and setFullYear functions are used, so your month value needs to be 0-based (Jan is 0).

UPDATE: As others have noted in the comments, the correct way now (V3 as of writing this edit) is to initialize the defaultDate property to a value that is

anything the Moment constructor accepts, including an ISO8601 date string like "2014-02-01"

as it uses Moment.js. Documentation here.

Updated example:

$('#calendar').fullCalendar({
    defaultDate: "2012-05-25"
});  // This will initialize for May 25th, 2012.

You have it backwards. Display the calendar first, and then call gotoDate.

$('#calendar').fullCalendar({
  // Options
});

$('#calendar').fullCalendar('gotoDate', currentDate);

As per machineAddict's comment, as of version 2 and later, year, month and day have been replaced by defaultDate, which is a Moment, supporting constructors such as an ISO 8601 date string or a Unix Epoch.

So e.g. to initialize the calendar with a given date:

$('#calendar').fullCalendar({
    defaultDate: moment('2014-09-01'),
    ...
});