What are the most common (usual) ways to make palettes with non-trivial functionality?

All palette state (i.e., variables which affect the palette and should be remembered between sessions) should be vectored through the palette's TaggingRules option, and its initialization should be done in the palette's NotebookDynamicExpression option. That, plus context isolation of any kernel functions you need to define should solve all of the points you raise, excepting the documentation issue.

An example palette which demonstrates these principles:

CreatePalette[
 Column[{Button["Print opener state", 
    MyPalette`Private`DoSomething[
     "The opener is " <> 
      If[CurrentValue[EvaluationNotebook[], {TaggingRules, "opener"}],
        "open", "closed"]]],
   OpenerView[{"Group of buttons", Column[{Button[1], Button[2]}]}, 
    Dynamic[CurrentValue[
      EvaluationNotebook[], {TaggingRules, "opener"}, False]]]}],
 NotebookDynamicExpression :> 
  Refresh[MyPalette`Private`DoSomething[MyPalette`Private`x_] := 
    Print[MyPalette`Private`x], None]]

Let's hit the items raised in this code one by one...

• The palette uses a kernel-defined function which is in NotebookDynamicExpression. The code is wrapped in Refresh[_,None] to ensure that it evaluates once only when the notebook is opened. The code is context isolated by hand. Note that Begin and End won't work here, although they would work inside of a package, or if you wrapped the code in ToExpression (e.g., Begin["foo`"];ToExpression["code"];End[]).
• A palette-wide state variable is stored in the palette's TaggingRules, which can be accessed by using CurrentValue[EvaluationNotebook[],{TaggingRules,"opener"}]. Because "opener" is a string, no symbols are introduced into any context.
• State variables will typically need to be initialized. I could do that in various standard ways, but I used the undocumented third argument to CurrentValue which sets it to False if it doesn't already have a value.
• Once the palette is installed, the TaggingRules setting will persist between instances of the palette, even if you quit Mathematica. Mathematica automatically serializes an installed palette's TaggingRules settings when you close it by storing the value into the global option PalettesMenuSettings.
• If you have multiple versions of the palette open, they'll each operate using independent state variables because the state variable is attached to the palette notebook. If multiple versions of the palette are installed under different names then the PalettesMenuSettings trick will store the TaggingRules separately.

You could generate the palette from code in a separate notebook, and have the generated palette use a unique context by setting CellContext -> Notebook when creating the palette notebook.

I think this should help with items 2, 3, and 5.

---

Example (there may be better ways..)

CreateDocument[
   {Cell[BoxData[MakeBoxes[x = 2]], "Input"]}, 
   CellContext -> Notebook]

If you then look at Context[x] in the created notebook, you get something like Notebook$$21$666892`

This is a supplementary answer to what John Fultz has provided.

Problem 1:

The problem is that sometimes I can't include all functionality inside the palette nor I can call Needs to load it after kernel restart because the package initialization is a little bit complicated.

What I'm fine with though is to be able to close the Palette/Dialog/GUInotebook as soon as the new session starts.

Solution:

Warning - it is based on not stable behavior that NotebookDynamicExpression is loaded only when the notebook is opened, and not when the kernel restarts. As said in comments in accepted answer, this should not be the case. But it is for more than 3 years.

So I will abuse here the fact that NotebookDynamicExpression only fires when the notebook is opened while Initialization every time the session was terminated. So we will check if Initialization is done in the same session and we can close the notebook otherwise.

This notebook will close itself after you close the kernel (which is restarted by dynamics):

CreateDocument[
 DynamicModule[{},
  Dynamic[{"Date :", DateString[]}, UpdateInterval -> 1],
  Initialization :> (
    If[CurrentValue[
       EvaluationNotebook[], {TaggingRules, "Opened"}] =!= $SessionID,
      NotebookClose[]]
    )
  ],
 TaggingRules -> {ParentList, "Opened" -> $SessionID},
 NotebookDynamicExpression :> Refresh[
   CurrentValue[
     EvaluationNotebook[], {TaggingRules, "Opened"}] = $SessionID
   , None]
 ]

Problem 2

I don't like to write full names like: NotebookDynamicExpression :> Refresh[MyPalette`Private`DoSomething[MyPalette`Private`x_] := Print[MyPalette`Private`x], None]]

Method:

You can inject the package to the palette the same way I've done this for a CDF: How can I include functions from a package into a CDF file?