How do I calculate needed pixel clock frequency?

There is more pixels in the video signal than just the x * y * refresh_rate would imply. Back in the bad old days of CRT's, it would take time for the electron beam to move from the end of the current line to beginning of the next. Likewise, it would take time for the beam to move from the bottom of th screen to the top for the next frame. The time it took was built into the video signaling standard, and essentially took the form of pixels that are never seen.

Without knowing the info on these blank pixels (sometimes called "overscan"), you cannot calculate the pixel clock required.

The best way to figure this stuff out is to Google for the approximate video standard you're trying to do, or look at the datasheet for the screen you are trying to drive.

This depends also on what kind of display you want to drive. Usually the standard equation for pixel clock is this-

Pixel clock = Horizontal_Active_Resolution X Vertical_Active_Resolution X Refresh_Rate X (1+ Blanking Period %).

Horizontal_Active_Resolution X Vertical_Active_Resolution X Refresh_Rate is the display resolution which in your case is 1280 X 1024 X 85.

Now since you want to calculate exact pixel clock required for this display, you must know the %blanking period. Usually it lies between 3% to 40% for most of the display.

The % blanking period is total time in which there is no active image being presented on the screen as suggested above in these answers also. The image below shows in detail regarding display size and blanking period which consists for both horizontal and vertical fields. In order to calculate the exact pixel clock you must know these details-

So effectively your pixel clock in this figure will be (HPW+HBP+HACT+HFP) X (VPW+VBP+VACT+VFP) X Frame Rate.

The Blanking period will consist of horizontally as horizontal pulse width (HPW), horizontal back porch (HBP) and horizontal front porch (HFP). It will also consist of vertically as vertical pulse width (VPW), vertical back porch (VBP) and vertical front porch (VFP).

All these details can be found in Display Datasheet.