Trouble with fonts when exporting a 2D-plot to vector formats

Mathematica uses two differen sets of styles for displaying graphics on screen and for exporting: ScreenStyleEnvironment for displaying and exporting in raster formats and PrintingStyleEnvironment for printing and exporting to EPS and PDF formats (see here for more details). One way to achieve consistent export to PostScript is to set PrintingStyleEnvironment the same as ScreenStyleEnvironment:

SetOptions[$FrontEnd, PrintingStyleEnvironment -> "Working"]

See also dedicated pitfalls answer.

Not knowing in what other external application you plan to process the exported PDF, I'm focusing mainly on the last paragraph of the question which asks for a way to outline fonts. To do this, you can simply define the function

outlinedExport[name_, gr_, opts : OptionsPattern[]] := 
 Export[name, 
  First@ImportString[ExportString[gr, "PDF"], "PDF", 
    "TextOutlines" -> True], FilterRules[{opts}, Options[Export]]]

Assuming your graphics object is g, you would do something like this:

outlinedExport["output.pdf", g, ImageSize -> 600]

This also works for other file formats, and accepts options such as ImageSize.

Edit

One additional caveat to watch out for: if your plot has a frame or axes that turn out to look too thin in the exported vector graphics, try adding options to your plot that explicitly fix the thickness, such as

AxesStyle->Thickness[.002]

or (if you have set Frame -> True:

FrameStyle -> Thickness[.002]

I've arrived at something that works for my scenario, though it's a bit hacky.

1. install the Mathematica Fonts on your system.
   
   On OS X they're in Mathematica.app/SystemFiles/Fonts/TrueType/MathematicaFonts.dfont
   copy this file to ~/Library/Fonts
   
   (To get inside the Mathematica.app bundle: right-click -> show package contents,
   to get to your Library, use "Go -> Go to Folder" in the Finder)
2. export plot as SVG
3. do a search & replace in the SVG (SVG is text after all), replacing 'Times' and 'SVGTimes' with 'Mathematica1'.
4. (optional) outline all strokes in your vector graphics application