How to change the default open file dialog path

There are a bunch of file locations in the registry at HKCU\Software\Microsoft\VisualStudio\10.0. The one you need to change is DefaultFileOpenLocation.

Yes in Tools->Options->Projects and Solutions->General

For Visual Studio 2017 and 2019 you'll have to navigate from Tools->Options (Alt+T, O) then Projects and Solutions->Locations, there are 3 Locations but you will only need to edit the Projects location and click OK. No need to restart VS.