How to navigate in lightning out?

I ran into the same issue. I have Lightning Components I want to use in VF pages AND in Lightning apps running in the Lightning Experience. I also didn't want to update the components to check to see which environment they were running in (no isSalesforce1, isVF, isLightning checks in any components). Ideally, the components raise the navigation events, and they work regardless of whether the host is the one.app container or a VF page.

After some digging, I got this working. The first step is to get the navigation events firing properly in the components. The code below assigns null to the event variable when running in a VF page because the force namespace library hasn't been loaded. The result is an error in the browser.

var event = $A.get('e.force:navigateToObjectHome');

event.setParams({
    scope: 'Contact'
});

event.fire();

Fortunately, the fix is straightforward. Add the dependency to your Lightning dependency app.

<aura:application access="GLOBAL" extends="ltng:outApp">
    <aura:dependency resource="c:myLightningComponent"/>
    <!-- Load the navigation events in the force namespace. -->
    <aura:dependency resource="markup://force:*" type="EVENT"/>
</aura:application>

With this change, the navigation events will fire properly in your components; however there's still a problem. Nothing happens. This brings us to the second step which is to add handlers for these events in the VF host page. These handlers will explicitly perform navigation by either calling the appropriate sforce.one method or setting window.location. You can set up the handlers in the VF page after the component is created. For example, you can handle the force:navigateToObjectHome event with the following code:

$Lightning.use("c:myLightningComponentDep", function() {
    var attributes = {}; // Set component attributes here

    $Lightning.createComponent('c:myLightningComponent', attributes, 'myLightningComponent',
        function(cmp) {
            $A.eventService.addHandler({
                event: 'force:navigateToObjectHome',
                handler: function(event) {
                    if (sforce && sforce.one) {
                        // VF page in S1 or Lightning Exp
                        sforce.one.navigateToObjectHome(event.$params$.scope);
                    } else {
                        // VF page in Classic
                        // Ugg! We need an if branch for each possible object type.
                        // Is there a better way of doing this?
                        if (event.$params$.scope === 'Contact') {
                            window.location = '{!URLFOR($Action.Contact.Tab, $ObjectType.Contact)}';
                        }
                    }
                }
            });
    });
});

This approach has worked well for me, and it has the added benefit of requiring no changes to the Lightning components. I haven't gotten this working in Salesforce1 yet as my organization doesn't use Salesforce1, although I assume the approach is similar.

Here is what I use on my page/template to control navigation based on the theme of the current context.

    <script>
        function goHome(){
            ForceUI.isSalesforce1() ? sforce.one.navigateToURL('/home/home.jsp',true) : window.location.href='/';
        }

        function goretURL(){
            ForceUI.isSalesforce1() ? sforce.one.navigateToURL('{!$CurrentPage.parameters.retURL}') : window.location.href='{!$CurrentPage.parameters.retURL}';
        }

        function gotoURL(u, isParent){ //isParent indicates the parent window should be redirected (think inline vf page)
            ForceUI.isSalesforce1() ? sforce.one.navigateToURL(u) : (isParent ? window.top.location.href = u : window.location.href=u);
        }


        (function(myContext){
            myContext.ForceUI = myContext.ForceUI || {};

            myContext.ForceUI.isSalesforce1 = function() {
                return((typeof sforce != 'undefined') && sforce && (!!sforce.one));
            }
        })(this);

    </script>

The last part of the code was used directly from here: https://developer.salesforce.com/docs/atlas.en-us.salesforce1.meta/salesforce1/vf_dev_best_practices_pages_multipurpose.htm

Not sure how applicable this will be to you since we do not see your page / component etc but if it can be used great.