Adding CSS and JS to <head> using a module layout.xml for block

First, your layout file should be placed in app/design/frontend/{interface}/{theme}/layout/.
Second. If you want to add the css and js files only in the grouped and simple product pages don't use the <default> layout handle.
Make your layout look like this:

<?xml version="1.0"?>
<layout>
    <my_handle><!-- declare a custom handle so you won't duplicate the code -->
        <reference name="head">
            <action method="addJs">
                <script>vendor/qtip/jquery.qtip.min.js</script>
            </action>
            <action method="addCss">
                <stylesheet>vendor/qtip/jquery.qtip.min.css</stylesheet>
            </action>
        </reference>
    </my_handle>
    <PRODUCT_TYPE_simple><!-- layout handle for simple products -->
        <update handle="my_handle" /> <!-- include the handle you declared above -->
    </PRODUCT_TYPE_simple>
    <PRODUCT_TYPE_grouped><!-- layout handle for grouped products -->
        <update handle="my_handle" /> <!-- include the handle you declared above -->
    </PRODUCT_TYPE_grouped>
</layout>

As this has not been pointed out clearly enough yet:

Directories

Layout XML files for a custom module should be theme independent and thus placed into

app/design/frontend/base/default/layout

You can override any of these XML files in your custom theme, but in most cases, it is better to have an additional theme specific layout file that adds changes. More info: What is the correct way/approach to modify a Magento template?

File names

Note that those files are sometimes referred to as "layout.xml files", but layout.xml is actually not a file name you would usually find in any Magento code. Your code works if you put your layout.xml file in the right location (see above), but convention is to use the lower case module name:

rating.xml

or better

vendor_rating.xml

Remember that the layout XML files from all modules are in one single directory, so the name must be unique!