How to create grouped/nested properties?

Try replacing your nested QtObject with a QML file. For example, I replaced it with BackgroundTheme.qml. This way, the property (which can correcly be called "grouped property") works correctly, in a binding and without any error.

BackgroundTheme.qml

import QtQuick 2.0

QtObject {
  property color pressed: "#CCCCCC"
  property color enabled: "#666666"
  property color disabled: "#555555"
}

MyButtonStyling.qml

import QtQuick 2.0

QtObject {       
    property BackgroundTheme background: BackgroundTheme {}
}