JSON Representation of Map with Complex Key

I would do this. The parts key of the top level object would be a JSONArray of JSONObject that have key's and value's. The key would be an object that is your PartDescriptor and the value would be your Part.

{
    "name":"theName",
    "parts":[
        {
            "key":{
                       "group":"theGroup",
                       "id":"theId"
                  },
            "value":{
                       "group":"theGroup",
                       "id":"theId",
                       "description":"theDescription",
                       "compat":"theCompat",
                       ...
                    }
        },
        ...
    ]
}

You don't need annotations or custom serializers. Assuming you already have getters for all the fields in Part and Machine, all that's really missing is a toString() on PartDescriptor. If, for some reason, you don't have getter functions, you'll need to annotate the fields of interest with @JsonProperty so Jackson knows which fields to include in the serialized output. However, it's preferable (and easier) to simply create getters.

The toString() on PartDescriptor should return the key you want to use in your mapping. As another answer suggests, you might simply concatenate the relevant fields:

@Override
public String toString() {
    return group + "|" + id;
}

Then you'll magically get this form when you attempt to serialize a Machine with Jackson's ObjectMapper:

{
  "name" : "Toaster",
  "parts" : {
    "Electrical|Descriptor1" : {
      "group" : "Electrical",
      "id" : "Part1",
      "description" : "Heating Element",
      "compat" : "B293"
    },
    "Exterior|Descriptor2" : {
      "group" : "Exterior",
      "id" : "Part2",
      "description" : "Lever",
      "compat" : "18A"
    }
  }
}

I'd do something like:

{
  "name": "machine name",
  "parts": [
     { "group": "part group", "id": "part id", "description": "...", ... },
     { "group": "part group", "id": "part id", "description": "...", ... },
     // ...
  ]
}

If the "id" for each Part is unique, then the "parts" property can be an object instead of an array, with the "id" of each part serving as the key.

{
  "name": "machine name",
  "parts": {
     "1st part id": { "group": "part group", "description": "...", ... },
     "2nd part id": { "group": "part group", "description": "...", ... },
     // ...
  }
}