Swagger: How to have a property reference a model in OpenAPI 2.0 (i.e. nest the models)?

The correct way to model it in OpenAPI 2.0 would be:

swagger: '2.0'
...

definitions:
  SomeModel:
    type: object
    properties:
      prop1:
        type: string
      prop2:
        type: integer
      prop3:
        $ref: '#/definitions/OtherModel'   # <-----

  OtherModel:
    type: object
    properties:
      otherProp:
        type: string

If you use OpenAPI 3.0, models live in components/schemas instead of definitions:

openapi: 3.0.1
...

components:
  schemas:
    SomeModel:
      type: object
      properties:
        prop1:
          type: string
        prop2:
          type: integer
        prop3:
          $ref: '#/components/schemas/OtherModel'   # <-----

    OtherModel:
      type: object
      properties:
        otherProp:
          type: string

Remember to add type: object to your object schemas because the type is not inferred from other keywords.