How do I set up a kind of "belongs_to :through" association without a direct belongs_to?

Use delegate in the model class.

class Team < ActiveRecord::Base
  belongs_to :division
  has_many :players

  delegate :league, to: :division
end

Reference: http://api.rubyonrails.org/classes/Module.html#method-i-delegate

You may try

    class Player
      belongs_to :team
      has_one :division, :through => :team
    end

You can define a helper method in your player model:

def division
  team.division
end

def league
  team.division.league
end

Of course, this only relates to readability of your code and does not affect the form of the database queries involved. If your statement generates multiple SQL queries but you want only one, check out the .include option here: Rails Guides - Active Record Query Interface