How to access map or object elements dynamically in Lightning?

Ok, I think I have got the closest solution so far.

What you can do is create a custom value provider, which, whilst it means you need to define some Javascript to process a Map, you only need to do this once and can pass any Map to it to iterate.

It's not the same as the old Apex version, but it works and is probably more customizable (for example, you could output both keys and values together)

Component code:

<aura:application extends="force:slds" >
  <aura:attribute name="providersAdded" default="false" type="Boolean" />
  <aura:handler name="init" value="{!this}" action="{!c.init}" />
  <aura:attribute name="mymap" type="Map" default="{'name':'France','capital':'Paris'}"/>    

  <aura:if isTrue="{!v.providersAdded}">
  <!--Note the use of the special value provider that allows Map iteration-->
  <aura:iteration items="{!i.mymap}" var="value">
      <div >{!value}</div>
      <!--France-->
      <!--Paris-->
  </aura:iteration>
  </aura:if>
</aura:application>

Controller.js:

init: function(component, event, helper) {
  component.addValueProvider(
      'i',
      {
          get: function(key, comp) {
              let localMap = component.get("v."+key); 
              return Object.keys(localMap).map(key => localMap[key]);
          },
      }
  );
}