Get the number of fields on an index

To build a bit further upon what the other answer provided, you can get the mapping and then simply count the number of times the keyword type appears in the output, which gives the number of fields since each field needs a type:

curl -s -XGET localhost:9200/index/_mapping?pretty | grep type | wc -l

You can try this:

curl -s -XGET "http://localhost:9200/index/_field_caps?fields=*" | jq '.fields|length'

Just a quick way to get a relative estimate in Kibana without writing a script (I don't believe this is 100% precise, but it's an easy way to tell if your dynamic fields are blowing up to huge numbers for some reason).

Run this query in the Kibana dev tools

GET /index_name/_mapping

Inside the Kibana output, perform a search for all instances of "type" (including quotes). This will count the instances and get you the answer. (In this example, 804)

enter image description here

This can be helpful if you scratching your head as to why you're getting the [remote_transport_exception] error of

Limit of total fields [1000] in index [index_name] has been exceeded


The first answer by Val solves the problem for me too. But I just wanted to list out some corner cases which can lead to misleading numbers.

  1. The document has fields with the word "type" in them.

For example

 "content_type" : {
   "type" : "text",
     "fields" : {
       "keyword" : {
          "type" : "keyword",
       }
     }
   },

This will match grep type thrice while it should do that only twice i.e. it should not match "content_type". This scenario has an easy fix.

Instead of

curl -s -XGET localhost:9200/index/_mapping?pretty | grep type 

Use

curl -s -XGET localhost:9200/index/_mapping?pretty | grep '"type"'

to get an exact match of '"type"'

  1. The document has a field of the exact name "type"

For example

"type" : {
  "type" : "text",
   "fields" : {
     "keyword" : {
       "type" : "keyword"
     }
   }
},

In this case also the match is thrice instead of twice. But using

curl -s -XGET localhost:9200/index/_mapping?pretty | grep '"type"'

is not going to cut it. We will have to skip fields with the "type" keyword as substring as well as an exact match. In this case we can add an additional filter like so:

curl -s -XGET localhost:9200/index/_mapping?pretty |\
grep '"type"' | grep -v "{"

In addition to the above 2 scenarios, if you are using the api programmatically to push numbers for tracking i.e. into something like AWS cloudwatch or Graphite, you can use the following code to call the API - get the data, and recursively search for the keyword "type" - while skipping any fuzzy matches and resolving deeper into fields with the exact name "type".

import sys
import json
import requests

# The following find function is a minor edit of the function posted here
# https://stackoverflow.com/questions/9807634/find-all-occurrences-of-a-key-in-nested-python-dictionaries-and-lists

def find(key, value):
  for k, v in value.iteritems():
    if k == key and not isinstance(v, dict) and not isinstance(v, list):
      yield v
    elif isinstance(v, dict):
      for result in find(key, v):
        yield result
    elif isinstance(v, list):
      for d in v:
        for result in find(key, d):
          yield result

def get_index_type_count(es_host):
  try:
    response = requests.get('https://%s/_mapping/' % es_host)
  except Exception as ex:
    print('Failed to get response - %s' % ex)
    sys.exit(1)

  indices_mapping_data = response.json()
  output = {}

  for index, mapping_data in indices_mapping_data.iteritems():
    output[index] = len(list(find('type', mapping_data)))

  return output

if __name__ == '__main__':
  print json.dumps(get_index_type_count(sys.argv[1]), indent=2)

The above code is also posted as a gist here - https://gist.github.com/saurabh-hirani/e8cbc96844307a41ff4bc8aa8ebd7459