AWS elastic-search. FORBIDDEN/8/index write (api). Unable to write to index

This error is the Amazon ES service actively blocking writes to protect the cluster from reaching red or yellow status. It does this using index.blocks.write.

The two reasons being:

Low Memory

When the JVMMemoryPressure metric exceeds 92% for 30 minutes, Amazon ES triggers a protection mechanism and blocks all write operations to prevent the cluster from reaching red status. When the protection is on, write operations fail with a ClusterBlockException error, new indexes can't be created, and the IndexCreateBlockException error is thrown.

When the JVMMemoryPressure metric returns to 88% or lower for five minutes, the protection is disabled, and write operations to the cluster are unblocked.

Low Disk Space

Elasticsearch has a default "low watermark" of 85%, meaning that once disk usage exceeds 85%, Elasticsearch no longer allocates shards to that node. Elasticsearch also has a default "high watermark" of 90%, at which point it attempts to relocate shards to other nodes.

ES could apply write block on index during rollovers, or Low disk space or memory.

In order to stop these errors you need to remove the write block on the index by setting index.blocks.write to false

curl -X PUT -H "Content-Type: application/json" \
'http://localhost:9200/{index_name}/_settings' \
  -d '{ "index": { "blocks": { "write": "false" } } }'

This error indicates that AWS ElasticSearch has placed a block on your domain based upon disk space. At 85%, ES will not allow you create any new indexes. At 90%, no new documents can be written.