Elasticsearch Bulk Index JSON Data
As of today, 6.1.2 is the latest version of ElasticSearch, and the curl command that works for me on Windows (x64) is
curl -s -XPOST localhost:9200/my_index/my_index_type/_bulk -H "Content-Type:
application/x-ndjson" --data-binary @D:\data\mydata.json
The format of the data that should be present in mydata.json remains the same as shown in @val's answer
What you need to do is to read that JSON file and then build a bulk request with the format expected by the _bulk
endpoint, i.e. one line for the command and one line for the document, separated by a newline character... rinse and repeat for each document:
curl -XPOST localhost:9200/your_index/_bulk -d '
{"index": {"_index": "your_index", "_type": "your_type", "_id": "975463711"}}
{"Amount": "480", "Quantity": "2", "Id": "975463711", "Client_Store_sk": "1109"}
{"index": {"_index": "your_index", "_type": "your_type", "_id": "975463943"}}
{"Amount": "2105", "Quantity": "2", "Id": "975463943", "Client_Store_sk": "1109"}
... etc for all your documents
'
Just make sure to replace your_index
and your_type
with the actual index and type names you're using.
UPDATE
Note that the command-line can be shortened, by removing _index
and _type
if those are specified in your URL. It is also possible to remove _id
if you specify the path to your id field in your mapping (note that this feature will be deprecated in ES 2.0, though). At the very least, your command line can look like {"index":{}}
for all documents but it will always be mandatory in order to specify which kind of operation you want to perform (in this case index
the document)
UPDATE 2
curl -XPOST localhost:9200/index_local/my_doc_type/_bulk --data-binary @/home/data1.json
/home/data1.json
should look like this:
{"index":{}}
{"Amount": "480", "Quantity": "2", "Id": "975463711", "Client_Store_sk": "1109"}
{"index":{}}
{"Amount": "2105", "Quantity": "2", "Id": "975463943", "Client_Store_sk": "1109"}
{"index":{}}
{"Amount": "2107", "Quantity": "3", "Id": "974920111", "Client_Store_sk": "1109"}
UPDATE 3
You can refer to this answer to see how to generate the new json style file mentioned in UPDATE 2.
UPDATE 4
As of ES 7.x, the doc_type
is not necessary anymore and should simply be _doc
instead of my_doc_type
. As of ES 8.x, the doc type will be removed completely. You can read more about this here