Friday, February 06, 2015

Auto-upload Elastisearch template mapping with Apache Camel

When feeding data into Elastisearch, one important step is to configure the correct template for the index/type so that, for instance, numeric fields are stored as numbers to ensure that they can be sorted by and/or confronted correctly.

The Elasticsearch Logstash plugin has a handy option just for this purpose. If you are not using Logstash you have to do it yourself, eithr through configuration mgmt, startup scripts or simply manaully launching the appropriate curl command.

If you have followed my previous post on using Apache Camel to feed sql data into Elasticsearch then it might come natural to attempt to use Camel also for the purpose of uploading the template mapping.
How hard can it be? Turns out it's prety simple, so let me present you with the solution right away and leave the nitty gritty details for later:


This route will run exactly only once at Camel startup and then fetch the file elmapping.json and PUT it into elastisearch. A sprinkle of groovy makes populating the body of the request a piece of cake (the route requires the camel-groovy and camel-script components).

I have then added an initialDelay to the other routes to allow enough time for elasticsearch to process and acknowledge the mapping.

Happy hacking!

No comments: