Posts

Showing posts from August, 2019

Faster tests of logstash configuration

Image
If you're using logstash in all but the simplest setups, you have probably started testing your logstash config along the way. This means writing ruby rspec tests like the following: we used to do the same, but over time we noticed that the test suite would take more and more time to to run. A quick investigation revealed that a lot of time was spent in the startup and shutdown of logstash itself, since Logstash would be started and stopped for each  sample  block. But what caused logstash to take such a long time to start and stop? Running rspec with the TEST_DEBUG=1 environment variable set revealed substantial logging related to starting the inputs and output plugins. The elasticsearch output in particular would complain a lot about not being able to connect to ES, with retries at regular intervals. I realized then that since we don't actually need the output and input plugins perhaps we can skip starting them. With a quick change to the configuration loading bl...