Faster tests of logstash configuration
If you're using logstash in all but the simplest setups, you have probably started testing your logstash config along the way. This means writing ruby rspec tests like the following:
we used to do the same, but over time we noticed that the test suite would take more and more time to to run. A quick investigation revealed that a lot of time was spent in the startup and shutdown of logstash itself, since Logstash would be started and stopped for each sample block.
But what caused logstash to take such a long time to start and stop?
Running rspec with the TEST_DEBUG=1 environment variable set revealed substantial logging related to starting the inputs and output plugins. The elasticsearch output in particular would complain a lot about not being able to connect to ES, with retries at regular intervals.
I realized then that since we don't actually need the output and input plugins perhaps we can skip starting them. With a quick change to the configuration loading block:
run times for the test suite were cut to 25% of their original time.
To make sure we don't accidentally introduce errors in the input and output configs we still run one simple suite with input and output both enabled by calling the function with the all argument set to true (default is false).
we used to do the same, but over time we noticed that the test suite would take more and more time to to run. A quick investigation revealed that a lot of time was spent in the startup and shutdown of logstash itself, since Logstash would be started and stopped for each sample block.
But what caused logstash to take such a long time to start and stop?
Running rspec with the TEST_DEBUG=1 environment variable set revealed substantial logging related to starting the inputs and output plugins. The elasticsearch output in particular would complain a lot about not being able to connect to ES, with retries at regular intervals.
I realized then that since we don't actually need the output and input plugins perhaps we can skip starting them. With a quick change to the configuration loading block:
run times for the test suite were cut to 25% of their original time.
To make sure we don't accidentally introduce errors in the input and output configs we still run one simple suite with input and output both enabled by calling the function with the all argument set to true (default is false).