Iguanas implemented Stresstest benchmark task tries to emulate a real case scenario under which an endpoint or application is under high stress. As in real life endpoints might get multiple simultaneous request within seconds, it is very important to verify that you application can handle this.
The stresstest emulates users or applications which will bombard the endpoint using a set of queries for a specific amount of time or a specific amount of queries executed. Each simulated user is called Worker in the following. As you might want to test read and write performance or just want to emulate different user behaviour, the stresstest allows to configure several workers. Every worker configuration can additionaly be started several times, hence if you want one configuration executed multiple times, you can simply tell Iguana to run this worker configuration the specified amount of time. However to assure that the endpoint can't just cache the repsonse of the first request of a query, every worker starts at a pre determined random query, meaning that the single worker will always start at that query to assure fairness in benchmark comparisons, while every worker will start at a different query.
To configure this task you have to first tell Iguana to use the implemented task like the following:
tasks: - className: "Stresstest"
Further on you need to configure the Stresstest using the configuration parameter like:
tasks: - className: "Stresstest" configuration: timeLimit: 600000 ...
As an end restriction you can either use
timeLimit which will stop the stresstest after the specified amount in ms or you can set
noOfQueryMixes which stops every worker after they executed the amount of queries in the provided query set.
Additionaly to either
noOfQueryMixes you can set the following parameters
- warmup (optional)
The queryHandler parameter let's the stresstest know what queries will be used.
Normally you will need the
InstancesQueryHandler which will use plain text queries (could be SQL, SPARQL, a whole RDF document). The only restriction is that each query has to be in one line.
You can set the query handler like the following:
tasks: - className: "Stresstest" queryHandler: className: "InstancesQueryHandler" ...
To see which query handlers are supported see Supported Queries
Workers (simulated Users)
Further on you have to add which workers to use. As described above you can set different worker configurations. Let's look at an example:
- className: "Stresstest" timeLimit: 600000 workers: - threads: 4 className: "SPARQLWorker" queriesFile: "/path/to/your/queries.txt" - threads: 16 className: "SPARQLWorker" queriesFile: "/other/queries.txt" fixedLatency: 5000
In this example we have two different worker configurations we want to use. The first want will create 4
SPARQLWorkers using queries at
/path/to/your/queries.txt with any latencym thus every query will be executed immediatly after another.
The second worker configuration will execute 16
SPARQLWorkers using queries at
/other/queries.txt using a fixed waiting time of
5000ms between each query.
Hence every worker will execute their queries independently from each other but will wait 5s after each of their query execution before executing the next one.
This configuration may simulate that we have a few Users requesting your endpoint locally (e.g. some of your application relying on your database) and several users querying your endpoint from outside the network where we would have network latency and other interferences which we will try to simulate with 5s.
A full list of supported workers and their parameters can be found at Supported Workers
In this example our Stresstest would create 20 workers, which will simultaenously request the endpoint for 60000ms (10 minutes).
Additionaly to these you can optionally set a warmup, which will aim to let the system be benchmarked under a normal situation (Some times a database is faster when it was already running for a bit)
The configuration is similar to the stresstest itself you can set a
timeLimit (however not a certain no of query executions), you can set different
workers, and a
queryHandler to use.
If you don't set the
queryHandler parameter the warmup will simply use the
queryHandler specified in the Stresstest itself.
You can set the Warmup as following:
tasks: - className: "Stresstest" warmup: timeLimit: 600000 workers: ... queryHandler: ...
That's it. A full example might look like this
tasks: - className: "Stresstest" configuration: # 1 hour (time Limit is in ms) timeLimit: 3600000 # warmup is optional warmup: # 10 minutes (is in ms) timeLimit: 600000 # queryHandler could be set too, same as in the stresstest configuration, otherwise the same queryHandler will be use. # workers are set the same way as in the configuration part workers: - threads: 1 className: "SPARQLWorker" queriesFile: "queries_warmup.txt" timeOut: 180000 queryHandler: className: "InstancesQueryHandler" workers: - threads: 16 className: "SPARQLWorker" queriesFile: "queries_easy.txt" timeOut: 180000 - threads: 4 className: "SPARQLWorker" queriesFile: "queries_complex.txt" fixedLatency: 100