7.7 KiB
Load testing
You may wonder how Gotenberg behaves under load.
In order to help you having an idea, we created a bunch of scenarios for the k6 load testing tool.
The Gotenberg container (version 6.0.0 and default options) was hosted on a AWS EC2 t2.micro instance (1 vCPU and 1 Go of RAM, low to average network performances).
The k6 scenarios have been performed on a MacBook Pro 2016 (2 GHz Intel Core i5 and 16 Go 1867 MHz LPDDR3).
HTML
The HTML scenario is quite simple:
- Ramp up from 0 to 100 virtual users, each one uploading as many time as possible the HTML test data.
- Stop when at least one response is not an HTTP 200 code.
$ k6 run --env MAX_VUS=100 --env BASE_URL=http://ec2-foo.eu-west-1.compute.amazonaws.com html.js
/\ |‾‾| /‾‾/ /‾/
/\ / \ | |_/ / / /
/ \/ \ | | / ‾‾\
/ \ | |‾\ \ | (_) |
/ __________ \ |__| \__\ \___/ .io
execution: local
output: -
script: html.js
duration: -, iterations: -
vus: 1, max: 100
done [==========================================================] 1m17.9s / 10m0s
✗ is status 200
↳ 99% — ✓ 201 / ✗ 2
✗ is not status 504
↳ 99% — ✓ 201 / ✗ 2
✓ is not status 500
checks.....................: 99.34% ✓ 605 ✗ 4
data_received..............: 46 MB 594 kB/s
data_sent..................: 9.6 MB 123 kB/s
✗ failed requests............: 2 0.025646/s
http_req_blocked...........: avg=1.63ms min=2µs med=4µs max=74.77ms p(90)=9.6µs p(95)=19.66ms
http_req_connecting........: avg=1.61ms min=0s med=0s max=74.59ms p(90)=0s p(95)=19.57ms
http_req_duration..........: avg=2.43s min=1.31s med=1.67s max=10.27s p(90)=5.44s p(95)=8.75s
http_req_receiving.........: avg=97.95ms min=103µs med=93.74ms max=202.48ms p(90)=114.51ms p(95)=132.99ms
http_req_sending...........: avg=231.53µs min=92µs med=201µs max=2.33ms p(90)=310.4µs p(95)=333.59µs
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=2.33s min=1.22s med=1.58s max=10.12s p(90)=5.33s p(95)=8.62s
http_reqs..................: 203 2.603019/s
iteration_duration.........: avg=2.43s min=1.31s med=1.68s max=10.27s p(90)=5.45s p(95)=8.75s
iterations.................: 203 2.603019/s
vus........................: 13 min=1 max=13
vus_max....................: 100 min=100 max=100
In our use case, when reaching 13 virtual users (~2,6 requests per second), some incoming requests cannot be fulfilled before 10 seconds (DEFAULT_WAIT_TIMEOUT value).
During this test, CPU usage was high and memory usage went from 339 MiB to a peak of 421 MiB before going back to 364 MiB.
Office
The Office scenario is the same as the HTML scenario, but with a document.docx.
$ k6 run --env MAX_VUS=100 --env BASE_URL=http://ec2-foo.eu-west-1.compute.amazonaws.com office.js
/\ |‾‾| /‾‾/ /‾/
/\ / \ | |_/ / / /
/ \/ \ | | / ‾‾\
/ \ | |‾\ \ | (_) |
/ __________ \ |__| \__\ \___/ .io
execution: local
output: -
script: office.js
duration: -, iterations: -
vus: 1, max: 100
done [==========================================================] 47.9s / 10m0s
✓ is not status 500
✗ is status 200
↳ 83% — ✓ 31 / ✗ 6
✗ is not status 504
↳ 83% — ✓ 31 / ✗ 6
checks.....................: 89.18% ✓ 99 ✗ 12
data_received..............: 2.6 MB 54 kB/s
data_sent..................: 3.4 MB 71 kB/s
✗ failed requests............: 6 0.125047/s
http_req_blocked...........: avg=4.39ms min=3µs med=4µs max=24.52ms p(90)=23.27ms p(95)=23.62ms
http_req_connecting........: avg=4.34ms min=0s med=0s max=24.42ms p(90)=23.15ms p(95)=23.52ms
http_req_duration..........: avg=5.34s min=1.74s med=4.33s max=10.83s p(90)=10.24s p(95)=10.25s
http_req_receiving.........: avg=42.49ms min=67µs med=49.1ms max=68.87ms p(90)=57.34ms p(95)=62.06ms
http_req_sending...........: avg=341.91µs min=215µs med=341µs max=724µs p(90)=443.99µs p(95)=514.79µs
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=5.3s min=1.7s med=4.27s max=10.83s p(90)=10.24s p(95)=10.25s
http_reqs..................: 37 0.771121/s
iteration_duration.........: avg=5.34s min=1.74s med=4.35s max=10.86s p(90)=10.24s p(95)=10.25s
iterations.................: 37 0.771121/s
vus........................: 8 min=1 max=8
vus_max....................: 100 min=100 max=100
In our use case, when reaching 8 virtual users (~0.8 requests per second), some incoming requests cannot be fulfilled before 10 seconds (DEFAULT_WAIT_TIMEOUT value).
During this test, CPU usage was high and memory usage went from 315 MiB to a peak of 788 MiB before going back to 307 MiB.
Merge
The Merge scenario is the same as the previous scenarios, but with a gotenberg.pdf and a gotenberg_bis.pdf.
$ k6 run --env MAX_VUS=100 --env BASE_URL=http://ec2-foo.eu-west-1.compute.amazonaws.com merge.js
/\ |‾‾| /‾‾/ /‾/
/\ / \ | |_/ / / /
/ \/ \ | | / ‾‾\
/ \ | |‾\ \ | (_) |
/ __________ \ |__| \__\ \___/ .io
execution: local
output: -
script: merge.js
duration: -, iterations: -
vus: 1, max: 100
done [==========================================================] 1m45.9s / 10m0s
✗ is status 200
↳ 98% — ✓ 165 / ✗ 3
✗ is not status 504
↳ 98% — ✓ 165 / ✗ 3
✓ is not status 500
checks.....................: 98.80% ✓ 498 ✗ 6
data_received..............: 69 MB 649 kB/s
data_sent..................: 70 MB 661 kB/s
✗ failed requests............: 3 0.028302/s
http_req_blocked...........: avg=2.28ms min=2µs med=5µs max=33.24ms p(90)=23.6µs p(95)=23.38ms
http_req_connecting........: avg=2.24ms min=0s med=0s max=33.1ms p(90)=0s p(95)=23.04ms
http_req_duration..........: avg=5.32s min=632.86ms med=5.24s max=10.17s p(90)=9.42s p(95)=9.91s
http_req_receiving.........: avg=122.07ms min=82µs med=114.72ms max=230.35ms p(90)=162.27ms p(95)=188.95ms
http_req_sending...........: avg=16.56ms min=349µs med=1.37ms max=223.25ms p(90)=12ms p(95)=149.76ms
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=5.18s min=573.03ms med=5.09s max=10.13s p(90)=9.29s p(95)=9.78s
http_reqs..................: 168 1.58493/s
iteration_duration.........: avg=5.33s min=633.87ms med=5.24s max=10.17s p(90)=9.43s p(95)=9.91s
iterations.................: 168 1.58493/s
vus........................: 18 min=1 max=18
vus_max....................: 100 min=100 max=100
In our use case, when reaching 18 virtual users (~1.6 requests per second), some incoming requests cannot be fulfilled before 10 seconds (DEFAULT_WAIT_TIMEOUT value).
During this test, CPU usage was high and memory usage went from 310 MiB to a peak of 604 MiB before going back to 331 MiB.