Performance test of Phoenix(Elixir), Phalcon(PHP7) and Go

Being an avid learner and a tech perfectionist for more than a decade I like to run various performance tests. And although I agree that all tests are subjective and skewed, sometimes there are grains of truth. I'm involved in several big PHP projects(see Homediary.com) on my day-to-day job where we use Phalcon - a clean, super fast framework and I also learn Elixir and Go. PHP got a serious performance boost with the recent release of PHP 7. Hearing only awesome stories about Phoenix/Elixir performance and with Go on the rise I decided to look for any existing projects measuring the performance of the above mentioned web frameworks.

Luckily, there was PETE, which stands for PErformance TEst of different web frameworks: phoenix (elixir), slim (PHP), phalcon (PHP). It is a simple single page web app showing a list of images. Nevertheless it consists of a controller, a view and a template: the View/Controller part of the MVC stack. That's exactly what can be evaluated from the performance point of view. The project didn't have a Go version of the app so I had to code one. I have submitted a pull request to have it included in the original repo.

I did my own test on three m4.xlarge AWS EC2 instances: one instance was used for wrk, another instance was used for nginx/Php-fpm/phalcon setup and the last instance was used for Elixir/Erlang stack. I reused the PHP instance to test Go version of the app by stopping nginx and php-fpm and launching the Go app.

I repeated every wrk launch several times to ensure consistency in numbers. The results are quite surprising. I expected to see a tight fight between Phoenix/Elixir and Go/stdlib but instead the Go version has been surpassed by Phalcon/PHP! In fact, the Go version of the gallery app is the slowest among the three.

Phalcon 3.0.2 with PHP 7
wrk -t4 -c100 -d60s --timeout 2000 http://35.162.148.251/gallery
Running 1m test @ http://35.162.148.251/gallery
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.00ms    5.40ms 227.06ms   90.31%
    Req/Sec     1.82k    58.01     2.06k    74.92%
  435522 requests in 1.00m, 772.52MB read
Requests/sec:   7255.99
Transfer/sec:     12.87MB
Phoenix 1.2.0 with Erlang/OTP 19
wrk -t4 -c100 -d60s --timeout 2000 http://35.160.152.71/gallery
Running 1m test @ http://35.160.152.71/gallery
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.80ms    4.30ms 228.66ms   74.56%
    Req/Sec     4.56k   546.57     9.26k    74.86%
  1091298 requests in 1.00m, 2.05GB read
Requests/sec:  18158.02
Transfer/sec:     34.95MB
Go 1.7.3
wrk -t4 -c100 -d60s --timeout 2000 http://35.162.148.251/gallery
Running 1m test @ http://35.162.148.251/gallery
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    19.65ms   19.01ms 223.41ms   85.76%
    Req/Sec     1.57k   144.09     2.13k    68.25%
  375624 requests in 1.00m, 651.61MB read
Requests/sec:   6256.53
Transfer/sec:     10.85MB

It is easy to see that Phalcon serves 7255 requests/sec but that pales in comparison with Phoenix's 18158 requests/sec (served with only 5.8ms avg latency!!). The Go version Go is the slowest with only 6256 requests/sec despite Go being a compiled language.

Update

Performance of the Go version can be improved by almost 5x by moving template parsing outside of the handler function so that templates are only parsed once during app initialization. I'm not 100% sure but I think that's how other frameworks deal with templates too, so that should be fair comparison. The updated results for Go look like this:

wrk -t4 -c100 -d60s --timeout 2000 http://35.162.251.83/gallery
Running 1m test @ http://35.162.251.83/gallery
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.49ms    3.66ms 218.68ms   84.18%
    Req/Sec     5.93k   407.07    23.34k    95.59%
  1416365 requests in 1.00m, 2.40GB read
Requests/sec:  23567.17
Transfer/sec:     40.88MB

This makes Go/stdlib the winner, Elixir/Phoenix the runner up (23% fewer requests than Go) and PHP/Phalcon taking the last place.

Update 2

My smart readers told me net/http can't do connection pooling and is not a very fast HTTP server in general and advised to use valyala/fasthttp. Well, the Go version of the app with fasthttp has just blown my socks off with a whopping 28816 requests/sec, a 22% increase!

wrk -t4 -c100 -d60s --timeout 2000 http://35.163.173.123/gallery
Running 1m test @ http://35.163.173.123/gallery
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.74ms    3.43ms 211.54ms   90.18%
    Req/Sec     7.24k   237.32     7.90k    68.79%
  1729095 requests in 1.00m, 2.96GB read
Requests/sec:  28816.34
Transfer/sec:     50.46MB

Andrew Katasonov

Read more posts by this author.