Tuning fpm-php: Not for the faint of heart

Tuning fpm-php: Not for the faint of heart

Another tech Sunday – they never get old 🙂 This is a somewhat long and boring post. More of a tech journal entry. But who knows, maybe someone else will find it interesting.

First let’s backtrack a little – what is CGI you ask?

source: https://www.irt.org/articles/js184/

An acronym for Common Gateway Interface, CGI was the first implementation of a server-side ‘gateway’ where http could interact with external applications such as databases. Although CGI is language agnostic, typically PERL scripts were used.

CGI is rarely used today due to inherent security vulnerabilities.

Then there’s fastcgi

And then there’s php-fpm (FastCGI process manager)

PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites.

These features include:

  • Adaptive process spawning (NEW!)
  • Basic statistics (ala Apache’s mod_status) (NEW!)
  • Advanced process management with graceful stop/start
  • Ability to start workers with different uid/gid/chroot/environment and different php.ini (replaces safe_mode)
  • Stdout & stderr logging
  • Emergency restart in case of accidental opcode cache destruction
  • Accelerated upload support
  • Support for a “slowlog”
  • Enhancements to FastCGI, such as fastcgi_finish_request() – a special function to finish request & flush all data while continuing to do something time-consuming (video converting, stats processing, etc.)

Here’s how h2o implements fcgi, via a unix socket

If you want to check info on unix sockets use this:

The cgi-fcgi executable

Looks like a good article here, but I chose not to go down this path.

Composer and php-fpm-status-cli

How do you get info on what’s going on under the hood with fpm?

Turns out there’s a status page for that!

You need to make some changes to the /etc/php/7.0/fpm/pool.d/www.conf file and uncomment the following line:

pm.status_path = /status

While it’s possible to output the php status page to the browser this requires additional work and, quite frankly, I just don’t see the point. As I’m also running the h2o webserver, there’s little documentation available online as to how to acheive this – unlike Apache and Nginx.

I stumbled across php-fpm-status-cli which will suit my needs nicely. To get it up and running I need to install Composer, a dependency manager for PHP.

To get Composer up and running the following did the trick:

2. Then install -fpm-status-cli

3. Finally, a bash script to make running it easier:

After installation I ran the script. Oops, an error, but one which is easily fixed.

The correct socket for me was php7.0-fpm.sock. That fixed, the script outputs the php status page as follows:

Finally, I’m able to get some really useful output using the command

Wow! That’s a lot of instances of admin-ajax.php. Too many, me thinks.

After some Googling I decide to install the Heartbeat Control wordpress plugin.

Although I still have a high number of admin-ajax.php scripts running, I am no longer seeing errors in the php-fpm log file.

Checking server connections

Here’s a neat trick to check the number of connections to your server:

Want to know what they mean? Here you go..

Tuning the www.conf process manager parameters

Why the need for tuning? Well, for me it all started with a high CPU utilization alert which popped up in Slack.

So i delved into the php-fpm log file and found the following:

As you can see, at the time my max.children was set to 5 which is way to low for my production site.

There is a lot of opinion surrounding the best way to run php-fpm from a performance perspective.

There are three ‘modes’ being 1. dynamic, 2. static and 3. ondemand.

The mode is set using the pm= parameter within www.conf

Here’s a good article about ondemand vs dynamic.

If you’re working on a high performance PHP setup, the ‘ondemand’ PM may not be for you. In that case, it’s wise to pre-fork your PHP-FPM processes up to the maximum your server can handle. That way, all your processes are ready to serve your requests without needing to be spawned first. However, for 90% of the sites out there, the ondemand PHP-FPM configuration is better than either static or dynamic.

If you’re after info on why the static mode may suit you, check this article out.

This means PHP-FPM is always set to the max capacity of your server’s resources regardless of current traffic. Idle process stay online waiting for traffic spikes and responding immediately, rather than having to wait on the pm to spawn children and then kill them off after x pm.process_idle_timeout expires.

It’s a good idea to review the contents of the fpm log file. Mine is /var/log/php7.0-fpm.log

Here’s the best description I found of the parameters which you will need to modify when performance tuning the php-fpm stack:

Debugging slow processes

The author of this page makes a point I have not seen expressed elsewhere – namely the need to understand which scripts are causing the bottleneck.

Raising number of child processes won’t help, you have 16 cores and 20 processes. This means that your OS will be forced to schedule processes and raising the number won’t make anything faster – you don’t even know if you are CPU or I/O bound.

To correctly solve this problem, you need to determine why PHP can’t keep up.

You can enable PHP-FPM slow_log feature by adding this to your pool:


Other references:
https://deliciousbrains.com/sql-query-optimization/
http://www.parkansky.com/tutorials/bdlogcgi.htm
https://serversforhackers.com/c/php-fpm-process-management

Leave a Reply

Be the First to Comment!

Notify of
avatar
wpDiscuz