PHP-FPM performance tuning for high traffic website

Sometimes, website users experiences high CPU load due to PHP-FPM processes.

Here at Ibmi Media, as part of our Server Management Services, we regularly help our Customers to fix the performance of their web servers.

In this context, we shall look into this error and how to get rid of it.

More about PHP-FPM? Why does it cause high load and affect website performance?

PHP-FPM signifies PHP FastCGI Process Manager. It is a FastCGI daemon which allows a website to handle high load. But sometimes this service itself causes high load and affect the performance of the website in general.

To troubleshoot this issue, start by checking the PHP-FPM error logs, which is available in a location such as "/var/log/php7.0-fpm.log".

This location depends on the installation path and PHP version. When we checked the error log in our case, we discovered some warning message as shown below;

[5-Nov-2020 11:12:11] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 9 idle, and 89 total children

From this message, the PHP-FPM pool service is very busy and is unable to handle the high traffic with the existing configuration parameters.

PHP-FPM can result to high server load due to different factors:

i. Too many processes running due to high traffic or attacks
ii. A related service such as MySQL or web server being abused
iii. PHP-FPM configuration is not properly mapped to available server resources
iv. Poorly coded or buggy applications abusing the server resources

How to fix PHP-FPM tuning to allow high load?

The first thing server owners tend to do for an immediate fix, is to restart the PHP-FPM service. But simply restarting the service does not fix it.

When we debug such load issues, we analyze the logs and server performance over a period of time. This helps us to pinpoint the problem and take prompt actions to solve it.

1. Switch process manager

Process manager is mainly of 3 types – dynamic, on-demand and static. The performance of the service drastically varies based on this type.

In dynamic type, the number of child processes is set dynamically based on the PHP-FPM parameters in conf file. But it is a bit memory-intensive type.

In static type, the number of child processes is fixed by pm.max_children parameter, but this type is not flexible for a server with changing web traffic. It also consumes too much memory.

In on-demand type, the PHP-FPM processes are spawned only on demand, based on the traffic. This type helps to manage varying traffic in memory restrained servers. But the overhead increases when there is so much traffic fluctuation.

The PHP-FPM process manager is determined after assessing the available server memory, traffic spikes, website pages, CPU usage, etc.

2. Tweak PHP-FPM parameters

PHP-FPM has a lot of configuration parameters which determine the way it performs. These parameters have to be determined based on available server resources such as RAM and CPU.

For instance, the total processes that run in a server would be approximately = (Total RAM / Memory per process) . In a server with multiple services, all of them are taken into account for the tuning process.

The PHP-FPM configuration file will be available at "/etc/php-fpm.conf" or some other location based on the service path. The contents of the file would look like:

listen.allowed_clients = 127.0.0.1

pm = dynamic

pm.max_children = 5

pm.start_servers = 2

pm.min_spare_servers = 2

pm.max_spare_servers = 5

pm.max_requests = 150

Some of the major parameters that we tune for performance are:

pm.max_children – This is used to set the total number of processes allowed

pm.start_servers – The number of child processes created on startup is defined by this

pm.min_spare_servers – Defines the minimum number of idle processes

pm.max_spare_servers – Sets the maximum number of idle server processes

pm.process_idle_timeout – The number of seconds an idle process will be alive

pm.max_requests – This sets the execution time of each child process and is used to curb memory leaks.

A common mistake server owners do, is to set very high values for these parameters, to get maximum performance. But if there are not enough resources, such high values can crash the server.

The value for ‘pm.max_children’ parameter is first determined based on the server resources, website traffic and content of the sites. Other parameters are then derived upon further testing.

3. Enable slow log

In cases where the parameters are fine, but a certain application or site is suspected to cause the high load, we enable the slow log option in PHP-FPM.

request_slowlog_timeout = 6s

slowlog = /var/log/php-fpm/slowlog-site.log

The log file mentioned for ‘slowlog’ parameter in the PHP-FPM config file records the requests that take more than 6 seconds to execute.

Analyzing these requests enable us to track and fix the resource-intensive requests and applications in the server.

4. Managing multiple pools

PHP-FPM has the option to configure multiple resource pools for separate applications. This helps to contain resources per application and for additional security.

By configuring parameters based on the pool, we restrict a single pool from using up all server resources. Unused pools are deleted for further management.

The process of PHP-FPM tuning for high load

Tweaking PHP-FPM for best performance, is not just a template fix. The configuration and parameters for the service would vary based on the server needs and resources.

During peak traffic, the performance would be different from that of normal hours. So, after doing the tweaks, we conduct stress tests to see how the service performs during high load.

Based on the server performance during stress test, we do further fine-tuning of the service. We also monitor the server load for a period of time, to ensure that the changes has indeed helped.

In this scenario, we were able to reduce the server load drastically, by performing the PHP-FPM tweaks.

But PHP-FPM tuning alone may not help in certain scenarios where services such as MySQL or web server too are culprits. In those cases, we perform additional tasks such as:

i. Fine-tuning the MySQL server and web server such as Apache and Nginx
ii. Limiting server resources based on service, applications, users, etc. to avoid server crash
iii. Setting up load balancers or clusters to distribute services in high traffic servers
iv. Monitoring the server 24/7 and conducting periodic server audits to detect and resolve issues then and there
v. Enabling page caching and optimizing code and queries to avoid applications from overloading server