Blog

This note is a mix of material.  The new content is really for the phpBB developers and assumes a detailed understanding of the phpBB architecture.  It’s aim is to raise and to discuss some aspects of phpBB performance tuning.  It will ultimately be superseded by a paper targeted at a more general readership on how to tune phpBB installations.  Some of the content is extract from my previous work aimed at this second readership, but which I have left in as general background.  I am assuming that the phpBB developers will skip over familiar issues.

Many facets of IT systems have followed a Moore’s Law-like trend over the last four decades; examples include memory capacity, memory, aggregate CPU performance and disk capacity.  Yet others facets have barely maintained a steady linear improvement (typically where physics introduces hard constraints), and a good example here is hard disk access times which have perhaps improved by an order of magnitude due the shrinking physical media size at the same time as memory capacities have improved by six orders.  30 years ago a VAX computer could execute perhaps 10-40K instructions whilst waiting for the next record to be loaded from disk; now a modern 64bit server can execute perhaps 100M instructions waiting for the same record.  Hence this exponential vs.  linear growth has caused some interesting modal shifts application and OS architectures.

For example Caching Technologies are often introduced to mitigate and sometime effectively to eliminate the consequences of these linear constraints, and in a modern computer systems there can be many layers of caching going on, but such caching systems can ‘fight’ and in the worst case you can be hit by cache coherency issues.  How effectively application exploit such technologies can have a material impact on application performance, and the focus of this paper is the phpBB application.

A significant majority of phpBB installations will be on Shared Hosting Services, though either deciated host or private virtual hosts will be used for larger boards, and the issues discussed will come into place and be relevant to any analysis.

System Caches

• PHP Cache.  The PHP architecture splits the execution of PHP code into two stages: the compilation of source into bytecode and its subsequent execution.  This facilitates caching within Apache so that the relatively expensive compile operations can be carried out on a just-in-time basis and stored in a cache.  Since complex applications such as phpBB, Drupal and MediaWiki have a large source base, this compilation would normally take perhaps 50-75% of CPU cycles; an effective caching strategy can therefore result in a 2-4 x decrease in CPU loading [1] ,[2].  There are a number of PHP accelerators avaialble but since our forums run on a Solaris Coolstack (CSK) stack, I have adopted the APC cache which is included in the CSK 1.3.1 distribution.  Enabling the APC cache will by default cause any compiled code to be loaded into an LRU cache, and on our system I have initially allocated 64Mb to this cache.  The APC can also store and retrieve data by key.
• MySQL Cache.  The MySQL database engine maintains a number of caches to accelerate performance: the main caches buffer key indexes in memory, but data rows, execution plans and query results are also cached.  The MySQL query cache on our server is currently configured at 32Mb, The caching algorithm [3] is similar to one I discuss below for phpBB SQL caching.  Whilst the effectiveness such caches is limited by the need to maintain coherences (so results relating to table X are flushed when ever X is updated), whereas the application tier can understand the context of the result sets and avoid some such flushes.  Nonetheless on our system, MySQL cache hit rates are over 75%.
• File System Cache.  *nix file systems maintain large directory and buffer caches in memory [4].  These enable very efficient access to frequently used files and in many cases can obviate the need for disk I/O altogether.  However, any file updates still have to be flushed back to disk in order to maintain file system on-disk integrity.  Under normal circumstances this won’t slow the application, because such writes are asynchronous to the application.  However the overall seek / transfer rate on to physical devices represents a hard physical upper constrains and the application can reach a tipping point where the demand flush rate exceeds the corresponding device physical rate, and the application will become I/O bound at that point.

PhpBB Application Caching

The phpBB application is design to deployed from low-end shared-service implementations running small forums though to large forums running on dedicated hardware; it has to largely neutral to host OS, database and http server implementations, and yet deliver an effective runtime efficiency.  The implementation therefore makes minimal assumptions about the underlying system stack, but in an attempt to accelerate execution performance phpBB implements a number of caching strategies around the use of dynamically generated PHP code maintained in a cache directory.  One of the reasons here is that if the server stack also includes a PHP accelerator, then these dynamic modules will also be cached at the bytecode level.

The application generated code falls into the following categories:

• Template Caching.  The phpBB applications architecture separates business logic from presentation and National Language (NL) support.  One of the key implementation elements here is that there is a HTML webpage associated with each display page.  This page has an extended markup which enable conditional and loop control code as well as application and NL field embedding.  Since this extension is not supported by the HTML engine itself, such pages need preprocessing to render the HTML.  In order to avoid the high overhead associated with this processing, each template HTML is processed on first reference and converted into the corresponding PHP page.  These PHP pages are stored in the forum’s cache directory with the name tpl_<style>_<function/page>.html.php.
• Dataset Caching.  phpBB also uses this cache whenever it needs to maintain persistent data.  The two main access methods are put and get.  The put has an optional an optional time-to-live (ttl) argument which defines the time validity of the stored object (with a default of 365 days).  If the data is retrieved within this time this value is returned.  However, storage is not assured (as the cache might be purged or be size limited) so the calling layer must also handle the need to recreate the object on cache-miss.
• SQL Query Caching.  The unified database implementation has a common access method sql_query which uses this dataset caching.  The method has an optional time-to-live (ttl) argument which is set to a non-zero value by the application layer if it can tolerate recent albeit not current data within the application context.  The default ttl is zero and in this case the cache is bypassed entirely and the query is passed to the database layer.  The cache is used for for non-zero ttls, with the query object being ‘uniquely’ identified by a moniker based on its MD5 digest, and its content the serialised result set.  In the case of a cache hit,the database call is optimised away, and the query result set is obtained by deserialising the cached object.  In the case of a cache miss, the sql query is passed to the database layer, and the cache is reloaded with the result set before it is returned to the calling application.

The template cache has extremely low volatility and therefore creating this dynamic code is an extremely effective mechanism for avoiding one-off processing.  The number of code objects is also relatively small (typically less than 100).  So this approach is also well matched to PHP accelerator technologies.

On the other hand, the data set and SQL query caches can generate (on our forums) hundreds or thousands of small and reasonably volatile objects.  The decision to move these into PHP code fragments was a mistake as this will tend to flush any PHP accelerator cache.  On my system, I have realised a small performance improvement by setting up an APC filter to exclude these files from the PHP cache.

apc.cache_by_default=1
apc.filters="(/cache/sql_|/cache/data_)"

Using the filestore to cache these is a good compromise, as is the use of an accelerator such memcache or even the APC data store, but not the APC code cache, as encoding them as a PHP file as opposed to a raw serialize stream seems an odd design decision.  However, even the file based cache can create performance problems.  We have well over a hundred active sub-forums across the NL forums.  Caching the highlighted query in the table below was generating 2-5 file creates and deletes per second whilst the query itself was already reasonably well cached in MySQL and registered as a runtime of 0.00s, so I removed the ttl for this which resulted in a noticeable improvement in system performance.

Note: You need to be careful with using any PHP accelerator which assumes that such file content is idempotent.  For example filename moniker is based on the SQL query alone, and this won’t change even if the phpBB caching layer itself refreshes the content.  Hence unless you have file modification detection enabled:

apc.stat=1

phpBB will end up including stale content.  Whilst this is the default setting, many server configuration guides recommend setting apc.stat=0 as this will result in improved filesystem and cache, so in some circumstances this will lead to somewhat bizarre results for the queries below.  (In my case this was compounded by a bug in the CSK 1.3.1 implementation of APC doesn’t process this setting properly.)

Appendix