RSS Feeds

A PHP Cache Overview and Cache Tuning

This note is a mix of material.  The new content is really for the phpBB developers and assumes a detailed understanding of the phpBB architecture.  It's aim is to raise and to discuss some aspects of phpBB performance tuning.  It will ultimately be superseded by a paper targeted at a more general readership on how to tune phpBB installations.  Some of the content is extract from my previous work aimed at this second readership, but which I have left in as general background.  I am assuming that the phpBB developers will skip over familiar issues.

Many facets of IT systems have followed a Moore's Law-like trend over the last four decades; examples include memory capacity, memory, aggregate CPU performance and disk capacity.  Yet others facets have barely maintained a steady linear improvement (typically where physics introduces hard constraints), and a good example here is hard disk access times which have perhaps improved by an order of magnitude due the shrinking physical media size at the same time as memory capacities have improved by six orders.  30 years ago a VAX computer could execute perhaps 10-40K instructions whilst waiting for the next record to be loaded from disk; now a modern 64bit server can execute perhaps 100M instructions waiting for the same record.  Hence this exponential vs.  linear growth has caused some interesting modal shifts application and OS architectures. 

For example Caching Technologies are often introduced to mitigate and sometime effectively to eliminate the consequences of these linear constraints, and in a modern computer systems there can be many layers of caching going on, but such caching systems can 'fight' and in the worst case you can be hit by cache coherency issues.  How effectively application exploit such technologies can have a material impact on application performance, and the focus of this paper is the phpBB application. 

A significant majority of phpBB installations will be on Shared Hosting Services, though either deciated host or private virtual hosts will be used for larger boards, and the issues discussed will come into place and be relevant to any analysis.

System Caches

PhpBB Application Caching

The phpBB application is design to deployed from low-end shared-service implementations running small forums though to large forums running on dedicated hardware; it has to largely neutral to host OS, database and http server implementations, and yet deliver an effective runtime efficiency.  The implementation therefore makes minimal assumptions about the underlying system stack, but in an attempt to accelerate execution performance phpBB implements a number of caching strategies around the use of dynamically generated PHP code maintained in a cache directory.  One of the reasons here is that if the server stack also includes a PHP accelerator, then these dynamic modules will also be cached at the bytecode level. 

The application generated code falls into the following categories:

The template cache has extremely low volatility and therefore creating this dynamic code is an extremely effective mechanism for avoiding one-off processing.  The number of code objects is also relatively small (typically less than 100).  So this approach is also well matched to PHP accelerator technologies. 

On the other hand, the data set and SQL query caches can generate (on our forums) hundreds or thousands of small and reasonably volatile objects.  The decision to move these into PHP code fragments was a mistake as this will tend to flush any PHP accelerator cache.  On my system, I have realised a small performance improvement by setting up an APC filter to exclude these files from the PHP cache. 

apc.cache_by_default=1
apc.filters="(/cache/sql_|/cache/data_)" 

Using the filestore to cache these is a good compromise, as is the use of an accelerator such memcache or even the APC data store, but not the APC code cache, as encoding them as a PHP file as opposed to a raw serialize stream seems an odd design decision.  However, even the file based cache can create performance problems.  We have well over a hundred active sub-forums across the NL forums.  Caching the highlighted query in the table below was generating 2-5 file creates and deletes per second whilst the query itself was already reasonably well cached in MySQL and registered as a runtime of 0.00s, so I removed the ttl for this which resulted in a noticeable improvement in system performance. 

Note: You need to be careful with using any PHP accelerator which assumes that such file content is idempotent.  For example filename moniker is based on the SQL query alone, and this won't change even if the phpBB caching layer itself refreshes the content.  Hence unless you have file modification detection enabled:

apc.stat=1

phpBB will end up including stale content.  Whilst this is the default setting, many server configuration guides recommend setting apc.stat=0 as this will result in improved filesystem and cache, so in some circumstances this will lead to somewhat bizarre results for the queries below.  (In my case this was compounded by a bug in the CSK 1.3.1 implementation of APC doesn't process this setting properly.)

Appendix

File

TTL

Query

includes/functions.php

60

SELECT COUNT(DISTINCT s.session_ip) as num_guests FROM phpbb3_sessions s WHERE s.session_user_id = 1 AND session_time >= N AND s.session_forum_id = N

style.php

300

SELECT s.style_id, c.theme_id, c.theme_data, c.theme_path, c.theme_name, c.theme_mtime, i.*, t.template_path FROM styles s, styles t, styles_theme c, styles_imageset i WHERE s.style_id = N AND t.template_id = s.template_id AND c.theme_id = s. theme_id AND i.imageset_id = s.imageset_id

includes/functions_privmsgs.php

300

SELECT forum_id, forum_name, parent_id, forum_type, left_id, right_id FROM forum ORDER BY left_id ASC

includes/functions_admin.php

360

SELECT forum_id, forum_name, parent_id, forum_type, left_id, right_id FROM forums ORDER BY left_id ASC

600

SELECT forum_id, forum_name, parent_id, forum_type, left_id, right_id FROM forums ORDER BY left_id ASC

includes/message_parser.php

600

SELECT * FROM smilies ORDER BY LENGTH(code) DESC

includes/functions_display.php

3600

SELECT m.*, u.user_colour, g.group_colour, g.group_type FROM (moderator_cache m) LEFT JOIN phpbb3_users u ON (m.user_id = u.user_id) LEFT JOIN phpbb3_groups g ON (m.group_id = g.group_id) WHERE m.display_on_index = 1 AND m.forum_id in (list)

includes/functions_display.php

3600

SELECT forum_name FROM forums WHERE forum_id = N

includes/functions_posting.php

3600

SELECT * FROM smilies WHERE display_on_posting = 1 ORDER BY smiley_order

includes/session.php

3600

SELECT ban_ip, ban_userid, ban_email, ban_exclude, ban_give_reason, ban_end FROM banlist WHERE ... ';

3600

SELECT s.style_id, t.template_storedb, t.template_path, t.template_id, t.bbcode_bitfield, t.template_inherits_id, t.template_inherit_path, c.theme_path, c.theme_name, c.theme_storedb, c.theme_id, i.imageset_path, i.imageset_id, i.imageset_name FROM styles s, styles_template t, styles_theme c, styles_imageset i WHERE s.style_id = $style AND t.template_id = s.template_id AND c.theme_id = s.theme_id AND i.imageset_id = s.imageset_id

3600

SELECT s.style_id, t.template_storedb, t.template_path, t.template_id, t.bbcode_bitfield, c.theme_path, c.theme_name, c.theme_storedb, c.theme_id, i.imageset_path, i.imageset_id, i.imageset_name FROM styles s, styles_template t,styles_theme c, styles_imageset i WHERE s.style_id = $style AND t.template_id = s.template_id AND c.theme_id = s theme_id AND i.imageset_id = s.imageset_id";

3600

SELECT image_name, image_filename, image_lang, image_height, image_width FROM styles_imageset_data WHERE imageset_id = 'N' AND image_filename <> AND image_lang IN ( language list)";

includes/bbcode.php

3600

SELECT * FROM bbcodes WHERE ...

includes/acp/acp_profile.php

3600

SELECT forum_name FROM forums WHERE forum_id = N

includes/acp/acp_permissions.php

3600

SELECT forum_id, forum_name, parent_id, forum_type, left_id, right_id FROM forums ORDER BY left_id ASC

includes/acp/auth.php

3600

SELECT forum_id, enable_indexing FROM forums

ncludes/acp/acp_search.php

3600

SELECT forum_id, enable_indexing FROM forums

includes/functions_content.php

3600

SELECT forum_id, forum_name, parent_id, forum_type, left_id, right_id FROM forums ORDER BY left_id ASC

3600

SELECT forum_name FROM forums ' . FORUMS_TABLE WHERE forum_id = N

3600

SHOW TABLE STATUS FROM db_name

includes/mcp/mcp_queue.php

3600

SELECT forum_id, forum_name FROM forums WHERE forum_id in (list);

3600

SELECT s.style_id, c.theme_id, c.theme_data, c.theme_path, c.theme_name, c.theme_mtime, i.*, t.template_path FROM styles s, styles_template t, styles_theme c, styles_imageset i WHERE s.style_id = N AND t.template_id = s.template_id AND c.theme_id = s.theme_id AND i.imageset_id = s.imageset_id