\CachableChunkedRefreshJob
Looking at the refresh BuildTask: We break the entire iteration over the $pages result-set into chunks depending on peak memory usage which is checked at each iteration. If meory exceeeds a preset limit, we pass processing of the cache-refresh onto a job queue.
The idea is that each chunk should be managable enough in terms of memory and
execution time to run even on the smallest of host setups, rather than iteratively
refreshing all objects in the same chunk of N 100s of pages, and consuming a ton
of system resources.
- Author: Deviate Ltd 2015 http://www.deviate.net.nz
Synopsis
class CachableChunkedRefreshJob
extends AbstractQueuedJob
implements
QueuedJob
{
- // members
- protected CacheableNavigationService $service;
- protected array $chunk = ;
- protected string $stage = '';
- protected number $subsiteID = 0;
- public static number $critical_memory_buffer = 2097152;
- // methods
- public void __construct()
- public void setService()
- public void setChunk()
- public void setStage()
- public void setSubsiteID()
- public CacheableNavigationService getService()
- public array getChunk()
- public string getStage()
- public number getSubsiteID()
- public string getTitle()
- public boolean jobFinished()
- public number chunkSize()
- public void process()
- public number getJobType()
- public string getSignature()
Hierarchy
Extends
- AbstractQueuedJob
Implements
- QueuedJob
Members
protected
- $chunk — array
- $service — CacheableNavigationService
- $stage — string
- $subsiteID — number
public
- $critical_memory_buffer
—
number
On each N iterations of the for-loop in $this->process(), we check to see if current peak memory exceeds buffer subtracted from memory_limit. If it does, we throw an exception which is caught by {@link QueuedJobService} to mark the job as broken.
Methods
public
- __construct() — Sets internal variables and persistent data for when job is created without constructor params, and process() is called in {@link QueuedJobService}.
- chunkSize()
- getChunk()
- getJobType() — Uses the QUEUED type, to ensure we make as efficient use of system resources as possible.
- getService()
- getSignature() — By default {@link AbstractQueuedJob} will only queue 1 "identical" job at a time, so an implementation of this method is necessary becuase we need to fire-off multiple jobs for processing a different chunk of objects.
- getStage()
- getSubsiteID()
- getTitle() — Pack all relevant info into the job's so that it's viewable in the "queuedjobs" CMS section. The title data in the title will appear inaccuarate when run via the main ProcessJobQueueTask.
- jobFinished()
- process() — The body of the job: Runs the memory-intensive refreshXX() method on each page of the passed $chunk, using the passed $service.
- setChunk()
- setService()
- setStage()
- setSubsiteID()