AntonyThorpe\Consumer\BulkLoader
The bulk loader allows large-scale uploads to SilverStripe via the ORM.
Data comes from a given BulkLoaderSource, providing an iterator of records.
Incoming data can be mapped to fields, based on a given mapping,
and it can be used to find or create related objects to be linked to.
Failed record imports will be marked as skipped
Modified from BetterBulkLoader of SilverStripe-ImportExport (https://github.com/burnbright/silverstripe-importexport) 2018-07-21
Synopsis
class BulkLoader
extends BulkLoader
{
- // members
- public array $mappableFields = ;
- public array $transforms = ;
- public function $recordCallback;
- protected BulkLoaderSource $source;
- protected Boolean $addNewRecords = true;
- protected boolean $relationLinkDefault = true;
- protected boolean $relationCreateDefault = true;
- protected boolean $publishPages = true;
- protected $mappableFields_cache;
- // methods
- public void setSource()
- public BulkLoaderSource getSource()
- public BulkLoader setRelationLinkDefault()
- public BulkLoader setRelationCreateDefault()
- public BulkLoader setPublishPages()
- public void deleteExistingRecords()
- public DataList getDataList()
- public array preview()
- public BulkLoaderResult load()
- protected BulkLoaderResult processAll()
- protected BulkLoaderResult processRecord()
- protected array columnMapRecord()
- protected boolean hasRequiredData()
- protected void transformField()
- protected boolean isRelation()
- protected string getRelationName()
- public mixed findExistingObject()
- public array getMappableColumns()
- public array scaffoldMappableFields()
- protected array getMappableFieldsForClass()
- protected void formatMappingFieldLabel()
- public void preprocessChecks()
- public BulkLoaderResult updateRecords()
- public BulkLoaderResult upsertManyRecords()
- public BulkLoaderResult deleteManyRecords()
- public BulkLoaderResult clearAbsentRecords()
Hierarchy
Extends
- SilverStripe\Dev\BulkLoader
Members
protected
- $addNewRecords
—
AntonyThorpe\Consumer\Boolean
Add new records while importing - $mappableFields_cache
—
array
Cache the result of getMappableColumns - $publishPages
—
boolean
Determines whether pages should be published during loading - $relationCreateDefault
—
boolean
The default behaviour creating relations - $relationLinkDefault
—
boolean
The default behaviour for linking relations - $source
—
AntonyThorpe\Consumer\BulkLoaderSource
Bulk loading source
public
- $mappableFields
—
array
Fields and corresponding labels that can be mapped to. - $recordCallback
—
AntonyThorpe\Consumer\function
Specify a colsure to be run on every imported record. - $transforms
—
array
Transformation and relation handling
Methods
protected
- columnMapRecord() — Convert the record's keys to the appropriate columnMap keys
- formatMappingFieldLabel() — Format mapping field laabel
- getMappableFieldsForClass() — Get the fields and labels for a given class
- getRelationName() — Given a record field name, find out if this is a relation name and return the name
- hasRequiredData() — Check if the given mapped record has the required data.
- isRelation() — Detect if a given record field is a relation field.
- processAll() — Import all records from the source
- processRecord() — Process a single record from source
- transformField() — Perform field transformation or setting of data on placeholder.
public
- clearAbsentRecords() — Clear property value if it doesn't exist as a value in the API data Note: only use with full set of API data otherwise values will be cleared in the dataobject that should be left
- deleteExistingRecords() — Delete all existing records
- deleteManyRecords() — Delete dataobjects that match to the API data
- findExistingObject() — Find an existing objects based on one or more uniqueness columns specified via {@link self::$duplicateChecks}
- getDataList() — Get the DataList of objects this loader applies to.
- getMappableColumns() — Get the field-label mapping of fields that data can be mapped into.
- getSource() — Get the BulkLoaderSource for this BulkLoader
- load() — Start loading of data
- preprocessChecks() — Check that the class has the required settings
- preview() — Preview a file import (don't write anything to the database).
- scaffoldMappableFields() — Generate a field-label list of fields that data can be mapped into.
- setPublishPages() — Set pages to published upon upload
- setRelationCreateDefault() — Set the default behaviour for creating new relation objects.
- setRelationLinkDefault() — Set the default behaviour for linking existing relation objects.
- setSource() — Set the BulkLoaderSource for this BulkLoader.
- updateRecords() — Update the dataobject with data from the external API
- upsertManyRecords() — Update/create many dataobjects with data from the external api