Anyone executing an INSERT INTO tt_content within a TYPO3 extension bypasses cache invalidation, the Reference Index, workspace versioning, and the DataHandler extension points intended by the Core (especially SC_OPTIONS hook classes under the legacy key t3lib/class.t3lib_tcemain.php, which maps to \TYPO3\CMS\Core\DataHandling\DataHandler). This works – until it doesn't. The DataHandler is the API that orchestrates these mechanisms reliably. This guide is written with a focus on TYPO3 v14; where necessary, transitions from v13 are mentioned.
Table of Contents
The Golden Rule
Why direct SQL sabotages cache, index, and workspaces.
DataMap & CmdMap
Create, Update, Delete, Copy, Move – everything via two arrays.
Execution & Errors
Transaction safety, errorLog, and substNEWwithIDs.
CLI & Scheduler
Admin context, Bootstrap, and Symfony Commands.
Hooks
processDatamap, processCmdmap, and practical listeners.
Reference Index & Cache
Cache tags, referenceindex:update, and clear_cacheCmd.
Workspace Compatibility
Versioned records and the new discard command (v14).
Common Pitfalls
The four mistakes lurking in almost every extension.
v14 Breaking Changes
Removed properties, ISO8601 date values, and migration.
The Golden Rule: No Raw SQL for TCA Tables
For pages, tt_content, and any other TCA-configured table: Use only the DataHandler. Raw SQL (INSERT, UPDATE, DELETE) bypasses all the integrity mechanisms that TYPO3 executes in the background.
| Feature | DataHandler | Raw SQL |
|---|---|---|
| Reference Index (sys_refindex) | ✓ | ✕ |
| Cache Invalidation | ✓ | ✕ |
| Version History (sys_history) | ✓ | ✕ |
| Workspace Compatibility | ✓ | ✕ |
| SC_OPTIONS Hooks / Documented Core Events | ✓ | ✕ |
| FlexForm Processing | ✓ | ✕ |
| MM Relations | ✓ | ✕ |
| Performance for Bulk Reads | Standard | Faster |
- Custom logging tables without TCA configuration
- Bulk analytics and reporting (read-only)
- Migration scripts with an explicit Reference Index rebuild afterwards
Since TYPO3 v13.0.1 (and v12.4.11), the DataHandler blocks write access to the sys_file table. Additional fields belong in sys_file_metadata. The background for this is the Security Advisory TYPO3-CORE-SA-2024-006.
The Two Core Structures
Everything the DataHandler does is controlled via two arrays: DataMap ($data) for creating and modifying, and CmdMap ($cmd) for moving, copying, and deleting.
DataMap: Creating and Updating Records
Syntax: $data[tablename][uid][fieldname] = value
For new records, use a unique string with the prefix NEW as the UID:
The DataHandler only processes fields configured in $GLOBALS['TCA']. Fields with "type" => "none" or invalid types are ignored.
CmdMap: Moving, Copying, Deleting Records
Syntax: $cmd[tablename][uid][command] = value
Soft-delete is used automatically if $GLOBALS['TCA'][$table]['ctrl']['delete'] is configured.
Execution, Error Handling, and New UIDs
The DataHandler is stateful. Use a new instance for every operation (in practice typically GeneralUtility::makeInstance(DataHandler::class)). You should not reuse the same instance across multiple independent write operations.
DataHandler execution flow: From initialisation to UID resolution
Basic Pattern
After process_datamap(), $dataHandler->substNEWwithIDs contains the mapping of NEW placeholders to real UIDs. Example: $dataHandler->substNEWwithIDs['NEW_hero'] returns the actual UID of the created record.
Advanced: Custom DB Transactions (with Caution)
The official documentation does not define a general transaction contract for the DataHandler. In extensions, the simple API (start → process_datamap() / process_cmdmap()) is usually sufficient at first. Only use your own beginTransaction() / commit() if you intentionally bundle multiple runs or custom SQL steps on the same connection – and after checking side effects and the DB connection.
CLI and Scheduler Usage
In the CLI context (Symfony Commands, Scheduler Tasks), there is no automatic backend user. You must call Bootstrap::initializeBackendAuthentication() before the DataHandler performs write operations. Otherwise, you will receive errors like Attempt to modify table "pages" without permission.
The _cli_ backend user requires the actual necessary permissions for your tables and pages. Do not manually set $GLOBALS['BE_USER']->user['admin'] = 1. Use a real BackendUserAuthentication record with appropriate permissions and optionally pass it as the third argument to start($data, $cmd, $backendUser). Check permissions using the standard TYPO3 mechanisms — not by using a faked admin flag.
Hooks: Reacting to Database Operations
The DataHandler offers SC_OPTIONS hook classes that allow you to react to Datamap and Cmdmap processes – for logging, notifications, external synchronisation, or validation.
The Core provides no PSR-14 events with names like BeforeRecordOperationEvent or AfterDatabaseOperationsEvent for every Datamap step. For reactions after DB writes, the hook classes on t3lib/class.t3lib_tcemain.php (e.g. processDatamapClass / processCmdmapClass) are the established pattern. Under TYPO3\CMS\Core\DataHandling\Event, the Core currently documents mostly Reference Index and link parsing events — e.g. IsTableExcludedFromReferenceIndexEvent, AppendLinkHandlerElementsEvent — see DataHandling Events in the Core Docs and \TYPO3\CMS\Core\DataHandling\Event.
Available Hooks
| Hook | Timing | Method |
|---|---|---|
| processDatamapClass | Before/After data operations | processDatamap_preProcessFieldArray, processDatamap_postProcessFieldArray, processDatamap_afterDatabaseOperations, processDatamap_afterAllOperations |
| processCmdmapClass | Before/After commands | processCmdmap_preProcess, processCmdmap_postProcess, processCmdmap_deleteAction |
| clearCachePostProc | After cache clearing | Custom method |
Registering a Hook
Practical Example: Reacting to New Records
$status:'new'or'update'$table: Name of the affected table$id: UID of the record (for'new': theNEWplaceholder)$fieldArray: Array of the actually written fields$dataHandler: The DataHandler instance (access tosubstNEWwithIDs)
Reference Index and Cache
Updating the Reference Index
The Reference Index (sys_refindex) maps all relationships between records. The DataHandler updates it automatically – after bulk operations or migrations, you should additionally check it via the CLI:
In TYPO3 v14, Check and update reference index is located under Admin Tools → System → Maintenance (EXT:install), no longer at the previous low-level entry point. On the CLI, vendor/bin/typo3 referenceindex:update remains useful for large instances.
Cache Invalidation
The DataHandler automatically invalidates the relevant caches after every operation via cache tags. For each affected record, the following tags are flushed:
- Table name: e.g.
pages,tt_content - Table + UID: e.g.
tt_content_123 - Page UID: e.g.
pageId_10
For manual cache clearing, use clear_cacheCmd:
Developing Workspace-Compatible
If your code runs in a workspace environment, the DataHandler automatically creates versioned records. This requires no special handling – as long as you use the DataHandler.
New in v14: The discard Command
TYPO3 v14 introduces a new DataHandler command that discards workspace changes – cleaner and more explicit than the previous version/clearWSID construct:
The Core accepts the UID of a live or workspace record in DataHandler::discard(); for a live UID, the workspace overlay row is resolved. See DataHandler::discard().
Common Pitfalls
v14 Breaking Changes
TYPO3 v14 removes several public properties of the DataHandler and changes the behaviour of date values. Here is the complete overview:
| Change | Forge Issue | Migration |
|---|---|---|
| userid and admin properties removed | #107848 | Backend user is read directly from $GLOBALS['BE_USER'] |
| storeLogMessages removed | #106118 | log() now always writes to sys_log – no configuration necessary |
| copyWhichTables, neverHideAtCopy, copyTree removed | #107856 | Values are read from BE_USER->uc (neverHideAtCopy, copyLevels) |
| New discard command | #107519 | Replaces version/clearWSID and version/flush for workspace operations |
| ISO8601 date values improved | #105549 | Remove workarounds with manual timezone offsets |
| List and Page module: Record API | #107356 / #92434 | Switch preview/transformations to `Record` objects instead of raw arrays |
ISO8601 Migration in Detail
Previously, TYPO3 incorrectly treated qualified ISO8601 date values with Z as local time. v14 corrects this:
Property Migration for v14
The TYPO3 Extension Scanner detects access to the removed properties as a Weak Match. Check your extensions before upgrading to v14.
Conclusion
DataHandler is Mandatory
For every TCA table. Raw SQL bypasses cache, Reference Index, and workspace integrity.
Two Arrays, Full Control
DataMap ($data) for Create/Update. CmdMap ($cmd) for Copy/Move/Delete. Always as a new instance.
Plan v14 Migration
Check removed properties, remove ISO8601 workarounds, use discard instead of clearWSID.
The DataHandler is not an optional convenience feature – it is the prerequisite for consistent, workspace-capable, and maintainable TYPO3 extensions. Invest the time to understand it, and your extensions will thank you with stability and future-proofing.
TYPO3 DataHandler Skill
Agent-optimised reference (Skill v2.0.0, compatibility TYPO3 v13–v14): DataMap/CmdMap, backend context, Reference Index, SC_OPTIONS hooks vs. documented PSR-14 events. In the same folder: SKILL.md, SKILL-PHP84.md, SKILL-CONTENT-BLOCKS.md.