TYPO3 DataHandler: Why Direct SQL in Extensions is a Risk

The DataHandler is the central API for secure database operations in TYPO3. A practical guide to DataMap, CmdMap, error handling, optional transactions, SC_OPTIONS hooks, and v14 breaking changes.

Overview

  • The DataHandler is the only secure method to manipulate TCA-configured tables in TYPO3 – without it, cache, Reference Index, and workspace compatibility break.
  • Two array structures control everything: DataMap ($data) for Create/Update and CmdMap ($cmd) for Copy/Move/Delete.
  • TYPO3 v14 removes several public properties and introduces the new discard command for workspace operations.
  • Hooks such as processDatamap_afterDatabaseOperations allow you to react to any database change – for logging, notifications, or synchronisation.
  • Why the DataHandler is mandatory for TCA tables.
  • How the $data and $cmd arrays are structured.
  • Error handling, optional DB transactions, and CLI usage.
  • Hooks for reacting to database changes.
  • All v14 breaking changes and migration steps.

Anyone executing an INSERT INTO tt_content within a TYPO3 extension bypasses cache invalidation, the Reference Index, workspace versioning, and the DataHandler extension points intended by the Core (especially SC_OPTIONS hook classes under the legacy key t3lib/class.t3lib_tcemain.php, which maps to \TYPO3\CMS\Core\DataHandling\DataHandler). This works – until it doesn't. The DataHandler is the API that orchestrates these mechanisms reliably. This guide is written with a focus on TYPO3 v14; where necessary, transitions from v13 are mentioned.


Table of Contents  

The Golden Rule

Why direct SQL sabotages cache, index, and workspaces.

DataMap & CmdMap

Create, Update, Delete, Copy, Move – everything via two arrays.

Execution & Errors

Transaction safety, errorLog, and substNEWwithIDs.

CLI & Scheduler

Admin context, Bootstrap, and Symfony Commands.

Hooks

processDatamap, processCmdmap, and practical listeners.

Reference Index & Cache

Cache tags, referenceindex:update, and clear_cacheCmd.

Workspace Compatibility

Versioned records and the new discard command (v14).

Common Pitfalls

The four mistakes lurking in almost every extension.

v14 Breaking Changes

Removed properties, ISO8601 date values, and migration.


The Golden Rule: No Raw SQL for TCA Tables  

For pages, tt_content, and any other TCA-configured table: Use only the DataHandler. Raw SQL (INSERT, UPDATE, DELETE) bypasses all the integrity mechanisms that TYPO3 executes in the background.

FeatureDataHandlerRaw SQL
Reference Index (sys_refindex)
Cache Invalidation
Version History (sys_history)
Workspace Compatibility
SC_OPTIONS Hooks / Documented Core Events
FlexForm Processing
MM Relations
Performance for Bulk ReadsStandardFaster
Exceptions where raw SQL is permitted
  • Custom logging tables without TCA configuration
  • Bulk analytics and reporting (read-only)
  • Migration scripts with an explicit Reference Index rebuild afterwards
sys_file is locked

Since TYPO3 v13.0.1 (and v12.4.11), the DataHandler blocks write access to the sys_file table. Additional fields belong in sys_file_metadata. The background for this is the Security Advisory TYPO3-CORE-SA-2024-006.


The Two Core Structures  

Everything the DataHandler does is controlled via two arrays: DataMap ($data) for creating and modifying, and CmdMap ($cmd) for moving, copying, and deleting.

DataMap: Creating and Updating Records  

Syntax: $data[tablename][uid][fieldname] = value

For new records, use a unique string with the prefix NEW as the UID:

TCA is a Prerequisite

The DataHandler only processes fields configured in $GLOBALS['TCA']. Fields with "type" => "none" or invalid types are ignored.

CmdMap: Moving, Copying, Deleting Records  

Syntax: $cmd[tablename][uid][command] = value

Soft-delete is used automatically if $GLOBALS['TCA'][$table]['ctrl']['delete'] is configured.


Execution, Error Handling, and New UIDs  

The DataHandler is stateful. Use a new instance for every operation (in practice typically GeneralUtility::makeInstance(DataHandler::class)). You should not reuse the same instance across multiple independent write operations.

DataHandler execution flow: From initialisation to UID resolution

Basic Pattern  

substNEWwithIDs

After process_datamap(), $dataHandler->substNEWwithIDs contains the mapping of NEW placeholders to real UIDs. Example: $dataHandler->substNEWwithIDs['NEW_hero'] returns the actual UID of the created record.

Advanced: Custom DB Transactions (with Caution)  

The official documentation does not define a general transaction contract for the DataHandler. In extensions, the simple API (startprocess_datamap() / process_cmdmap()) is usually sufficient at first. Only use your own beginTransaction() / commit() if you intentionally bundle multiple runs or custom SQL steps on the same connection – and after checking side effects and the DB connection.


CLI and Scheduler Usage  

In the CLI context (Symfony Commands, Scheduler Tasks), there is no automatic backend user. You must call Bootstrap::initializeBackendAuthentication() before the DataHandler performs write operations. Otherwise, you will receive errors like Attempt to modify table "pages" without permission.

Backend User and Permissions

The _cli_ backend user requires the actual necessary permissions for your tables and pages. Do not manually set $GLOBALS['BE_USER']->user['admin'] = 1. Use a real BackendUserAuthentication record with appropriate permissions and optionally pass it as the third argument to start($data, $cmd, $backendUser). Check permissions using the standard TYPO3 mechanisms — not by using a faked admin flag.


Hooks: Reacting to Database Operations  

The DataHandler offers SC_OPTIONS hook classes that allow you to react to Datamap and Cmdmap processes – for logging, notifications, external synchronisation, or validation.

PSR-14 vs. SC_OPTIONS (TYPO3 v14)

The Core provides no PSR-14 events with names like BeforeRecordOperationEvent or AfterDatabaseOperationsEvent for every Datamap step. For reactions after DB writes, the hook classes on t3lib/class.t3lib_tcemain.php (e.g. processDatamapClass / processCmdmapClass) are the established pattern. Under TYPO3\CMS\Core\DataHandling\Event, the Core currently documents mostly Reference Index and link parsing events — e.g. IsTableExcludedFromReferenceIndexEvent, AppendLinkHandlerElementsEvent — see DataHandling Events in the Core Docs and \TYPO3\CMS\Core\DataHandling\Event.

Available Hooks  

HookTimingMethod
processDatamapClassBefore/After data operationsprocessDatamap_preProcessFieldArray, processDatamap_postProcessFieldArray, processDatamap_afterDatabaseOperations, processDatamap_afterAllOperations
processCmdmapClassBefore/After commandsprocessCmdmap_preProcess, processCmdmap_postProcess, processCmdmap_deleteAction
clearCachePostProcAfter cache clearingCustom method

Registering a Hook  

Practical Example: Reacting to New Records  

Parameters of the afterDatabaseOperations Hook
  • $status: 'new' or 'update'
  • $table: Name of the affected table
  • $id: UID of the record (for 'new': the NEW placeholder)
  • $fieldArray: Array of the actually written fields
  • $dataHandler: The DataHandler instance (access to substNEWwithIDs)

Reference Index and Cache  

Updating the Reference Index  

The Reference Index (sys_refindex) maps all relationships between records. The DataHandler updates it automatically – after bulk operations or migrations, you should additionally check it via the CLI:

v14: Reference Index under Maintenance

In TYPO3 v14, Check and update reference index is located under Admin Tools → System → Maintenance (EXT:install), no longer at the previous low-level entry point. On the CLI, vendor/bin/typo3 referenceindex:update remains useful for large instances.

Cache Invalidation  

The DataHandler automatically invalidates the relevant caches after every operation via cache tags. For each affected record, the following tags are flushed:

  • Table name: e.g. pages, tt_content
  • Table + UID: e.g. tt_content_123
  • Page UID: e.g. pageId_10

For manual cache clearing, use clear_cacheCmd:


Developing Workspace-Compatible  

If your code runs in a workspace environment, the DataHandler automatically creates versioned records. This requires no special handling – as long as you use the DataHandler.

New in v14: The discard Command  

TYPO3 v14 introduces a new DataHandler command that discards workspace changes – cleaner and more explicit than the previous version/clearWSID construct:

Which UID for `discard`?

The Core accepts the UID of a live or workspace record in DataHandler::discard(); for a live UID, the workspace overlay row is resolved. See DataHandler::discard().


Common Pitfalls  


v14 Breaking Changes  

TYPO3 v14 removes several public properties of the DataHandler and changes the behaviour of date values. Here is the complete overview:

ChangeForge IssueMigration
userid and admin properties removed#107848Backend user is read directly from $GLOBALS['BE_USER']
storeLogMessages removed#106118log() now always writes to sys_log – no configuration necessary
copyWhichTables, neverHideAtCopy, copyTree removed#107856Values are read from BE_USER->uc (neverHideAtCopy, copyLevels)
New discard command#107519Replaces version/clearWSID and version/flush for workspace operations
ISO8601 date values improved#105549Remove workarounds with manual timezone offsets
List and Page module: Record API#107356 / #92434Switch preview/transformations to `Record` objects instead of raw arrays

ISO8601 Migration in Detail  

Previously, TYPO3 incorrectly treated qualified ISO8601 date values with Z as local time. v14 corrects this:

Property Migration for v14  

Use the Extension Scanner

The TYPO3 Extension Scanner detects access to the removed properties as a Weak Match. Check your extensions before upgrading to v14.


Conclusion  

DataHandler is Mandatory

For every TCA table. Raw SQL bypasses cache, Reference Index, and workspace integrity.

Two Arrays, Full Control

DataMap ($data) for Create/Update. CmdMap ($cmd) for Copy/Move/Delete. Always as a new instance.

Plan v14 Migration

Check removed properties, remove ISO8601 workarounds, use discard instead of clearWSID.

The DataHandler is not an optional convenience feature – it is the prerequisite for consistent, workspace-capable, and maintainable TYPO3 extensions. Invest the time to understand it, and your extensions will thank you with stability and future-proofing.

Für technisch Interessierte

TYPO3 DataHandler Skill

Agent-optimised reference (Skill v2.0.0, compatibility TYPO3 v13–v14): DataMap/CmdMap, backend context, Reference Index, SC_OPTIONS hooks vs. documented PSR-14 events. In the same folder: SKILL.md, SKILL-PHP84.md, SKILL-CONTENT-BLOCKS.md.

Auf GitHub öffnen

Let's talk about your project

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.