Reducing Memory Spikes in TYPO3 Backend Loops

A TYPO3 backend module often looks harmless: fetch a list, loop over it, and render rows or export a report. The trouble begins when the list contains tens of thousands of records. PHP may retain every database row, hydrated object, related record and temporary value until the request ends, so a loop that appears linear can produce a sharp memory peak and an out-of-memory failure.

The fix is rarely a faster foreach. It is a change in how data crosses the database boundary and how much state survives each iteration. Practical improvements come from measuring allocation, paginating queries, reducing hydration and verifying a safer rollout across Australian hosting environments.

Measure where the memory actually goes

A spike should be treated as a measurement problem, not proof that a PHP loop is slow. In TYPO3, a module may call a repository, convert a QueryResult to an array, then build another array for display. Every copy keeps references alive while the request runs. Establish whether the curve rises during the database fetch, object hydration, relation loading or export. A foreach can expose the problem without causing it.

Record memory_get_usage() at several points and compare it with memory_get_peak_usage(). The first shows current allocation; the second preserves the high-water mark that matters most in a shared PHP process. A profiler can attribute growth to a repository, mapper or view, while a small CLI reproduction makes measurements repeatable. Capture elapsed time as well: a change that lowers memory but multiplies database round trips may be a poor trade.

Use a representative synthetic fixture rather than production records. Match the number of pages, relations and fields closely enough to expose eager loading. Check PHP memory limits, database driver settings and TYPO3 extensions, because the same loop can behave differently on PHP 8.1 and 8.3, or with buffered and unbuffered PDO connections. Raising memory_limit can hide the symptom, but it does not make a large request safer.

Replace bulk retrieval with bounded reads

The largest gain usually comes from bounding rows before PHP sees them. With TYPO3's Doctrine DBAL connection and QueryBuilder, apply setFirstResult() and setMaxResults() or an equivalent LIMIT clause instead of executing an unbounded statement. fetchAllAssociative() is convenient, but it materialises the complete result; with a page size, it materialises only that page. Request only the columns needed by the module, including the UID needed for later updates.

Large tables also make deep offset pagination progressively slower. Where ordering and data consistency allow it, use keyset pagination: retain the last UID seen and select the next bounded range. Chunk primary-key ranges rather than building one enormous UID array, then retrieve the full fields required for each chunk. Keep the sort stable during a run so records do not jump between pages and disappear from the result.

Generators reduce PHP container overhead, but they cannot undo a query that the database driver already buffered. TYPO3's result iteration can yield rows one at a time, yet MySQL and PostgreSQL buffering behaviour still matters. For a CSV export, stream each fetched row into fputcsv() instead of joining it into a large string or collecting an array. A dedicated CLI command is often better than a long-running web request, especially for scheduled jobs.

Keep each iteration cheap

Once reads are bounded, keep the work inside each iteration small. Prefer a plain associative row when the module only needs a few scalar fields; building a full domain object can pull in metadata and relations that the template never displays. Avoid converting a QueryResult with toArray(). If objects are essential, hydrate only the required fields and release each model when its final reference is gone.

Relation loading is a common hidden multiplier. A list of 10,000 items that lazily loads two related records each can trigger 20,000 queries and retain thousands of proxies. Fetch relation identifiers in a controlled query, or use an appropriate join, and cache only repeated lookups needed within the current request. Give that cache an explicit lifetime; an unbounded static cache simply moves the spike to the next worker.

Temporary arrays, closures, exception traces and verbose logger context can survive after the useful row has been processed. Keep references short-lived, write batch progress rather than every record, and avoid capturing the entire result inside a callback. unset() helps only when no other reference remains, and gc_collect_cycles() is not a substitute for bounded design. Measure peak memory after each batch to confirm that the curve is flat rather than steadily climbing.

Apply a practical optimisation checklist

A repeatable checklist keeps the optimisation tied to evidence. Run it against the same fixture before and after the change, then save both timings and peak readings with the PHP and database versions. This makes a regression obvious and prevents a large refactor from obscuring which edit delivered the benefit.

• Set a measurable baseline first, including peak memory, request duration, result count and database query count.
• Fetch only the columns and relations the module truly needs, and avoid eager hydration of full object graphs.
• Apply database-level limits before converting results to arrays, and use stable chunking or keyset pagination for large tables.
• Process rows as they arrive, using generators or direct iteration where appropriate, and stream exports instead of building one output buffer.
• Bound relation lookups and caches, releasing row and object references after each completed batch.
• Compare counts, totals and representative output before rollout, then retain a rollback path for the previous query strategy.

For engineers working from home in Brisbane, home routine planning can protect the sleep and meal breaks that a long profiling session may otherwise consume. Keep the run bounded, save checkpoints and stop if the environment becomes unstable; household commitments deserve the same protection as a production deploy.

Validate and roll out safely

Local operating conditions shape the test plan. Australian digital agencies and ecommerce teams around Sydney, Melbourne and Brisbane often maintain TYPO3 estates alongside PHP versions that cannot be upgraded immediately. Reproduce the memory curve on the same database driver and PHP configuration as production; a developer laptop can hide driver buffering or make an old extension look harmless. Run checks outside peak retail periods such as Black Friday and the Christmas rush, when slow administration affects merchants and support teams.

Rollout logs need the same data-minimisation discipline as the module itself. For organisations covered by the Privacy Act 1988, the Australian Privacy Principles constrain how personal information is collected and used, while the Notifiable Data Breaches scheme governs eligible privacy breaches. Log record IDs, counts and timing metadata rather than names, addresses or content. The same rule applies to adjacent experiments such as mastercard casino comparisons, where debugging output must still exclude live account and payment details.

Schedule batch verification in UTC and display local times clearly. Sydney and Melbourne move between AEST and AEDT, while Brisbane remains on AEST, so a wall-clock cron rule can run twice or be skipped during the daylight-saving change. Use an explicit UTC instant or a date-based schedule instead. Start with one measured module today: capture its peak allocation, replace bulk retrieval with bounded batches, compare counts and totals, then publish the change behind a rollback switch. Make that sequence the standard for every memory-sensitive TYPO3 backend task.