Magento 2 Catalog Search Broken? Unmasking the OpenSearch Reindex Race Condition

Magento 2 Catalog Search Broken? Unmasking the OpenSearch Reindex Race Condition

As e-commerce migration experts, we often encounter complex technical challenges within the Magento ecosystem. A recent GitHub issue (#41746) highlights a critical vulnerability in Magento Open Source 2.4.8-p5 (and 2.4-develop) related to the catalogsearch_fulltext indexer when utilizing OpenSearch. This issue, if triggered, can lead to a completely broken storefront search and category navigation, rendering the site unusable until a full reindex is successfully completed.

The Core Problem: Overlapping Reindex Processes

The reported bug describes a race condition that occurs when two full reindex processes for catalogsearch_fulltext overlap. This is particularly problematic in environments with large catalogs where reindexing can take significant time, increasing the likelihood of concurrent operations. The scenario typically unfolds when a long-running reindex (Process A) is interrupted or overlapped by another reindex triggered by an indexer:reset followed by indexer:reindex (Process B). This often happens in real-world scenarios due to scheduled cron jobs, deployment scripts, or manual intervention.

How the Search Index Gets Corrupted

The issue stems from several critical flaws in how Magento's Elasticsearch adapter (shared with OpenSearch) handles index creation, deletion, and alias switching during concurrent reindexing:

  • Weak Locking Mechanism: Magento's indexer status guard (working status) is bypassed by indexer:reset or invalidate(), allowing a second full reindex to start even if another is in progress.
  • Premature Index Deletion: Process B's cleanIndex() method, designed to remove old or incomplete indexes, inadvertently deletes the temporary index that Process A is actively building.
  • Silent Auto-Creation of Corrupted Indexes: Process A, unaware its target index has been deleted, continues to write bulk data to the original index name. With OpenSearch's default action.auto_create_index: true, this silently recreates the index, but with default OpenSearch settings and dynamic mapping—lacking Magento's crucial analyzers and field configurations.
  • Alias Switch to a Broken Index: When Process A eventually completes, its updateAlias() method points the live search alias to this newly auto-created, corrupted index. Simultaneously, it often deletes the correct index that Process B might have successfully built, leading to data loss and an unusable search.

This results in an incomplete search index with incorrect field types (e.g., price_0_1 as text instead of double), and missing Magento-specific analyzers (like prefix_search). On the storefront, this manifests as category pages displaying no products and quick search failing with OpenSearch\Common\Exceptions\BadRequest400Exception errors, such as "Text fields are not optimised for operations that require per-document field data like aggregations and sorting."

Reproducing the Issue

The issue author, ilkis, provided a detailed script to reliably reproduce this race condition, even on small sample databases, by simulating a long reindex process:

 Fulltext::executeByDimensions().
 * The only difference is a delay per product, to simulate a large catalog on a small sample DB.
 *
 * Usage (from the Magento root): php slow_full_reindex.php  
 */
use Magento\Framework\App\Area;
use Magento\Framework\App\Bootstrap;
use Magento\Framework\App\State as AppState;
use Magento\Framework\Indexer\StateInterface;

require __DIR__ . '/app/bootstrap.php';

$om = Bootstrap::create(BP, $_SERVER)->getObjectManager();
$om->get(AppState::class)->setAreaCode(Area::AREA_CRONTAB);

$storeId = (string)($argv[1] ?? '1');
$delayUs = (int)($argv[2] ?? 100) * 1000;
$indexerId = 'catalogsearch_fulltext';

$indexer = $om->get(\Magento\Framework\Indexer\IndexerRegistry::class)->get($indexerId);
$indexerData = $om->get(\Magento\Framework\Indexer\ConfigInterface::class)->getIndexer($indexerId);

// Same as Indexer::reindexAll(): mark the indexer as "working"
$indexer->getState()->setStatus(StateInterface::STATUS_WORKING)->save();

$dimensi
    'scope' => $om->get(\Magento\Framework\Indexer\DimensionFactory::class)->create('scope', $storeId),
];
$handler = $om->get(\Magento\CatalogSearch\Model\Indexer\IndexerHandlerFactory::class)
    ->create(['data' => $indexerData, 'batchSize' => 20]);
$fullAction = $om->get(\Magento\CatalogSearch\Model\Indexer\Fulltext\Action\FullFactory::class)
    ->create(['data' => $indexerData]);

// Same as Fulltext::executeByDimensions() for a full reindex
$handler->cleanIndex($dimensions);
echo date('H:i:s'), " A: cleanIndex() done, new index prepared; writing documents
";

$documents = (function () use ($fullAction, $storeId, $delayUs) {
    foreach ($fullAction->rebuildStoreIndex((int)$storeId) as $id => $document) {
        usleep($delayUs);
        yield $id => $document;
    }
})();
$handler->saveIndex($dimensions, $documents);   // addDocs() per batch, then updateAlias()
echo date('H:i:s'), " A: saveIndex() done, alias switched
";

$indexer->getState()->setStatus(StateInterface::STATUS_VALID)->save();

By running this script in one terminal and then executing bin/magento indexer:reset catalogsearch_fulltext && bin/magento indexer:reindex catalogsearch_fulltext in another while the first is still active, the issue can be observed in a controlled environment. This precise reproduction method is invaluable for developers and system administrators debugging similar production outages.

Proposed Solutions and Immediate Workaround

The issue report suggests several potential fixes, including implementing a more robust, per-indexer lock that isn't released by indexer:reset, validating the prepared index's integrity (e.g., using UUIDs or _meta markers) before switching aliases, and ensuring cleanIndex() only targets true leftover indexes. These architectural improvements would significantly enhance the stability of Magento's search indexing.

For immediate mitigation, the crucial workaround is to avoid running indexer:reset catalogsearch_fulltext or invalidating the indexer while a full reindex is already in progress. If a reindex is needed, simply running bin/magento indexer:reindex catalogsearch_fulltext without a preceding reset is safer, as the built-in lock for the working status will prevent a second process from starting. This S1 severity issue underscores the importance of understanding intricate system behaviors and implementing careful operational procedures in e-commerce platform maintenance.

Start with the tools

Explore migration tools

See options, compare methods, and pick the path that fits your store.

Explore migration tools