Remodeling search at Supply Hero: A migration journey to OpenSearch Service with radial search

Have you ever ever looked for one thing like “low fats yogurt” at any on-line grocery retailer and seen how the outcomes appear to grasp what you imply? As an alternative of solely exhibiting gadgets with a precise match, the top-ranked merchandise are sometimes semantically associated. You may see gadgets like “Greek yogurt” or “yogurt with 0.5% fats,” even when just one phrase matches lexically. That is the facility of semantic search, and when mixed with conventional lexical search, it creates a hybrid search expertise that delivers each precision and recall.

At Supply Hero, one of many world’s main on-line meals supply platforms, the search crew has been utilizing semantic seek for grocery verticals since 2024. What began as a proof-of-concept has advanced right into a production-grade hybrid search system powered by Amazon OpenSearch Service. This method combines radial vector search with lexical retrieval to ship extremely related product outcomes at scale.

On this publish, we stroll via how Supply Hero migrated their semantic search infrastructure to Amazon OpenSearch Service, why they selected radial search over conventional k-nearest neighbor (k-NN) search, and the optimizations that made the system quick, cost-effective, and versatile for experimentation.

Legacy system overview

The unique semantic search system was constructed as a standalone service utilizing SpringBoot and Apache Lucene 9.9, deployed on Kubernetes. The retrieval stream labored as follows:

1. A person begins a search on the appliance.
2. The semantic search system retrieves the highest 50 nearest-neighbor candidates from a static in-memory Lucene index.
3. These candidates handed via a filtering layer to take away out-of-stock gadgets.
4. The filtered semantic outcomes have been merged with a parallel set of lexical search outcomes.
5. A remaining rating step mixed each candidate units to supply the response.

The crew iterated on this method over seven variations and performed a number of A/B assessments to refine the strategy. The preliminary system carried out nicely, nonetheless because the enterprise scaled, a number of ache factors emerged:

• Scalability limitations: Working vector indices as static, in-memory buildings inside Kubernetes pods meant that scaling required provisioning bigger pods or including replicas. Each choices have been costly and operationally complicated.
• Multi-model experimentation was troublesome: Working A/B/C assessments with three totally different product embedding mannequin variants required becoming all fashions inside a Kubernetes stateless workload. This created reminiscence stress and complex deployment pipelines.
• Operational overhead: Managing index builds, deployments, and model rollouts for a customized Lucene-based service required important engineering effort in comparison with a managed service.

Structure modernization with OpenSearch Service

By the top of 2025, Supply Hero had migrated their complete search infrastructure from self-managed Elasticsearch 7.x on Google Kubernetes Engine (GKE) to the absolutely managed Amazon OpenSearch Service 3.x. This migration created a pure alternative to consolidate the legacy semantic search service into OpenSearch as nicely.

The brand new structure separates considerations into two distinct pipelines: an ingestion pipeline for indexing product embeddings, and an inference pipeline for real-time hybrid retrieval.

Ingestion pipeline

For the ingestion pipeline, Supply Hero selected Amazon OpenSearch Ingestion (OSIS) to sync product embedding knowledge from Amazon Easy Storage Service (Amazon S3) to the OpenSearch area.

The stream works as follows:

1. ML mannequin
2. Airflow job: An present Apache Airflow job periodically generates product embeddings utilizing an exterior machine studying (ML) mannequin and periodically dumps the outcomes (product dad or mum ID + embedding vector) to an S3 bucket.
3. OpenSearch Ingestion pipeline: An OpenSearch Ingestion pipeline is configured with a scheduled S3 scan that performs a nightly scan from S3 and updates the brand new k-NN index in OpenSearch Service.

model: '2'
embedding-pipeline:
  supply:
    s3:
      acknowledgments: true
      scan:
        buckets:
          - bucket:
              identify: my-bucket-name
              filter:
                include_prefix:
                  - vector-search/json-index/newest
        vary: PT24H
        scheduling:
          interval: PT24H
      aws:
        area: eu-central-1
        sts_role_arn: arn:aws:iam:::position/osis-pipeline-role
      codec:
        ndjson: {}
      compression: none
  staff: '1'
  sink:
    - opensearch:
        hosts:
          - "https://..es.amazonaws.com"
        aws:
          serverless: false
          area: eu-central-1
          sts_role_arn: arn:aws:iam:::position/search-xxx
        index_type: customized
        index: emb_products_v1
        template_content: ...
        template_type: index-template
        routing: '${global_entity_id}'
        document_id: '${global_entity_id}:${master_code}'
        max_retries: '3'

As a result of the index shops product dad or mum IDs and embeddings are regenerated in batch, there isn’t any want for real-time updates. This enables the crew to refresh and force-merge the index as soon as per day, leading to extremely optimized section buildings and quick retrieval speeds (p99 < 35 ms throughout peak hours).

Organising the OSIS pipeline required just a few strains of Terraform, making it easy to provision and keep as infrastructure-as-code.

Inference pipeline

On the retrieval aspect, the system runs a hybrid search technique that mixes radial vector search with lexical search in parallel:

1. Question embedding: A person’s search question first reaches the Question Understanding (QU) service, the place it’s encoded into an embedding utilizing the identical reside ML mannequin employed for product embeddings. To optimize efficiency, embeddings for prime queries are cached.
2. Parallel lexical and semantic retrieval:
   • A radial k-NN search runs in opposition to the product embeddings index utilizing min_score to retrieve all semantically comparable merchandise above a similarity threshold.
   • A lexical BM25 search runs in opposition to the product catalog index.
     

     Evaluating p95 OpenSearch time for each lexical and semantic search.

3. ID decision and stock filter: As a result of the k-NN index shops product dad or mum IDs, a decision step maps these to particular person product IDs through a secondary index that maintains close to real-time stock updates. This strategy satisfies two key enterprise necessities inside a single retrieval name: product-id decision and real-time availability filtering.
4. Merge and re-rank: A customized post-processing step combines outcomes from each lexical and radial search, applies re-ranking logic, and returns the ultimate consequence set.

Why radial search?

Conventional k-NN search in OpenSearch makes use of a top-k strategy: you ask for the okay nearest neighbors, and also you get precisely okay outcomes no matter how comparable they really are. This works nicely for a lot of use circumstances, nevertheless it has a basic limitation for product search. It all the time returns a hard and fast variety of outcomes, even when a few of these outcomes are usually not semantically related.

Radial search solves this by flipping the paradigm. As an alternative of asking “give me the 50 closest gadgets,” you ask “give me all gadgets which can be a minimum of this comparable.” That is executed utilizing the min_score parameter within the k-NN question:

GET product-embeddings/_search
{
  "question": {
    "knn": {
      "embedding": {
        "vector": [0.12, 0.45, 0.78, ...],
        "min_score": 0.72
      }
    }
  }
}

When utilizing radial search with cosine similarity because the house kind, OpenSearch normalizes scores utilizing the associated method (rating = (1 + cosine_similarity) / 2), as documented within the OpenSearch knn-spaces reference.

This implies a min_score of 0.72 within the question instance, doesn’t straight correspond to cosine similarity. As an alternative, 0.72 is the normalized OpenSearch rating which interprets to 44% cosine similarity (that’s, cosine_similarity = 2 × 0.72 – 1 = 0.44).

In the event you want outcomes with a minimum of 90% cosine similarity, apply the method:

min_score = (1 + 0.90) / 2 = 0.95. So, you’ll set “min_score”: 0.95 in your question.

This strategy gives a number of benefits for product search:

• High quality over amount: Low-relevance outcomes are excluded on the retrieval stage fairly than counting on downstream re-ranking to filter them out.
• Variable consequence set dimension: The system naturally adapts to question specificity. Area of interest queries return fewer, extra exact outcomes. Broad queries return extra candidates for the re-ranker to work with. For instance, a extremely particular question like “Oatly oat milk barista version” may return 5 outcomes, whereas a broader question like “milk” may return 200.
• Higher recall-precision trade-off: By tuning the min_score threshold, the crew can straight management the steadiness between returning too many irrelevant outcomes and lacking related ones.

How Supply Hero chosen the edge for radial search

Selecting the best min_score threshold is necessary. Set it too excessive and also you miss related merchandise. Set it too low and also you flood the re-ranker with noise.

Supply Hero approaches threshold choice via systematic experimentation. To attain optimum precision throughout numerous markets, a tailor-made min_score threshold is assigned to every nation and question kind. These thresholds are meticulously decided via rigorous offline evaluations, which use historic person interplay and manually labeled knowledge to ascertain a tough estimate. This preliminary estimate is then additional refined and validated via a collection of reside A/B experiments.

Analysis of the brand new search system

One of many key benefits of the brand new structure is how naturally it helps experimentation. At Supply Hero, we retailer three variants of product embeddings inside a single doc:

PUT product-embeddings/_doc/1?routing=FP_DE
{
  "master_product_code": "abc123",
  "embedding_variant_1": [0.12, 0.45, 0.78, ...],
  "embedding_variant_2": [0.21, 0.4, 0.98, ...],
  "embedding_variant_3": [0.13, 0.65, 0.58, ...],
  "global_entity_id": "FP_DE"
}

On this instance, embedding_variant_1, embedding_variant_2, and embedding_variant_3 are generated from three totally different fashions for A/B/C testing. After every take a look at, the profitable variant is designated because the management, whereas the opposite two are changed with new fashions for additional experimentation. With this strategy, the crew can iterate repeatedly whereas sustaining fixed house complexity.

Optimizations of huge scale manufacturing system

Engine improve: OpenSearch 2.17 to three.3

Manufacturing metrics from one of many busiest nations.

OpenSearch 3.x launched important efficiency enhancements for vector search workloads. Submit-upgrade to OpenSearch 3.3, we noticed a ~18% discount in p95 latency for k-NN queries.

For Supply Hero’s use case, the k-NN search latency was already very low on OpenSearch 2.17 (p99 of 20–30 ms), which meant the improve to three.3 was not strictly mandatory for all clusters. The cluster serving the management group in A/B assessments nonetheless runs on OpenSearch 2.17.

Shard routing

To attenuate cross-shard overhead throughout k-NN queries, Supply Hero carried out customized shard routing primarily based on geographic market. As a result of every market (for instance, Germany, Sweden, and Finland) has its personal product catalog, routing queries to market-specific shards avoids pointless fan-out throughout your complete index.

That is an instance of methods to configure routing at index time and search time utilizing the _routing subject:

PUT product-embeddings/_doc/1?routing=FP_DE
{
  "master_product_code": "abc123",
  "embedding_variant_1": [0.12, 0.45, 0.78, ...],
  "embedding_variant_2": [0.21, 0.4, 0.98, ...],
  "embedding_variant_3": [0.13, 0.65, 0.58, ...],
  "global_entity_id": "FP_DE"
}

And at question time:

GET product-embeddings/_search?routing=FP_DE
{
  "question": {
    "knn": {
      "embedding_variant_2": {
        "vector": [0.12, 0.45, 0.78, ...],
        "min_score": 0.72
      }
    }
  }
}

This ensures {that a} question for the German market solely hits shards containing German merchandise, decreasing latency and compute overhead.

Refresh interval

As a result of the product embedding index is up to date solely as soon as per day through the OSIS batch pipeline, there isn’t any want for the default 1-second refresh interval. Supply Hero configured the index with an extended refresh interval throughout ingestion and triggers a guide refresh + power merge after the nightly batch completes.

Impression on the enterprise

The migration from self-managed Lucene on Kubernetes to Amazon OpenSearch Service achieved a ~50% discount in p95 latency, dropping response occasions from a variable 200ms+ to a steady 100ms baseline. This transition considerably improved system consistency by eliminating the excessive variance and rhythmic latency spikes seen within the earlier structure.

Finish service latency after rolling out semantic search with OpenSearch for foodpanda and yemeksepeti.

Past uncooked latency, the operational advantages have been important:

• Lowered infrastructure complexity: Eliminating the standalone Lucene service eliminated a complete deployment pipeline, monitoring stack, and on-call rotation.
• Sooner experimentation: New embedding fashions will be examined by creating a brand new index and adjusting question routing, with out requiring code deployments.
• Price effectivity: Utilizing OpenSearch’s managed infrastructure and the batch ingestion sample (refresh as soon as per day) diminished compute prices in comparison with working always-on Kubernetes pods with in-memory indices.

Conclusion

By combining radial search with lexical retrieval, Supply Hero’s crew constructed a system that adapts dynamically to question intent. It returns exact outcomes for particular queries and broader candidate units for common ones.

The migration to Amazon OpenSearch Service demonstrates how a managed search platform can simplify the operational complexity of vector search whereas enhancing efficiency.

To get began with vector search on Amazon OpenSearch Service, see the AI search documentation and the OpenSearch radial search information.

Concerning the authors