Retrieval Pipeline: Hybrid Arabic Search With Stable Citations

July 21, 2026Emad Ashraf6 min read

Ask Zico does not ask one search system for “the answer.” It builds a controlled candidate set, keeps every candidate tied to an application-owned source ID, and delays loading full evidence until ranking is complete.

The ingestion article explains the artifacts this pipeline consumes.

The runtime contract

Given a question, retrieval must return inspectable chunks with valid public metadata. A high similarity score is insufficient if the chunk cannot be hydrated, belongs to a disabled library, or violates a request filter.

flowchart TD
    query["Arabic question"] --> normalize["Normalize for search"]
    normalize --> router["Route to semantic domains"]
    router --> vector["Vectorize candidates"]
    router --> lexical["Lexical-domain shards"]
    vector --> policy["Apply library filters and exclusions"]
    lexical --> policy
    policy --> fusion["Fuse and rank by stable chunk ID"]
    fusion --> kv["Hydrate original text from KV"]
    kv --> result["Evidence candidates plus retrieval metadata"]

The order is deliberate: route, retrieve broadly, apply policy, fuse identities, then hydrate. It avoids moving full documents through stages that need only IDs, scores, and metadata. The deterministic chunk_id is the join key throughout: it identifies the vector record, lexical candidate, KV source record, citation, evaluation expectation, and later feedback event.

1. Normalize without rewriting the evidence

The query normalizer reduces search-only variation such as selected Arabic letter forms, diacritics, punctuation, whitespace, and known transliteration patterns. Arabic's orthographic and morphological variation makes this a retrieval concern, but the rules remain deliberately narrow so that search convenience does not rewrite the evidence (Farghaly and Shaalan, 2009). The original user question is still available for answer generation and diagnostics.

A query like لحن بيك إثرونوس may need lexical help for a transliterated hymn title, while a conceptual question about fasting may benefit more from semantic similarity. Normalization creates comparable signals; it does not decide which source is correct.

2. Route to one or more domains

The domain router maps query signals to internal areas such as Bible, rites, hymns, teaching, history, or general content. Routing narrows lexical work and informs retrieval filters, but it is not a single-label classifier. Ambiguous questions may activate multiple domains.

Source-library handling is separate. A domain describes meaning; a library describes the corpus artifact. Runtime feature flags decide whether specialist libraries are available in the current environment.

3. Gather complementary candidates

Vectorize is queried directly for semantic candidates, with each Vectorize vector id set to the same stable chunk_id used everywhere else. Lexical shards remain deliberately lean: they index exact Arabic titles, Coptic, transliterated terms, and liturgical vocabulary inside the routed domains instead of duplicating every source body. Dense retrieval supplies semantic recall; lexical retrieval protects exact-name and rare-term matches that embeddings can miss (Dense Passage Retrieval). The implementation can trigger or strengthen lexical retrieval when names, rare terms, or weak vector scores make it valuable.

flowchart TD
    q["Question"] --> semantic["Semantic signal"]
    q --> exact["Exact and transliterated terms"]
    semantic --> vector["Vector candidate list"]
    exact --> lexical["Lexical candidate list"]
    vector --> join["Join on chunk ID"]
    lexical --> join
    join --> ranked["Fused ranking"]

Hybrid retrieval is not “run two searches and concatenate.” Scores from different systems are not directly comparable. The fusion stage uses ranks and configured weights, deduplicates by chunk ID, and records which signals contributed to the final order. Reciprocal-rank-style fusion provides the research basis for combining independently ranked result lists without treating their raw scores as comparable (Cormack, Clarke, and Buettcher, 2009).

4. Apply filters before evidence enters context

Filters enforce product policy. They can exclude a library, restrict a request to selected source groups, or remove chunks that should not participate in a particular environment. An exclusion is not a low score; it is a rule.

Seneksar adds a date-sensitive case. When a question implies a liturgical date, structured date metadata can prioritize the appropriate record. If the date intent is missing or ambiguous, the system should not pretend that a generic semantic match has resolved it.

flowchart TD
    candidates["Fused candidates"] --> enabled{"Library enabled?"}
    enabled -->|"No"| drop["Exclude"]
    enabled -->|"Yes"| date{"Date-sensitive source?"}
    date -->|"No"| rank["Keep rank"]
    date -->|"Yes"| match{"Question date matches metadata?"}
    match -->|"Yes"| boost["Prefer matching record"]
    match -->|"No or unclear"| cautious["Keep cautious or fall back"]

5. Hydrate original text

Only the final candidate IDs are hydrated from KV. Hydration restores the original Arabic text, title, public URL, and citation metadata. Missing hydration is logged as an invariant failure because the model must never answer from an orphaned vector.

Retrieval can remain broader than the final prompt. The answer-context selector chooses a smaller evidence set based on relevance, coverage, and token limits. This protects the model from being asked to find a decisive passage buried in a long mixed context—a documented weakness even for long-context models (Lost in the Middle). This separation lets retrieval eval measure discovery while answer eval measures whether the chosen evidence was sufficient and correctly used.

Artifact-preparation and evaluation scripts

The runtime contract is produced and checked by explicit scripts in the Ask Zico source repository:

generate-rag-jsonl.mjs creates the chunk JSONL and stable chunk_id records.
prepare-vectorize-embeddings.mjs embeds search_text and writes Vectorize upload records whose id is chunk_id.
prepare-assistant-kv-bulk.mjs writes the KV bulk payload: original chunk records plus compact domain and facet lexical shards.
run-rag-retrieval-eval.mjs consumes the retrieval fixture JSONL and produces local retrieval reports; run-assistant-worker-retrieval-eval.mjs exercises the Worker retrieval path.

Evaluated knobs, not magic constants

Candidate counts, vector-versus-lexical weights, domain thresholds, exclusions, and context size are configuration backed by evals. They are product knobs because changing them affects both recall and the probability of distracting the answer model.

A dated Wa3zat build checkpoint found the expected source in the top five for 39 of 40 curated questions, with 33 at rank one and no ID/hydration invariant failures. That result justified continuing the controlled pipeline; it is not a current production SLA and says nothing by itself about answer correctness.

Failure cases that shaped the system

Exact names retrieved semantically related material but missed the named source, which strengthened lexical fallback.
A new library improved its own focused set while regressing an older set, which made mixed and per-library regression mandatory.
Larger candidate pools improved recall but added CPU and context pressure, which separated retrieval breadth from answer-context size.
Date-sensitive content exposed the difference between topic similarity and the record appropriate for a particular day.

The general lesson is to make every retrieval stage emit enough information to explain the next decision. A single final score is not an operational story.

Next: turning hydrated evidence into a safe answer or deliberate fallback.

References

Series Navigation

← Previous

Ingestion Pipeline: Turning Arabic Site Content Into Citation-Ready RAG Chunks

Grounded Answer Generation: Turning Retrieved Arabic Evidence Into Safe Answers

Series Navigation

Tags