manus-skills/.kortix/opencode/skills/coding/mongodb-search-and-ai/references/lexical-search-querying.md

Lexical Search - Querying

This guide covers query patterns and optimization techniques for MongoDB Atlas Search.

Table of Contents

• $search vs $searchMeta
• Query Patterns
• Query Optimization
• Query Performance Analysis

---

$search vs $searchMeta

Both stages must be the first stage in an aggregation pipeline.

Stage	Use When
$search	You need matching documents, with or without metadata
$searchMeta	You only need metadata (count, facets) — no documents returned

$searchMeta shares the following fields with $search: index, all operator names (e.g. text, range, compound), concurrent (parallelizes search across segments on dedicated search nodes only — ignored otherwise), and returnStoredSource.

---

Query Patterns

Operator Reference

Operator	Description
autocomplete	Search-as-you-type from incomplete input
compound	Combines multiple operators into a single query
embeddedDocument	Queries fields inside arrays of objects
equals	Exact match on boolean, date, number, objectId, token, uuid
exists	Tests for presence of a field
geoShape	Queries shapes by spatial relation (geo type, indexShapes: true)
geoWithin	Queries points within a region (geo type)
hasAncestor	Queries ancestor-level fields when using returnScope
hasRoot	Queries root-level fields when using returnScope
in	Queries single values or arrays of values
moreLikeThis	Finds documents similar to a given document
near	Queries values near a number, date, or geo point
phrase	Searches for terms in a specific order
queryString	Boolean/field-specific query syntax
range	Queries values within a numeric, date, string, or objectId range
regex	Regular expression matching on string fields
text	Full-text analyzed search on string fields
vectorSearch	Semantic search with lexical pre-filters (vector type in search index)
wildcard	Wildcard pattern matching on string fields

---

Count Results

Use the count option in $searchMeta to count matching documents without fetching them. Also works in $search via the $SEARCH_META aggregation variable when you need both results and count.

// Count only (recommended)
db.movies.aggregate([
  {
    $searchMeta: {
      range: { path: "year", gte: 2010, lte: 2015 },
      count: { type: "lowerBound" }  // or "total" for exact count
    }
  }
])
// Returns: { count: { lowerBound: NumberLong(1001) } }

// Count alongside results using $SEARCH_META
db.movies.aggregate([
  {
    $search: {
      text: { path: "title", query: "<query>" },
      count: { type: "total" }
    }
  },
  { $project: { title: 1, meta: "$SEARCH_META" } },
  { $limit: 10 }
])

type	Behavior
lowerBound	Approximate. Exact up to threshold (default 1000), rough above it.
total	Exact count. Slower on large result sets.

Note: Count affects performance — use only when needed (e.g., first page of paginated results).

---

Pagination with searchSequenceToken

Cursor-based pagination using tokens. More efficient than $skip alone for deep pagination.

Step 1 — Get tokens from the initial query:

db.movies.aggregate([
  {
    $search: {
      index: "<index-name>",
      text: { path: "title", query: "summer" },
      sort: { released: 1, _id: 1 }  // Sort on a unique field to prevent tie-ordering issues
    }
  },
  { $limit: 10 },
  {
    $project: {
      title: 1, released: 1,
      paginationToken: { $meta: "searchSequenceToken" }
    }
  }
])

Step 2 — Next page using searchAfter:

db.movies.aggregate([
  {
    $search: {
      index: "<index-name>",
      text: { path: "title", query: "summer" },
      searchAfter: "<token-from-last-document-on-previous-page>",
      sort: { released: 1, _id: 1 }  // maintain the same sort order
    }
  },
  { $limit: 10 },
  { $project: { title: 1, paginationToken: { $meta: "searchSequenceToken" } } }
])

Use searchBefore with the first document's token on the current page to go to the previous page — results are returned in reverse order. Combine searchAfter with $skip to jump pages.

Key constraint: Query semantics (operator, path, query value, sort) must be identical between the initial query and any searchAfter/searchBefore query.

---

Retrieve Arrays of Objects with returnScope

Return each element of an embedded document array as an individually scored document. Works in both $search and $searchMeta.

Requirements:

• Array field indexed as embeddedDocuments type with storedSource defined on the fields to return
• returnStoredSource: true in the query
• All operator paths must be nested under returnScope.path (use hasAncestor or hasRoot to query outside it)

Index:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "funding_rounds": {
        "type": "embeddedDocuments",
        "dynamic": true,
        "storedSource": {
          "include": ["round_code", "raised_currency_code", "raised_amount"]
        }
      }
    }
  }
}

Query:

db.companies.aggregate([
  {
    $search: {
      range: { path: "funding_rounds.raised_amount", gte: 5000000, lte: 10000000 },
      returnStoredSource: true,
      returnScope: { path: "funding_rounds" }
    }
  },
  { $limit: 5 }
])

Only fields defined in storedSource within the embedded document are returned — root-level fields are excluded. When returnScope is specified, all query paths must start with returnScope.path.

---

Advanced Query Syntax (queryString)

Use case: Complex search with boolean operators, wildcards, and field-specific queries.

Fields configuration:

// Add to mappings.fields in your index:
{
  "title": { "type": "string" },
  "director": { "type": "string" },
  "year": { "type": "number" }
}

Query patterns:

// Boolean operators
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      queryString: {
        defaultPath: "title",
        query: "detective AND (noir OR thriller) NOT comedy"
      }
    }
  }
])

// Field-specific searches
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      queryString: {
        defaultPath: "title",
        query: "title:inception AND director:nolan"
      }
    }
  }
])

// Wildcards and ranges
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      queryString: {
        defaultPath: "title",
        query: "star* AND year:[2010 TO 2020]"
      }
    }
  }
])

Supported syntax:

• Boolean: AND, OR, NOT
• Grouping: (term1 OR term2)
• Wildcards: * (0+ chars), ? (single char)
• Ranges: [min TO max] for numbers/dates
• Field-specific: fieldName:value

Key considerations:

• Great for building search UIs with advanced options
• Users can construct complex queries without API changes
• Validate/sanitize user input to prevent injection

---

Searching Nested Arrays (embeddedDocument)

Use case: Search within arrays of objects where element-wise comparisons are required (similar to $elemMatch), or each element must be scored independently.

Fields configuration:

// Add to mappings.fields in your index:
{
  "title": { "type": "string" },
  "reviews": {
    "type": "embeddedDocuments",  // Required for array search
    "fields": {
      "author": { "type": "string" },
      "text": { "type": "string" },
      "rating": { "type": "number" }
    }
  }
}

Query pattern:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      embeddedDocument: {
        path: "reviews",
        operator: {
          compound: {
            must: [
              { text: { query: "excellent", path: "reviews.text" } }
            ],
            filter: [
              { range: { path: "reviews.rating", gte: 4 } }
            ]
          }
        },
        score: { embedded: { aggregate: "maximum" } }  // or sum, minimum, mean
      }
    }
  }
])

Score aggregation options:

• sum: Add scores from all matching array elements
• maximum: Use highest score from array elements
• minimum: Use lowest score from array elements
• mean: Average scores from array elements

Key considerations:

• Each array element is indexed as a separate document
• Use embeddedDocuments field type, not regular document
• Score aggregation controls how array matches affect overall document score
• Performance can be degraded due to complexity of parent-child joins

---

Search Highlighting

Use case: Show users which parts of documents matched their query.

Fields configuration:

// Add to mappings.fields in your index:
{
  "title": { "type": "string" },
  "plot": { "type": "string" }
}

Query pattern:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "detective noir",
        path: "plot"
      },
      highlight: {
        path: "plot",
        maxCharsToExamine: 500000,  // Default
        maxNumPassages: 5            // Number of snippets
      }
    }
  },
  {
    $project: {
      title: 1,
      plot: 1,
      highlights: { $meta: "searchHighlights" },
      score: { $meta: "searchScore" }
    }
  }
])

Highlight result structure:

{
  "highlights": [
    {
      "path": "plot",
      "texts": [
        { "value": "A ", "type": "text" },
        { "value": "detective", "type": "hit" },
        { "value": " investigates a murder in ", "type": "text" },
        { "value": "noir", "type": "hit" },
        { "value": " Los Angeles", "type": "text" }
      ],
      "score": 1.23
    }
  ]
}

Key considerations:

• type: "hit" indicates matched terms
• type: "text" is surrounding context
• Multiple passages returned for long documents
• Use in search results UI to show match context

---

Compound Queries

Compound queries combine multiple operators efficiently:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      compound: {
        must: [
          { text: { query: "detective", path: "plot" } }  // Required, affects score
        ],
        should: [
          { text: { query: "mystery", path: "genre" } }   // Optional, boosts score
        ],
        filter: [
          { range: { path: "year", gte: 2000 } }          // Required, no score impact
        ],
        mustNot: [
          { text: { query: "comedy", path: "genre" } }    // Excludes results
        ]
      }
    }
  }
])

Clause types:

• must: Required matches that affect scoring
• should: Optional matches that boost scores
• filter: Required matches that don't affect scoring (faster)
• mustNot: Exclusions

Performance tips:

• Use filter instead of must for criteria that shouldn't affect scoring (faster)
• Put most selective criteria in must or filter first
• Limit should clauses to 3-5 for best performance

---

Query with Synonyms

When your index is configured with synonyms, specify the synonym mapping name in your query:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "car chase",
        path: "description",
        synonyms: "synonym-mapping-name"  // Reference the mapping from your index
      }
    }
  }
])

Note: When you specify a synonym mapping name, MongoDB Search automatically searches for the query terms AND all their synonyms (e.g., "car" also matches "automobile", "vehicle").

---

Using Multi Analyzers

Query specific analyzer variants of a field:

// Standard fuzzy search
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "Action",
        path: "title"  // Uses default analyzer
      }
    }
  }
])

// Exact match using keyword analyzer
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "Action",
        path: "title.keywordAnalyzer"  // Uses alternate analyzer
      }
    }
  }
])

Use case: Support both fuzzy and exact matching on the same field without duplicating data.

---

Autocomplete

Search-as-you-type on fields indexed as autocomplete type (see lexical-search-indexing.md).

Option	Description
query	String to search
path	Field indexed as autocomplete
tokenOrder	any (tokens in any order; sequential matches score higher) or sequential (tokens must be adjacent)
fuzzy	{ maxEdits: 1|2, prefixLength: <n>, maxExpansions: <n> }

To score exact matches higher, index the field as both autocomplete and string types and query using compound.

---

Facet

Groups results into buckets by field values or ranges. Use with $searchMeta for metadata only, or with $search + $SEARCH_META variable for results and metadata.

{ "$searchMeta": { "facet": {
    "operator": { <operator> },
    "facets": {
      "<facet-name>": { "type": "string|number|date", "path": "<field>", ...options }
    }
} } }

Facet type	Field index type	Bucket definition
string	token	Top N unique string values. numBuckets defaults to 10.
number	number	Numeric ranges via boundaries array + optional default bucket
date	date	Date ranges via boundaries array + optional default bucket

---

geoShape

Query shapes by spatial relation. Field must be indexed as geo type with indexShapes: true. Required fields: geometry (GeoJSON Polygon, MultiPolygon, or LineString), path, and relation:

relation	Meaning
contains	Indexed geometry contains the query geometry
disjoint	No overlap between geometries
intersects	Geometries overlap
within	Indexed geometry is within the query geometry (not supported for LineString or Point)

---

geoWithin

Query geographic points within a region. Field must be indexed as geo type. Specify one of:

• box: { bottomLeft: <GeoJSON Point>, topRight: <GeoJSON Point> }
• circle: { center: <GeoJSON Point>, radius: <meters> }
• geometry: GeoJSON Polygon or MultiPolygon

For both geo operators: longitude must be specified before latitude; longitude range [-180, 180], latitude range [-90, 90].

---

Query Optimization

Sorting Search Results

Use the sort option inside $search to sort at the mongot level (more efficient than a $sort stage after). Supports: boolean, date, number, objectId, uuid, and string (must be indexed as token type). Cannot sort on embeddedDocuments type fields.

db.collection.aggregate([
  {
    $search: {
      text: { ... },
      sort: { "fieldName": -1, "title": 1, score: { $meta: "searchScore" } }
    }
  },
  { $limit: 10 }
])

Sort by score:

sort: { score: { $meta: "searchScore", order: 1 } }  // ascending (lowest score first)
sort: { score: { $meta: "searchScore" } }             // descending (default)

Null/missing values: Appear first in ascending sort by default. Use noData: "highest" to push them last:

sort: { "field": { order: 1, noData: "highest" } }

Key rules:

• sort inside $search only works on indexed fields — use $sort after for non-indexed or computed fields
• For searchSequenceToken pagination, sort must include a unique field (e.g., _id) to avoid tie-ordering
• Arrays: ascending uses smallest element, descending uses largest

Using Stored Source

Retrieve frequently accessed fields directly from the search index instead of the database:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: { query: "detective", path: "plot" },
      returnStoredSource: true  // Retrieve from mongot, not DB
    }
  },
  { $limit: 20 },
  { $match: { rating: { $gte: 7 } } }  // Filter on stored fields
])

Requirements:

• Fields must be configured in storedSource in your index definition
• Dramatically improves performance by avoiding database lookups
• Especially beneficial when filtering or sorting after $search

---

$match After $search

Minimize blocking stages after $search — prefer encapsulating filter logic inside the $search stage itself using compound.filter. This avoids additional mongod operations and makes full use of the Atlas Search index.

Prefer compound.filter over $match for fields indexed in the search index (string, token, number, date, boolean, objectId, uuid, geo):

// Prefer this
{ $search: { compound: { must: [{ text: { ... } }], filter: [{ range: { path: "year", gte: 2000 } }] } } }

// Avoid this where possible
{ $search: { text: { ... } } },
{ $match: { year: { $gte: 2000 } } }

If you must use $match (e.g., for non-indexed or computed fields), use storedSource + returnStoredSource to avoid a full document lookup in mongod:

{ $search: { text: { ... }, returnStoredSource: true } },
{ $match: { storedField: { $exists: true } } }

---

Query Performance Analysis

Use explain to analyze query performance:

db.collection.explain("executionStats").aggregate([
  { $search: { /* ... */ } }
])

Important: Atlas Search explain output differs from standard MongoDB explain. It shows execution on the search engine (mongot) side with Lucene-specific statistics, not standard MongoDB execution plans.