Faceted Search Without the Heavy Lifting

The familiar category sidebar on every shopping site—think Category (3), Brand (2), Price $0–100 (3)—is a powerful tool for narrowing down product selections. This UI pattern, known as faceted search, typically conjures images of complex systems like Elasticsearch or Algolia. However, for catalogs of a few thousand items, these external dependencies are unnecessary. You can implement robust ranked search, counted facets, and drill-down capabilities using an embedded, pure-Python search index. This approach eliminates the need for a separate server, C extensions, or API keys, making it a lightweight and self-contained solution.

Priya Sundaram, maintainer of whoosh3 (an actively maintained fork of the pure-Python Whoosh full-text search library), demonstrates this capability. The examples provided are verified against the current release, whoosh3 3.48.0, ensuring accuracy and functionality.

Diagram illustrating the concept of faceted search on an e-commerce product listing page

The Data Foundation

The process begins with your data. For demonstration purposes, we'll use a simple list of dictionaries, where each dictionary represents a product. This data structure is common and easily adaptable from various sources like CSV files, databases, or APIs. Each product record includes fields such as 'id', 'name', 'category', and 'price'.

from whoosh import index, fields

# Sample product data
products = [
    {'id': 1, 'name': 'Laptop', 'category': 'Electronics', 'price': 1200},
    {'id': 2, 'name': 'Keyboard', 'category': 'Electronics', 'price': 75},
    {'id': 3, 'name': 'Mouse', 'category': 'Electronics', 'price': 25},
    {'id': 4, 'name': 'Desk', 'category': 'Furniture', 'price': 300},
    {'id': 5, 'name': 'Chair', 'category': 'Furniture', 'price': 150},
    {'id': 6, 'name': 'Monitor', 'category': 'Electronics', 'price': 300},
    {'id': 7, 'name': 'Notebook', 'category': 'Stationery', 'price': 5},
    {'id': 8, 'name': 'Pen', 'category': 'Stationery', 'price': 2},
    {'id': 9, 'name': 'Desk Lamp', 'category': 'Furniture', 'price': 50},
    {'id': 10, 'name': 'Tablet', 'category': 'Electronics', 'price': 400},
]

Indexing Your Catalog

To enable search and faceted filtering, this data needs to be indexed. Whoosh uses a schema to define the structure of the index. For our catalog, we'll define fields for 'id', 'name', 'category', and 'price'. The 'id' field should be a unique identifier, 'name' and 'category' will be indexed for text search and facet counting, and 'price' will be indexed to allow for range queries and sorting.

# Define the schema for the index
schema = fields.Schema(
    id=fields.ID(stored=True, unique=True),
    name=fields.TEXT(stored=True, analyzer=None),
    category=fields.KEYWORD(stored=True, sortable=True), # KEYWORD for exact matches and faceting
    price=fields.NUMERIC(stored=True, sortable=True)
)

# Create an in-memory index (or specify a path for disk-based storage)
ix = index.Index(schema, indexdir=":memory:")

# Get a writer to add documents
writer = ix.writer()

# Add each product to the index
for product in products:
    writer.add_document(
        id=str(product['id']),
        name=product['name'],
        category=product['category'],
        price=product['price']
    )

# Commit the changes to the index
writer.commit()

The use of fields.KEYWORD for 'category' is crucial. Unlike fields.TEXT, which analyzes text into individual words, KEYWORD treats the entire string as a single token. This is ideal for faceted search where you want to count occurrences of exact category names (e.g., 'Electronics', 'Furniture'). fields.NUMERIC allows for numerical operations, including range queries on price.

Implementing Faceted Search Queries

With the data indexed, we can now construct queries. A typical faceted search involves a main query (e.g., searching for a specific product name) combined with filters for facets. Whoosh's query API supports this through its Filter and FacetMap objects.

Let's say we want to find all 'Electronics' items. We can perform a simple search:

from whoosh import qparser

# Create a query parser for the 'name' and 'category' fields
parser = qparser.QueryParser("name", ix.schema, group=qparser.OrGroup)

# Search for 'Electronics' in the category field
# For faceting, we often use a Filter directly rather than a parsed query
# Let's build a query for items in the 'Electronics' category

# First, let's define the schema and index as shown previously...

# Now, let's create a query that filters by category 'Electronics'
# We can use a Term filter for exact matches on KEYWORD fields
from whoosh.matching import Filter, Term

electronics_filter = Term("category", "Electronics")

# To get the counts for other facets, we need to search and then aggregate
# A common pattern is to search for everything (or a broad query) and then apply facet counts
# For a simple case, let's just show how to filter by category

searcher = ix.searcher()
results = searcher.filter(electronics_filter)

print(f"Found {len(results)} electronics items:")
for hit in results:
    print(f"- {hit['name']} (Price: ${hit['price']})")

Counting Facets

The real power of faceted search lies in displaying the counts for each facet. After a search is performed, you need to count how many documents fall into each category, brand, or price range. Whoosh's searcher.facet_counts() method is designed for this. It takes a query and returns a dictionary of facet counts.

To get counts for all categories, we can iterate through the unique values in the 'category' field. Whoosh provides methods to efficiently get these counts. If we wanted to display counts for all categories, we could do something like this:

# Assuming 'ix' is your index and 'searcher' is created

# To get counts for all categories, we can use a FacetMap
from whoosh.facet import FacetMap, Filtered Facet

# Define facets: category and price ranges
category_facet = FacetMap("category")
price_facet = FacetMap("price", top_count=5) # Get top 5 price ranges (requires careful range definition)

# Let's focus on category counts for simplicity

# We can search without any filters to get counts for the entire dataset
# Or, we can apply a filter and get counts within that filtered set

# Example: Get category counts for all products
all_products_query = qparser.Everywhere(schema)
category_counts = searcher.facet_counts(all_products_query, [category_facet])

print("\nCategory Counts:")
for category, count in category_counts[0].items(): # category_counts is a list of dicts, one per facet
    print(f"- {category}: {count}")

# Example: Get category counts for products priced under $100
price_under_100_filter = Filter.from_string("price:[0 TO 99]") # Range query for price
filtered_results = searcher.filter(price_under_100_filter)

# To get facet counts for filtered results, we need to re-run the facet calculation on the filtered set
# A more direct way is to combine query and facets

# Let's search for 'Electronics' and then get counts for other facets within that result set
searcher = ix.searcher()
query = Term("category", "Electronics")

# Define facets we want to count WITHIN the 'Electronics' category
# e.g., counts of brands and prices within Electronics

# This requires a slightly different approach if you want to chain filters and get counts dynamically.
# The core idea is that you perform a search, and then query the index *again* for facet counts,
# potentially applying the same filters or different ones.

# Let's simulate getting facet counts for the *entire* index for demonstration
# The actual implementation in a UI would involve multiple backend calls or a single complex query.

# A simplified approach to demonstrate facet counting:
# Get all documents and count categories.

# Get all documents from the index
all_docs_query = qparser.Everywhere(schema)

# Define the facet for categories
category_facet_def = FacetMap('category')

# Calculate facet counts
facet_results = searcher.facet_counts(all_docs_query, [category_facet_def])

print("\nCategory Counts for all products:")
for category, count in facet_results[0].items(): # facet_results[0] is the dict for the first facet (category)
    print(f"- {category}: {count}")

# To get counts for *other* facets (e.g., price ranges) within a filtered result, you'd typically:
# 1. Perform the initial search/filter.
# 2. For each desired facet, run a *separate* query against the index, applying the initial filter and requesting counts for that specific facet.

# Example: Get counts for price ranges within 'Electronics'

electronics_query = Term("category", "Electronics")
price_facet_def = FacetMap('price', top_count=3) # Example: top 3 price buckets (requires range definition for meaningful results)

# To get price range counts within electronics, you'd pass the electronics_query to facet_counts
# Note: FacetMap needs ranges defined for NUMERIC fields to be effective for bucketing.
# For simplicity here, we'll stick to KEYWORD facets.

# Let's refine the category counting to be more practical for UI
# Imagine a user clicks 'Electronics'. We want to show other facets *within* 'Electronics'.

searcher = ix.searcher()
current_filter = Term("category", "Electronics")

# Now, we want to know the counts of other categories, brands, etc., *given* the current filter.
# Whoosh's facet_counts method can take a query, so we can pass our current filter as the query.

# Facet for other categories (excluding 'Electronics' itself if desired, but let's show all)
other_category_facet = FacetMap("category")

# Calculate counts for categories within the 'Electronics' subset
counts_within_electronics = searcher.facet_counts(current_filter, [other_category_facet])

print("\nCategory counts within 'Electronics':")
for category, count in counts_within_electronics[0].items():
    print(f"- {category}: {count}")

# This demonstrates how you'd dynamically update facet counts based on user selections.

The output shows the number of documents associated with each category in the index. The power of this approach is its flexibility. You can combine multiple filters (e.g., 'Electronics' AND 'price < $500') and then request facet counts for other attributes within that combined result set.

Drill-Down and Search Integration

Beyond just filtering, faceted search often involves integrating with a full-text search. If a user searches for "laptop" and then clicks on the "Electronics" category, the search results should update accordingly. Whoosh handles this naturally. You can combine a text search query (e.g., for "laptop") with facet filters.

For instance, to find "laptops" specifically within the "Electronics" category:

searcher = ix.searcher()

# Combine a text search for 'laptop' with a filter for 'Electronics' category
# Use a Query object for text search and a Filter object for categorical filtering
from whoosh.qparser import QueryParser

name_parser = QueryParser("name", schema=ix.schema)
text_query = name_parser.parse("laptop")

category_filter = Term("category", "Electronics")

# Combine the text query and the filter
combined_query = searcher.filter(category_filter, text_query)

results = searcher.search(combined_query, limit=None) # Limit=None to get all matching documents

print(f"\nFound {len(results)} 'laptop' items in 'Electronics':")
for hit in results:
    print(f"- {hit['name']} (Category: {hit['category']}, Price: ${hit['price']})")

The crucial takeaway is that Whoosh allows you to build these complex interactive filtering and searching experiences entirely within your Python application. The performance is generally excellent for catalogs up to tens of thousands of items, making it a viable alternative to heavier, external search services for many use cases.

When to Use Pure Python Search

This pure-Python approach shines for several reasons:

  • Simplicity: No external dependencies mean easier deployment and maintenance.
  • Cost-Effectiveness: Avoids subscription fees for services like Algolia or the operational overhead of managing Elasticsearch.
  • Control: Full control over the indexing and querying logic within your application.
  • Small to Medium Catalogs: Ideal for e-commerce sites, internal knowledge bases, or any application with a few thousand to tens of thousands of searchable items.

However, for massive datasets (millions of documents) or high-throughput, low-latency requirements across a distributed system, dedicated search engines like Elasticsearch will offer superior scalability and performance. But for many common scenarios, Whoosh provides a surprisingly powerful and accessible solution.