RAG for Ecommerce: Why Product Catalogs Don't Belong in Vector Databases

If you're building an AI system to handle customer questions on Shopify, WooCommerce, or any platform, you will face a problem. Some queries have stable answers, and others change every time someone buys something. Treating both the same way is where most RAG for ecommerce projects quietly fail.

Here's what we learned building a bot that auto-replies to comments on a brand's Instagram and Facebook posts. Also, why is "just embed the whole product catalog" the wrong default, even though it's the first thing everyone tries?

Why Embedding Your Product Catalog Breaks RAG

We built a bot that replies to comments on a brand’s Instagram and Facebook posts. The bot needed to answer two types of questions.

The first type covered stable information:

• How long do returns take?
• What is the shipping policy?
• Do products include a warranty?
• Can sale items be exchanged?

The second type covered live product information:

• Is this hoodie in stock?
• Does it come in blue?
• What is the current price?
• Is medium still available?

Some comments needed both.

For example:

“Do you have the blue hoodie in medium, and can I return it if it does not fit?”

The system had to decide whether to search the vector database, fetch data from Shopify, or use both sources.

The Fix: Structured RAG That Splits by Data Type

Once you separate what actually needs semantic retrieval from what needs a live source-of-truth call, the architecture gets simpler. This is sometimes called hybrid retrieval or structured RAG, pairing vector search with live data the kind of system a LangChain developer typically builds and maintains.

• Policies, FAQs, brand voice, shipping info: this barely changes, and the question isn't asking for an exact live number. This is exactly what RAG is good at. It goes in the vector store, ideally with product-aware chunking so related context doesn't get split across chunks.
• Price, stock, variants: pulled live from Shopify Admin API at query time. The number the customer gets is the number that's true right now.

That split creates a new problem, though: given an incoming comment, how do you know which pipeline it belongs to, and what happens when it needs both pipelines?

How the Router Chooses a Pipeline

Three labels:

• INVENTORY
• RAG
• BOTH

INVENTORY sends the comment to Shopify. RAG sends it to the vector database. Both runs both pipelines and combines the results before the reply is written. The router looks for signals in the comment.

A few signals it leans on:

• Product names, "in stock," "price," "size" → pulls toward the inventory path
• "Return," "shipping," "policy," "warranty" → pulls toward the RAG path

A clear product mention also increases the need for Shopify data. This is a controlled form of agentic RAG because the underlying Python logic chooses which source to use. It does not have open access to many tools. It selects only one of the three allowed routes.

Why Keyword Rules Are Not Enough

Keyword rules are useful for simple comments, but social media language is often unclear.

A shopper may write:

• “Still got these?”
• “What about the red one?”
• “Got this in large?”
• “Can I send it back?”
• “How much for this?”

These comments may depend on the post caption.

If the caption says, “Winter essentials: our nylon hoodie is back,” the word “this” probably refers to the hoodie.

An LLM router can read both the caption and the comment. It can understand short phrases, spelling errors, and implied product mentions that basic rules may miss.

Rules can still act as a quick first step. The model handles harder cases.

Example One: Inventory Only

The router sees a product reference and an availability question. It returns INVENTORY.

The vector database is skipped.

The system extracts:

• Product: Nylon Hoodie
• Color: Blue
• Requested detail: Availability

It then builds a Shopify query and receives the current product data.

The bot replies:

“Yes, the nylon hoodie is currently available in blue.”

No RAG lookup was needed.

Example Two: RAG Only

Comment:

“How long do returns take?”

The router sees a return-policy question and no live product request. It returns RAG.

Shopify is skipped.

The system retrieves the return policy from the vector database and gives it to the response model.

The bot replies:

“Returns are processed within five to seven business days after the item is received.”

This is the type of information that belongs in embeddings.

Example Three: Both Sources

Comment:

“Do you have the blue hoodie in medium, and can I return it?”

The router sees both inventory and policy intent. It returns BOTH.

Shopify provides the current size and color availability.

RAG provides the return policy.

The final reply combines both results:

“The blue hoodie is available in medium, and it can be returned if it meets the return policy requirements.”

Why Live Search Needs Dedicated Risk Controls

Shopify's GraphQL API is cost-based, not request-count-based. You get a points budget that refills over time, and each query eats into it based on how much it asks for. The tricky part: when you run out, Shopify doesn't hand you a clean 429. It returns HTTP 200, which appears successful, with a THROTTLED error buried in the response body. Watching status codes alone won't catch this; you have to read the body, catch the error, then back off and retry.

This matters most exactly when it matters most: a post going viral. If every comment triggered a live Shopify call, the points budget would burn through fast, right when volume is highest. That's the real argument for the router and a caching layer built on the right infrastructure. And for a rate limiter sitting in front of the Shopify path, so a sudden spike gets throttled gracefully instead of hammering the API and torching the whole budget.

Common Mistakes Teams Make With RAG for Ecommerce

A few patterns show up repeatedly in teams making this mistake for the first time:

• Re-embedding on every webhook and calling it "solved." It shrinks the staleness window but doesn't fix the fact that vector search is the wrong retrieval method.
• Using keyword rules alone for routing. They break the moment a customer implies a product without naming it directly, or phrases a question in an unexpected way.
• Running both pipelines on every comment "just to be safe." This is the same waste as over-embedding; most comments need exactly one path, and paying for both by default.
• Treating a 200 response as success. On cost-based APIs like Shopify's GraphQL endpoint, a throttled request can still return HTTP 200 with the error sitting inside the response body.

The Takeaway

RAG isn't wrong for ecommerce. Embedding your entire product catalog is. The moment you separate "things that need semantic retrieval" from "things that need a live source-of-truth call," you stop fighting the tool. A router even a simple one is what makes that split actually work in production without doubling your API costs on every message.

Frequently Asked Questions

Do I need a vector database for ecommerce product search?

For descriptions, FAQs, and policies, yes. For live price and stock, no. Those should come from a direct API call to your store (Shopify, WooCommerce, etc.) at query time, not from an embedding.

Why does my AI chatbot quote the wrong price or stock count?

Almost always because it's retrieving that answer from a vector store instead of a live API; embeddings are frozen at index time; prices and inventory move continuously, so the two are structurally incompatible.

Can't I re-embed on every inventory webhook to fix staleness?

It narrows the lag window but doesn't fix the underlying issue: you're still using a similarity-search tool to answer an exact-value lookup question, and you're paying compute costs to re-embed on every sale.

How do you decide which comments need a live Shopify lookup vs. RAG?

An LLM-based router reads each comment for signals that product names and stock/price language point to inventory, while policy and return language point to RAG and can trigger both when a comment needs them. Low-confidence outputs default to RAG as the safer path.

What is structured or hybrid RAG for ecommerce?

It's an architecture that pairs vector search for unstructured, slow-changing content (FAQs, policies) with deterministic, live lookups for structured, fast-changing fields (price, stock, variants), rather than forcing everything through a single embedding pipeline.