AI-Powered Product Recommendations That Actually Convert: Moving Past Collaborative Filtering

A fashion retailer replaced their collaborative filtering recommendation engine with a deep learning model that incorporated browsing context, visual similarity, and real-time session behavior. Click-through rates on product recommendations went from 2.8% to 7.1%, and the conversion rate on recommended products jumped from 1.2% to 3.4%. The revenue attributed to recommendations grew from 11% to 24% of total online sales.

Collaborative filtering, the approach that powers most ecommerce recommendation engines with the logic of "customers who bought X also bought Y," has been the default for over two decades. It works well enough for large catalogs with dense purchase data (Amazon built their early recommendation system on it). But for most mid-market retailers, it has significant limitations that newer approaches address.

Where Collaborative Filtering Falls Short

The cold start problem is the most obvious limitation. Collaborative filtering needs purchase history to make recommendations. For new products with zero purchases, the system has nothing to work with. For new customers with no purchase history, it can only offer generic bestseller lists. In ecommerce, where new products launch weekly and new visitors represent 40-60% of traffic, this gap is substantial.

Collaborative filtering also struggles with the popularity bias. It tends to recommend popular products because those products appear in more purchase histories. This creates a feedback loop: popular products get recommended more, which drives more purchases, which makes them appear in even more recommendations. Long-tail products that might be perfect for a specific customer rarely surface because they lack enough co-purchase data.

The sparsity problem compounds both issues. Even on a site with thousands of products and hundreds of thousands of customers, the purchase matrix is extremely sparse. Most customers have bought a tiny fraction of the catalog, so the overlap between any two customers is minimal. This makes it hard to find reliable co-purchase patterns for anything other than the most popular items.

Finally, collaborative filtering ignores context. It does not consider what the customer is doing right now in their session. A customer who has been browsing running shoes for 20 minutes should see different recommendations than the same customer browsing dress shoes, even though their purchase history is identical. Collaborative filtering only sees the static purchase history, not the dynamic session behavior.

The Modern Approach: Hybrid Deep Learning Models

Current state-of-the-art recommendation systems combine multiple signal types through a deep learning architecture. The inputs typically include interaction sequences (not just purchases but also views, add-to-carts, search queries, and time spent on product pages), product embeddings that capture item attributes (category, brand, price, visual features extracted from product images, text features from descriptions), user embeddings that capture customer attributes and behavioral patterns, and contextual signals like time of day, device type, traffic source, and current session activity.

The model learns to predict the probability that a given customer will engage with a given product, given all of these inputs. During inference (when generating recommendations for a live session), the model scores all candidate products and surfaces the top N by predicted engagement probability.

Visual similarity is one of the most impactful additions for fashion, home decor, and lifestyle categories. A convolutional neural network extracts visual features from product images, and these features are used to identify products that look similar in style, color, or aesthetic. When a customer is browsing a mid-century modern walnut coffee table, the visual similarity model can recommend other mid-century modern furniture pieces even if the purchase data does not connect them (because few customers buy multiple coffee tables).

Real-Time Session Personalization

The biggest performance gain comes from incorporating real-time session behavior. A sequence model (transformer-based or recurrent neural network) processes the customer's current session as a stream of events: page views, search queries, category navigation, filter selections, and dwell time on each product. This session sequence is encoded into a session embedding that captures the customer's current intent.

The session embedding updates with every action the customer takes. After viewing three blue dresses in the $50-80 range, the session model learns that the customer is interested in blue dresses in that price range, and recommendations shift accordingly. This happens within the current session, without waiting for purchase data that collaborative filtering would require.

A beauty retailer implementing session-based recommendations saw that recommendation relevance (measured by click-through rate) increased progressively during a session. After 1 page view, recommendations performed similarly to the old collaborative filtering system. After 5 page views, click-through rates were 2.1x higher. After 10 page views, they were 3.4x higher. The more the model learned about the current session, the better its recommendations became.

Handling the Cold Start

For new products, the hybrid model uses content-based features (product attributes, images, descriptions) to generate initial recommendations without any purchase data. A new dress can be recommended to customers who have shown interest in similar dresses by style, color, price, and brand, even though nobody has bought it yet. As purchase data accumulates, the model blends collaborative signals with content signals, gradually shifting the weight as the data matures.

For new visitors with no history, the real-time session model kicks in after just a few interactions. Even without knowing who the customer is, the model adapts recommendations based on what they are doing right now. The first product page they visit provides a weak signal. The second and third provide much stronger signals. By the time they have browsed five or six products, the recommendations are significantly personalized even for a completely anonymous visitor.

Measuring Recommendation Quality

Click-through rate on recommendations is the most commonly tracked metric, but it can be misleading. A recommendation that drives clicks but not conversions is generating traffic to pages that do not convert, which is a waste of valuable real estate. The metrics that matter are revenue per recommendation impression (total revenue from recommended products divided by total recommendation impressions shown), conversion rate on recommended products vs. overall site conversion rate, diversity of recommended products (are you showing the same bestsellers to everyone, or genuinely personalizing?), and catalog coverage (what percentage of your catalog appears in recommendations over a 30-day period).

A/B testing recommendation algorithms is essential because offline metrics (precision, recall, nDCG) do not always correlate with business outcomes. A model that is slightly less precise in predicting clicks but shows more diverse products might generate more incremental revenue because it helps customers discover products they would not have found otherwise.

For ecommerce retailers still running basic collaborative filtering or rule-based recommendations, the gap in performance compared to modern deep learning approaches is widening. The infrastructure to deploy these models has matured significantly in the past two years, with platforms like Algolia Recommend, Amazon Personalize, and Dynamic Yield offering managed recommendation services that handle the model training and serving complexity. The investment is measured in months of setup, not years, and the revenue impact is measurable within the first quarter of deployment.