fix(hellofresh): search only the meal-kit products #11

Merged

TimCane merged 2 commits from fix/hellofresh-products-filter into main

2026-07-19 16:23:42 +01:00

TimCane commented

2026-07-19 16:21:43 +01:00

(Migrated from github.com)

Summary

Found by the first live HelloFresh crawl. The search endpoint was being called without the products filter the site's own requests carry, so it returned add-on products alongside recipes.

Three of the first five "recipes" stored were Fruit Selection | Apples, Pear, Orange, Easy Peelers, Greek Style Natural Yoghurt | 6 x 75g Packs and SlooOW Stone Oven White Baguette - one ingredient, one step, zero prep time each. A meal planner would have offered a baguette as a dinner.

Measured live against the API:

Query	Total	First results
no filter	35,084	yoghurt, baguette, fruit (`steps=1`)
`products=classic-box\|veggie-box\|meal-plan\|classic-plan`	24,545	real meal kits (`steps=6`)

The filter is now HelloFreshOptions, bound from Sources:hellofresh, rather than hardcoded.

Docs updated with what the live run taught us

The catalogue is 24,545 recipes, not 3,880. That figure came from the HAR, which captured a cuisine-filtered query. A full pass is roughly 3,070 requests and about 1h45m at the configured 2s delay - worth knowing before scheduling one
Cloudflare blocks by IP reputation. One ProtonVPN London exit was refused outright ("Sorry, you have been blocked" - a WAF block, no challenge), while another Proton London exit served 200. Since there is no challenge to solve, a headless browser or FlareSolverr does not help; reconnecting gluetun for a different exit does. README says so where someone hitting a 403 will look

Test plan

117 tests pass (72 unit, 45 integration)
Re-crawled live after the fix: 5 fetched, 5 changed, all genuine meal kits with 11-15 ingredients, every one carrying an amount, 6 steps each
Cross-source search_recipes for "chicken" over the MCP server returned results from both sources - 4 HelloFresh and 1 Gousto - with prep times and calories

## Summary Found by the first live HelloFresh crawl. The search endpoint was being called without the `products` filter the site's own requests carry, so it returned add-on products alongside recipes. Three of the first five "recipes" stored were `Fruit Selection | Apples, Pear, Orange, Easy Peelers`, `Greek Style Natural Yoghurt | 6 x 75g Packs` and `SlooOW Stone Oven White Baguette` - one ingredient, one step, zero prep time each. A meal planner would have offered a baguette as a dinner. Measured live against the API: | Query | Total | First results | |---|---|---| | no filter | 35,084 | yoghurt, baguette, fruit (`steps=1`) | | `products=classic-box\|veggie-box\|meal-plan\|classic-plan` | 24,545 | real meal kits (`steps=6`) | The filter is now `HelloFreshOptions`, bound from `Sources:hellofresh`, rather than hardcoded. ## Docs updated with what the live run taught us - **The catalogue is 24,545 recipes, not 3,880.** That figure came from the HAR, which captured a *cuisine-filtered* query. A full pass is roughly 3,070 requests and about 1h45m at the configured 2s delay - worth knowing before scheduling one - **Cloudflare blocks by IP reputation.** One ProtonVPN London exit was refused outright ("Sorry, you have been blocked" - a WAF block, no challenge), while another Proton London exit served 200. Since there is no challenge to solve, a headless browser or FlareSolverr does not help; reconnecting gluetun for a different exit does. README says so where someone hitting a 403 will look ## Test plan - 117 tests pass (72 unit, 45 integration) - Re-crawled live after the fix: 5 fetched, 5 changed, all genuine meal kits with 11-15 ingredients, every one carrying an amount, 6 steps each - Cross-source `search_recipes` for "chicken" over the MCP server returned results from **both** sources - 4 HelloFresh and 1 Gousto - with prep times and calories