⬡ Harvester
⬡ Harvester

بوت

Boots 51 listings · schema about 19 hours old

Use with AI

Paste this into Claude or any AI agent that can make web requests. It explains the public API — endpoints, parameters, limits — so the agent can search listings on its own. Also at /api/v1/prompt. 

You can use the Harvester listings API: a public, read-only JSON API over products
scraped from online shops (mostly Jordanian). Call it with plain HTTP GET requests.

Base URL: https://harvester.mehyar.ing/api/v1
No authentication, no API key. GET only. Every response is JSON, errors included.

## Endpoints

GET /api/v1/meta
  Describes the API itself: every endpoint and parameter, legal values, the limits,
  and live corpus statistics (how many listings and sites, the typical price range).
  Call this first — it is always current.

GET /api/v1/sites
  Every shop that has live listings: id, name, domain, status, listing_count.
  Use it to turn a shop name into a site_id.

GET /api/v1/listings
  Search and filter live listings (delisted ones are excluded). Parameters, all optional:
    q            Free-text search over title and description. Truncated to 200 characters.
                 Arabic text is matched word by word (first 8 words only) and is slower — keep it short.
    search_mode  How q matches: exact, any, all, fuzzy. Default all.
                   all   = every word must appear (narrowest)
                   any   = at least one word
                   exact = the phrase as a substring
                   fuzzy = typo-tolerant match on the title, any script
    site_id      One id or a comma-separated list (from /api/v1/sites).
    site         Substring of the shop's domain, e.g. site=opensooq. ANDed with site_id.
    category_id  One id or a comma-separated list. Ids appear on each listing.
    min_price    Inclusive lower bound on the raw price number.
    max_price    Inclusive upper bound on the raw price number.
    sort         relevance, recent, oldest, price_asc, price_desc. Default: relevance when q is given, else recent.
                 Listings without a price sort last.
    limit        Page size. Default 25, max 100 (larger values are clamped).
    offset       Rows to skip. Max 10000 (larger values are clamped).
    include      Comma-separated extra fields. Only value: raw_data (see below).
  Returns { meta: {...}, listings: [...] }. meta.returned is the row count of this page.

GET /api/v1/listings/:id
  One listing in full: untruncated description, raw_price (the original price text),
  availability, price_history, lowest/highest price seen, active, removed_at, and raw_data.
  Works for delisted listings too — check `active`.

GET /api/v1/prompt
  This text, as text/plain.

## Examples

  https://harvester.mehyar.ing/api/v1/meta
  https://harvester.mehyar.ing/api/v1/sites
  https://harvester.mehyar.ing/api/v1/listings?q=iphone&sort=price_asc&limit=10
  https://harvester.mehyar.ing/api/v1/listings?q=لابتوب&site=opensooq&max_price=400
  https://harvester.mehyar.ing/api/v1/listings?q=makita+drill&search_mode=fuzzy
  https://harvester.mehyar.ing/api/v1/listings?site=opensooq&q=تويوتا&include=raw_data&limit=20
  https://harvester.mehyar.ing/api/v1/listings/12345

## The listing object

id, title, price, currency, url, description, seller, condition, images, site {id, name,
domain, status}, category {id, name}, first_seen_at, last_seen_at.
Any field except id, title and url can be null — shops don't all publish everything.

- currency is per listing and can be null. Never compare or sum prices whose currency
  differs or is null. min_price/max_price ignore currency.
- In list responses description is cut to 400 characters and
  description_truncated is true; fetch /api/v1/listings/:id for the full text.
- This is scraped data, not a live shop. A price is what the shop showed at last_seen_at.
- site.status describes our scraper, not the data. A "broken" site's listings are still
  valid — don't filter them out.
- category is null when a listing came from a whole-catalogue scrape.
- Always give the user the listing's url; it is the real product page.

## raw_data

Everything the scraper captured that has no dedicated field. Always on
/api/v1/listings/:id; on /api/v1/listings only with include=raw_data (it is large,
so ask for it only when you need it, and prefer small pages when you do).
Its shape depends on the site. Sensitive values (e.g. phone reveal tokens) are removed.

For OpenSooq (domain jo.opensooq.com) raw_data holds:
  specs        { fieldName: value in Arabic } — the listing's real attributes.
  specs_en     { fieldName: English label } — same fields, English.
  spec_labels  { fieldName: Arabic name of the field }
  breadcrumbs  category path, e.g. ["الأردن", "سيارات ومركبات", "آليات ثقيلة", "جرافة"]
  seller_id, seller_rating, seller_listings, seller_member_since, seller_is_shop,
  phone_masked (last digits hidden), listing_id, lastmod, description_ld
  payload      the full listing object from the OpenSooq page, for anything else.

TRAP: in specs_en, numeric fields (mileage, year ranges, sizes, …) are SEARCH BUCKETS,
not values — e.g. "+200,000" for mileage. For real numbers read the same fieldName from
specs, which holds the actual value (in Arabic; digits may be Arabic-Indic or Latin).
Use specs_en only to understand what a field means. seller_is_shop is unreliable.

## Limits and errors

- Rate limit: 60 requests per 60 seconds per IP. Over it you get HTTP 429
  with a Retry-After header: stop and wait that many seconds before the next request.
- A query running over 5 seconds is cancelled with HTTP 503 code=query_too_expensive.
  Don't retry it as-is — narrow it (add site_id/category_id, fewer words, or search_mode=fuzzy).
- meta.total_estimate stops counting at 1000. If meta.total_is_capped is true, the real
  total is at least that — say "1000+", never an exact number.
- 404 = unknown id or path, 400 = malformed parameter. Error bodies are
  { error: { code, message } }; the message says what to do.

## How to work well

1. Start with /api/v1/meta (and /api/v1/sites if the user names a shop).
2. Search with q in the default mode. Nothing back? Try search_mode=any, then fuzzy —
   most misses are a typo, a synonym, or English words against Arabic titles.
3. Too much back? Narrow with site, site_id, category_id or a price bound rather than
   paging deeper. You cannot page past offset 10000.
4. Paginate politely: one request at a time, never in parallel. Use limit=100 when
   scanning and a small limit when you only need examples. Stop as soon as you have
   what you need.
5. Reuse results you already have instead of re-fetching them.