nz-price-hub

nz-price-hub API

Base URL: https://hub.w1.nz. The API process listens on 127.0.0.1:8080. Caddy terminates TLS for hub.w1.nz and reverse-proxies to that port.

The read API is public. No credential is required. /admin is a separate password login for the operator dashboard.

Compression

Every JSON, HTML, and Markdown response is gzip-compressed when the client allows gzip, and also when Accept-Encoding is absent. The server sets Content-Encoding: gzip and Vary: Accept-Encoding.

The body is uncompressed only when the client sends Accept-Encoding: identity or sets gzip;q=0.

Use curl --compressed so the client stores a decompressed body. Responses include an ETag. A matching If-None-Match returns 304.

Rate limit

Each client IP has a token bucket of 10 requests per second with a burst of 40. The limiter trusts X-Forwarded-For only when the direct peer is loopback (Caddy on the same machine). Otherwise it uses the TCP peer address.

Over the limit the status is 429, the body is JSON {"error":"rate limit exceeded"}, and Retry-After is set.

limit query values are capped (prices and search at 500, store lists at 1000, nearest at 50). limit=0 means the cap.

Errors

StatusWhen
400Missing or invalid query parameter
404Unknown path
405Method not allowed
429Per-IP rate limit, or too many failed admin logins
401Admin password rejected
403Admin action rejected

Error bodies are JSON {"error":"..."} for /v1/* and /healthz.

GET /healthz

Liveness. Example: {"ok":true,"latest_date":"2026-10-13","stores":396,"price_rows":12000}.

GET /v1/dates

Days available on disk, newest first.

{"latest":"2026-10-13","dates":[{"date":"2026-10-13","rows":100,"stores":396,"rows_by_chain":{"paknsave":40}}]}

GET /v1/stores

Query: chain, region, q, lat, lng, limit.

chain is paknsave, newworld, or woolworths. region is an exact match, case-insensitive. q matches name, suburb, address, region, or store id. When both lat and lng are present, only stores that have coordinates are returned, sorted by haversine distance, with distance_km. Otherwise they are sorted by chain, then name, and stores without coordinates are included.

curl --compressed "https://hub.w1.nz/v1/stores?region=Auckland&limit=20"

Store object: chain, store_id, name, region, address, lat, lng, optional suburb, postcode, and distance_km when a point was given.

GET /v1/stores/nearest

Query: lat and lng required, optional chain, limit default 3.

curl --compressed "https://hub.w1.nz/v1/stores/nearest?lat=-36.748&lng=174.748&chain=woolworths&limit=3"

Same store objects as /v1/stores, nearest first. Stores without coordinates are omitted.

GET /v1/prices

Query: chain, store (id or name fragment), date (YYYY-MM-DD or omitted for latest), department (produce, meat, or all), q (name substring), limit (default 50), offset (default 0).

curl --compressed "https://hub.w1.nz/v1/prices?chain=paknsave&store=Albany&department=produce&q=apple&limit=20"
{"date":"2026-10-13","total":2,"limit":50,"offset":0,"prices":[{"chain":"paknsave","store_id":"...","name":"Apples","price":3.99,"unit_price":3.99,"unit":"kg"}]}

Price fields: chain, store_id, store_name, date, department, product_id, name, brand, size, price, unit_price, unit, promo, was_price, url, category, scraped_at, unit_or_size, price_per, raw_price_text. Empty optional fields are omitted.

GET /v1/search

Query: q required, plus chain, store, lat, lng, date, limit.

Without a point, the search covers every store in the selected day (still filtered by chain and store when set). With lat and lng, it uses the single nearest store of each chain that has price rows for that day, then applies chain if set. An explicit store wins over the point.

curl --compressed "https://hub.w1.nz/v1/search?q=milk&lat=-41.29&lng=174.78&limit=10"

Fresh data is produce and meat only, so a search only hits those departments. Results are sorted by unit price, then shelf price, then name.

GET /v1/compare

Query: q required, plus either lat and lng, or stores, or neither.

Winners are sorted by unit price when unit_price is greater than zero, otherwise by price.

curl --compressed "https://hub.w1.nz/v1/compare?q=banana&lat=-36.85&lng=174.76"
curl --compressed "https://hub.w1.nz/v1/compare?q=banana&stores=woolworths:2683184,paknsave:65defcf2-bc15-490e-a84f-1f13b769cd22"

GET /v1/runs

Recent scraper runs from runs.sqlite, newest first (up to 100).

{"runs":[{"run_id":1,"started":"2026-10-13T06:00:01+13:00","finished":"2026-10-13T06:04:10+13:00","store":"2683184","chain":"woolworths","status":"ok","rows":800,"error":"","duration":249.2}]}

status is one of:

GET / and GET /SKILL.md and GET /docs

/ is this service described as HTML for agents. /SKILL.md is the same text as text/markdown with YAML frontmatter (name, description). /docs is this reference as HTML.

Admin

/admin asks for the operator username and password from /etc/nzhub/admin.conf. The session cookie is HttpOnly, Secure, SameSite=Strict, and lasts 12 hours. Sign-out is a POST that must include the session CSRF value. Failed sign-in attempts are rate-limited per IP.

Pages, after sign-in:

Data

Daily snapshots are SQLite files prices-YYYY-MM-DD.sqlite under /var/lib/nzhub. Each file lists all New Zealand stores and the prices scraped that day. The Sydney host scrapes Woolworths at 06:00 Pacific/Auckland. PAK'nSAVE and New World caches are imported from /var/lib/nzhub/incoming/*.sqlite (moved to incoming/done/ after a successful read). Call counts live in stats.sqlite. Run rows live in runs.sqlite.

Skill install

mkdir -p ~/.codex/skills/nz-price-hub ~/.claude/skills/nz-price-hub .agents/skills/nz-price-hub
curl --compressed -fsSL -o ~/.codex/skills/nz-price-hub/SKILL.md https://hub.w1.nz/SKILL.md
cp ~/.codex/skills/nz-price-hub/SKILL.md ~/.claude/skills/nz-price-hub/SKILL.md
cp ~/.codex/skills/nz-price-hub/SKILL.md .agents/skills/nz-price-hub/SKILL.md