Record the product and the observation
A product name is a starting point, not a dependable matching key. Variants such as size, pack count, capacity, and model can change the meaning of a price. Define whether the identifier is supplied by the source or created for the agreed dataset.
Keep the observation separate from the product identity. Price, availability and promotion describe a source at a particular time. A source URL and timestamp help the receiving team investigate a discrepancy and distinguish a recent observation from an older one.
Make missing values interpretable
In the example below, the second product is out of stock and its price is null. Null means there is no usable price observation in this illustrative record; it does not mean the price is zero. The feed carries a currency code so that the numeric value has a defined unit.
Do not infer availability from a missing price, or a source failure from an empty file. For a real engagement, define field-level null semantics, record-level exceptions and run-level status separately. A delivery can arrive successfully while collection is incomplete.
| Product / variant | Price | Availability | Observed (UTC) |
|---|---|---|---|
| Luma bottle / 500 ml | USD 49 | in stock | 2026-10-03T09:00:00Z |
| Luma bottle / 750 ml | Not observed | out of stock | 2026-10-03T09:00:00Z |
Inspect the exact JSON records
[
{
"product_id": "luma-500",
"product": "Luma bottle",
"variant": "500 ml",
"price": 49,
"currency": "USD",
"availability": "in_stock",
"source_url": "https://shop.example.invalid/products/luma-500",
"observed_at": "2026-10-03T09:00:00Z"
},
{
"product_id": "luma-750",
"product": "Luma bottle",
"variant": "750 ml",
"price": null,
"currency": "USD",
"availability": "out_of_stock",
"source_url": "https://shop.example.invalid/products/luma-750",
"observed_at": "2026-10-03T09:00:00Z"
}
]Validate the contract before integration
Review records for identity collisions, expected types, currency, timestamp format, required fields, and variant consistency. Compare a representative sample with its source pages. Agree whether the delivery is a full snapshot or an incremental update, and how deletions or discontinued products will be represented.
CSV and JSON can express the same observations, but conversion needs a documented policy. This example uses an empty CSV cell for a JSON null; a real receiving system must agree that convention. The downloads are fictional teaching examples, not a live feed or a representation of a customer project.
- Define record identity and update semantics
- Specify currency, units and timezone
- Agree missing-value and exception handling
- Confirm expected totals and validation checks
- Test the receiving system with accepted samples
Illustrative data and templates explain the method. They do not establish source coverage, a customer outcome, or an agreed delivery commitment.
Explore the relevant services
Price intelligence ↗ — Compare offers with the context that matters: product variants, promotions, availability, and delivery charges. Turn scattered prices into a useful view of your market.
Custom web crawling ↗ — Commission a web data collection service around your sources, fields, and delivery requirements. From a defined extraction to an ongoing data feed, agree the records your team needs and how they should arrive.