Mobile Proxy Energy Data
Energy splits neatly in two: wholesale markets that publish extensively through transparency platforms, and retail tariffs that will not tell you anything without a local postcode. They need completely different approaches.
- Wholesale is published deliberately — transparency platforms and exchanges exist to make market data available.
- Retail tariffs need a postcode — supply offers are regional, and comparison sites ask before they answer.
- Regulators publish per market — price caps, standing charges and licence conditions are national.
- Use the API where there is one — crawling a transparency platform that offers downloads is the wrong choice.
Reach each market’s regulator and suppliers directly.
Collect the offers a local household is actually shown.
The Two Halves of Energy Data
Energy divides more cleanly than any other vertical in this cluster, and the two halves need completely different approaches. Wholesale market data is published deliberately, extensively and for reuse. Retail tariff data is regional, gated behind a postcode, and behaves like store-level retail.
Getting this division right at the start saves most of the effort, because the mistake in each direction is expensive: crawling a platform that offers bulk downloads, or trying to collect regional supply offers without a local address and wondering why the answers are thin.
The split also decides how the work should be staffed. Wholesale ingestion is an integration task: register for API access, respect the published rate limit, and back off predictably. Retail collection is an operations task: a pool of exits mapped to postcodes, watched for coverage gaps, and refreshed on the market's own cadence rather than a fixed interval.
Wholesale Markets and Transparency Platforms
European transmission system operators report generation, load, cross-border flows and outages through a central transparency platform, national operators publish their own detail, and the day-ahead power exchanges publish settled prices. In the United States the federal energy information agency plays a comparable role. All of it exists to be reused.
Where a platform publishes bulk files or a documented interface, use it. Crawling a service that is actively trying to hand you the data is the clearest example in this whole cluster of choosing the wrong tool — slower, less complete, more fragile, and worse for both parties.
Which Platforms Publish What
The ENTSO-E Transparency Platform is the anchor for the European Union: generation by fuel type, cross-border flows, day-ahead prices and outage notices for every member state through one registered API. Nord Pool runs the Nordic and Baltic auctions separately, and national operators such as National Grid ESO in Great Britain and RTE in France add their own balancing and system detail on top. In the United States the pattern is regional rather than national: the federal Energy Information Administration publishes aggregated statistics, while each independent system operator — PJM, ERCOT, CAISO, MISO and NYISO — runs its own feed for day-ahead and real-time prices inside its own footprint.
Day-Ahead vs Real-Time
Day-ahead prices settle in a single auction that closes the afternoon before delivery, one price per hour per zone. Real-time prices settle every five to fifteen minutes as the grid operator corrects for what actually happened against the forecast, and in the American markets they are also split by location — a price at one substation can differ from a price thirty miles away when a transmission line between them is congested. A feed carrying only the day-ahead number misses the volatility that time-of-use tariffs and battery-dispatch algorithms are built to react to.
Retail Tariffs and Regional Offers
The retail half is where collection genuinely applies, and where a local exit matters. Domestic and business supply offers are regional, so comparison services and suppliers ask for a postcode before quoting anything, and several restrict or degrade access from foreign addresses entirely. Collecting what a household in a particular region is actually offered means asking from that market with a local postcode — the same pattern as store-level retail data.
supplier … tariff_name … market GB region … # network region, not just country postcode_used … unit_rate … # per kWh, with currency standing_charge … # the component that gets forgotten contract_length … observed_at 2026-08-24T09:14:02Z
Time-of-Use and Dynamic Tariffs
Smart-meter-driven tariffs reprice electricity by the half hour rather than by a flat annual rate, tracking the wholesale market up or down within the same day. Collecting one correctly means pulling the full rate curve for the day on its publication schedule, not a single snapshot — a tariff that looks cheap on average can carry a handful of expensive peak-hour blocks a daily check will never see. The publication window is itself regional: a dynamic tariff for a Nordic household follows Nord Pool's schedule, not the supplier's own clock.
Exit Fees and Contract Terms
A unit rate and a standing charge describe what a customer pays while on a tariff, not what it costs to leave one. Exit fees, minimum contract lengths and renewal terms usually sit on a separate quote page or PDF rather than the comparison summary, and a dataset built only from headline rates will misrank a tariff that looks cheapest until the exit fee is added back in. Business energy complicates this further: many suppliers only quote a business after a sign-in or a broker referral, so a business tariff observation often needs a persisted session behind the local exit rather than one stateless request.
EV Charging Networks and Session Pricing
Public EV charging sits next to retail energy rather than inside it, and it has its own access pattern. A charging network's app or map shows price per kWh or per minute, connector availability and any roaming surcharge, and all three are gated by location the same way a retail tariff is — a session priced for a network's home market can differ from the price the same charger shows a roaming account, and some networks will not resolve pricing for a session started from outside the country at all.
Two things make this noisier than retail tariffs. The billing model itself varies by jurisdiction — some regions require per-kWh billing by law, others still allow per-minute billing, and a network operating across a border runs both on the same hardware depending on where the charger sits. Availability and price also move together in real time as a queue forms at a popular site, so one page load describes a moment rather than a stable rate. A dataset worth building here samples repeatedly through peak hours from a local exit in each country the network covers.
Regulatory and Network Data
The regulatory layer moves the retail market directly and is cheap to follow. Price caps and their revisions, standing charge components, licence conditions, network charges, and supplier failure or transfer notices are all national, published by the national regulator, and low volume. A cap revision reprices an entire country at once, which makes it the single highest-value thing to be watching.
Network Charges and Grid Connection Data
Distribution network charges are set regionally even when the retail tariff is priced nationally, so two customers on the same supplier and the same headline rate can end up with different totals once the local network component is added. Grid connection queues are the newer addition to this layer: as renewable and battery projects queue for capacity, several transmission operators now publish queue status and constraint data alongside their existing outage feeds, and that queue position is itself commercially relevant to anyone tracking new generation capacity in a region.
Cadence follows the events rather than a clock. A weekly sweep of retail tariffs covers ordinary movement, with tighter attention around a known regulatory date. Wholesale figures update constantly, but that is precisely the part you should be taking from a published feed rather than collecting.
For the neighbouring regulated sources see government data, and for the same postcode-gated pattern in shops, retail data.
Collect Each Energy Market Locally
Live PXM2 locations — pick the markets whose tariffs and regulatory data you follow:
France
India
Poland
Frequently Asked Questions
Where does wholesale energy data come from?
From platforms built to publish it. European transmission system operators report generation, load, cross-border flows and outages through a central transparency platform, national operators publish their own detail, and the day-ahead power exchanges publish settled prices. In the United States the federal energy information agency plays a comparable role. All of it is intended for reuse, and most of it offers structured downloads or interfaces.
So why would I need a proxy for energy data?
For the retail half, which behaves nothing like the wholesale half. Domestic and business supply offers are regional, and comparison services and suppliers ask for a postcode before quoting anything. Several also restrict or degrade access from foreign addresses. Collecting what a household in a particular region is actually offered means asking from that market with a local postcode — the same pattern as store-level retail.
Should transparency platforms be crawled?
No, and this is worth being firm about. Where a platform publishes bulk files or a documented interface, using it is faster, more complete, more stable and considerably kinder than parsing pages. Crawling a service that is actively trying to hand you the data is the clearest example in this whole cluster of choosing the wrong tool.
What regulatory material is worth tracking?
Price caps and their revisions, standing charge components, licence conditions, network charges, and supplier failure or transfer notices. These are national, published by the national regulator, and they move the retail market directly — a cap revision reprices an entire country. They are also low-volume and predictable, which makes them cheap to follow properly.
How often does energy data change?
It depends entirely on which half. Wholesale prices settle daily and intraday data moves constantly, but that is the part you should be taking from a published feed rather than collecting. Retail tariffs change on a scale of weeks and around regulatory events, so a weekly sweep with tighter attention around a cap revision covers it comfortably.
Related Mobile Proxy Guides
Energy retail behaves like store-level retail; energy regulation behaves like the rest of the public sector.
Business use cases
Core mobile proxy guides
Quote Tariffs From the Right Region
Dedicated 4G/5G modems with unlimited bandwidth and unlimited rotations — carrier IPs that let a local postcode return a real answer.
Get a Mobile Proxy