Mobile Proxy Government Data
Public procurement is the commercially valuable half of government data, and it is aggressively national: every country runs its own tender portal, publishes to its own thresholds, and expects a domestic visitor.
- Tenders are national first — the European TED service aggregates above threshold, and everything below it stays on the national portal.
- Deadlines make cadence non-negotiable — a tender found after its closing date is worth nothing at all.
- Open data portals want to be used — where an API exists, use it; a crawl is the worse option for both sides.
- Registers contain people — company officers and beneficial owners are personal data even in a public register.
Reach each country’s procurement service as a local visitor.
Daily collection that catches a tender in time to bid.
Government Data Access
Government data has a division worth understanding before writing any code, because getting it wrong means building a crawler for something that was already available as a download.
Open data services generally publish bulk files and documented interfaces precisely so that nobody has to crawl the website. Using them is faster, more complete, more stable and considerably kinder than parsing pages, and where one exists it is almost always the right answer. Collection belongs to the material that genuinely only exists as a web page — which, for procurement in particular, is a surprising amount of it.
The second thing to know is that national portals are built for domestic suppliers and behave accordingly. Search interfaces, document downloads and notice archives are frequently degraded or restricted for foreign address ranges, and several apply harder bot mitigation to them for the entirely reasonable reason that they see almost no legitimate domestic traffic from those ranges. Collecting each country from inside it removes a whole class of intermittent, hard-to-attribute failure.
| Material | Best route | Why |
|---|---|---|
| Statistical releases | Bulk download or documented interface | Published for reuse; crawling adds nothing but load |
| Above-threshold tenders | The European Tenders Electronic Daily service | Large contracts must be advertised there, so coverage is reliable |
| Below-threshold tenders | National and regional portals, collected locally | Most opportunities by count, and published nowhere else |
| Award notices and contract records | Portal, with the document attachments | The commercially useful detail is in the attachments, not the notice |
Why the portal breaks before the proxy does
National procurement portals rarely run deliberate anti-bot defenses in the way a commercial platform does. What they run instead is aging, fragmented infrastructure: search endpoints built for a human clicking through a handful of results per session, session cookies that expire mid-crawl, and per-IP rate limits set low because nobody expected sustained automated traffic from a single address. A residential or datacenter IP hitting that infrastructure from outside the country gets treated as the exception it is. A carrier IP with the country's own mobile network behind it looks like the ordinary traffic the portal was built to serve, and inherits none of the throttling aimed at foreign ranges.
Public Records Scraping
Company registers, beneficial ownership registers, licensing records and planning applications are the other half of this vertical, and they behave like the national portals above: one per country, each with its own model, several with deliberate limits on bulk extraction.
That last point is the one that catches people. Public availability is not the same as unrestricted reuse. Several jurisdictions place explicit limits on bulk extraction and republication of register data, and registers routinely name individuals — company officers, beneficial owners, licence holders — which makes them personal data regardless of the register being open to anyone. Purpose and retention matter, and it is a question for your own legal advice rather than a technical setting.
Beneficial ownership access has gotten narrower, not wider
Anyone who built a beneficial-ownership collector before late 2022 needs to check it still works, because a court ruling that year found that unrestricted public access to that specific class of register was disproportionate to its anti-money-laundering purpose. General public search was withdrawn or gated across the affected jurisdictions, while access for parties who can show a legitimate interest — journalists, banks doing due diligence — was kept, usually behind a registration step. A scraper built against the older open version will find a login wall where a register used to be, and the fix is registering proper access, not routing around the login.
Tender and Contract Monitoring
Procurement is the commercially valuable core of government data, and it is the one genuinely deadline-driven use case in this entire cluster. A tender notice has a closing date. A notice discovered after that date is worth precisely nothing — unlike a price or a ranking, there is no partial credit for being late, and no way to backfill the opportunity.
That single fact justifies a tighter cadence here than anywhere else a business is collecting. Daily is the floor for markets you actively bid into, and same-day discovery is a reasonable target given that response windows are frequently measured in a small number of weeks.
notice_id … portal ted | national | regional market FR published_at … closing_at … # the field the whole pipeline exists for cpv_codes [ … ] # classification, for routing to the right team buyer … value_band … attachments [ … ] # the specification usually lives here first_seen 2026-08-24T09:14:02Z
Classification is the other practical difficulty. Common procurement vocabulary codes make routing tractable within Europe, but national portals vary in how consistently they apply them, and below-threshold notices are frequently classified loosely or not at all. Most working systems end up combining code matching with keyword matching rather than trusting either alone.
TED has a real API — use it, do not crawl it
The Tenders Electronic Daily service publishes above-threshold EU notices through a documented search API that needs no authentication for read-only queries and returns structured records filterable by country, buyer, CPV code and date. Roughly two thousand four hundred notices land there on a typical weekday. None of that needs a proxy of any kind: it is a public API built for exactly this kind of polling. The proxy budget for this cluster belongs to the below-threshold national portals in the table above, which is also where competition per notice tends to be lowest.
Award notices matter more than the tender itself
A closing date is worth tracking so you do not miss the chance to bid. An award notice is worth tracking so you learn who won, at what price, and how many bidders competed. A contract re-awarded to the same incumbent at a similar price with a single bidder is a market you probably cannot win on price; one with five bidders and a wide price spread says the buyer is genuinely open. Award notices publish later, on their own schedule, so they need their own polling cadence rather than riding along with the notice that preceded them.
Regulatory Data Collection
The regulatory half moves more slowly and matters to a different audience: businesses trying to remain compliant across several markets at once. Gazettes, regulator publications, consultations, guidance and enforcement decisions all follow publication schedules, which makes them well suited to index polling rather than crawling.
- Poll the index, fetch what is new — Publication is scheduled and sequential. There is no reason to re-fetch an archive.
- Collect consultations, not just outcomes — Drafts and responses signal direction months ahead of the final instrument.
- Store the document, derive the fields — Formats change; a raw archive lets you re-extract without re-collecting.
- Keep the source language — Translate downstream. A translated regulatory text is a summary and should never be the record of what was published.
- Run off-peak and single-threaded — These are public services on modest budgets. Patient collection is both correct and, because it never provokes a defence, more reliable.
For court and case law sources see legal data, and for the general collection discipline, web scraping best practices.
Reach Every National Procurement Portal
Live PXM2 locations — pick the countries whose tenders and records you monitor and collect from inside each:
France
India
Poland
Frequently Asked Questions
Where are public tenders published?
In layers. In Europe, contracts above defined value thresholds must be advertised through the Tenders Electronic Daily service, which makes it an excellent single starting point for large opportunities. Everything below those thresholds — which is the majority by count and often the more winnable work — is published only on national and regional portals, each with its own search interface, notice format and classification scheme.
Why does a local exit help for government sources?
National portals are built for domestic suppliers and behave that way. Search interfaces, document downloads and notice archives are frequently degraded or restricted for foreign address ranges, and several apply harder bot mitigation to them for the straightforward reason that they see little legitimate domestic traffic from those ranges. Collecting each country from inside it removes an entire class of intermittent, hard-to-diagnose failure.
How quickly does tender data have to be collected?
Daily, and it is the one genuinely deadline-driven vertical in this cluster. A tender notice has a closing date, and a notice discovered after it has passed has no value whatsoever — unlike a price or a ranking, there is no partial credit for being late. That single fact usually justifies a tighter cadence here than anywhere else a business is collecting.
Should I be scraping open data portals at all?
Usually not. Government open data services generally publish bulk downloads and documented interfaces specifically so that nobody has to crawl the website, and using them is faster, more complete and more stable than parsing pages. Reserve collection for the material that is genuinely only available as a web page — which, for procurement, is a surprising amount of it.
Are public registers free of data protection concerns?
No. Company registers, beneficial ownership registers and procurement award notices routinely name individuals, and public availability is not the same as unrestricted reuse — several jurisdictions place explicit limits on bulk extraction and republication of register data. Purpose and retention matter, and it is a question for your own legal advice rather than a technical setting.
Related Mobile Proxy Guides
Procurement and legal publishing are the two public-sector halves of this cluster and are usually run together.
Business use cases
Core mobile proxy guides
Catch Every Tender in Time
Dedicated 4G/5G modems with unlimited bandwidth and unlimited rotations — carrier IPs inside every market whose procurement you monitor.
Get a Mobile Proxy