My favourite MCPs: Apify

Some data can't be fetched, only harvested. Apify is the one I reach for when a normal page grab won't work — and the check I run before I bother.

Not sure what an MCP server is or how it’s used? Check out my explainer here.

Apify is a marketplace of ready-made scrapers. Someone else has already built and maintained the thing that pulls listings off a property site or profiles off a professional network, and you rent it by the run. They call them Actors; think of an Actor as a small robot you hire for thirty seconds. The MCP server turns any of them into a tool an assistant can call mid-conversation.

The reason it earns a place alongside plain web search is that some data can’t be fetched politely. Sites behind logins, pages that only load as you scroll, anti-robot checks, anything that needs a session. A normal fetch reads one page. Apify collects a hundred and hands back records with the same fields in every row, which is a spreadsheet rather than a wall of text.

I spent about a day on it this week and got four things out of it, in ascending order of usefulness.

The first was a reusable delivery path. A scrape finishes, it fires a webhook, my self-hosted n8n checks the request is really mine, fetches the results, formats them and sends them to Telegram. Built once. Every use since has been a new task, a schedule and a webhook — about ten minutes of clicking and no new code. Six scrapes now run every three hours for under five cents a day.

The second was proving the security gate could fail before I trusted it. Each webhook carries a shared secret and the pipe drops anything whose secret doesn’t match. So I fired one with the wrong secret and watched it get rejected. Then I fired one with the right secret and a deliberately fake results ID, which was admitted and then failed at the fetch with “not found” rather than “not authorised” — proving the credential underneath worked too. A check you have only ever seen pass tells you nothing.

The third stung. Five weeks ago I deleted a scheduled scan that watched for a particular kind of freelance listing, because the free plan’s result allowance was spent and it could no longer return anything. The paid plan removed exactly that blocker on day one, and nothing in my system ever went back to look at something switched off for a reason that had since expired. Five weeks dark. It runs again now, and I write the un-pause condition down next to anything I turn off.

The fourth is the check that saves the most time, because it stops work rather than starting it. Look at the user counts on the Actors in your niche before you commit. I nearly built a lane around Spanish government data: ten relevant Actors existed, median about three and a half users each. The Actor for one big Spanish property portal had 1,859. That gap is the market telling you nobody scrapes open government data, because it’s already published cleanly. The lane closed before it cost me a week.

Making it repeatable

The one-off ask is direct:

Run the profile scraper on this URL and give me the career history as a table, with dates.

The standing version lives in a skill — a text file of instructions the assistant reads when the job comes up, in plain English rather than code:

Before scraping anything:
- Check the Actor's user count in the store first. Low usage
  usually means the data is already published cleanly. Say so
  and stop.
- Prefer a maintained Actor over writing a scraper.
- Schedule it and deliver the result to me. Don't leave a job
  that depends on me remembering to run it.
- Every webhook carries the shared secret, and I want the decoy
  test run before we trust the gate.
- Tell me what the run cost.

The first rule is doing the most work there. It’s the only one that regularly ends with the assistant telling me not to bother, which is the outcome I most need and am least likely to reach on my own enthusiasm.

Apify’s free plan comes with $5 of credit and no card required, and compute bills at twenty cents a unit. Run one Actor by hand in their console first; if it gives you what you want, then wire up the MCP server.

The scrapes ran at six this morning while I was asleep. That’s the whole change.