How to track competitor hiring with a scheduled weekly scan

Last updated

To track competitor hiring, put your competitors' names into companies mode on Datapika's Indeed and ZipRecruiter scraper and schedule the run once a week on Apify. Each company comes back with a yes or no has_active_postings signal, verified posting counts split by board, the live ZipRecruiter job total (593 for the sample company in the actor README), a firmographic profile, and every job posted inside a window you can stretch to 720 hours. Records cost $0.0005 each, company and job rows alike, so the scan is cheap and the week-over-week diff of totals and titles is the intelligence.

What does a weekly scan of a named competitor return?

One company record per name, plus one flat row per job when emitJobRecords is on. The record opens with company_query and matched_company_name, so you can see which employer the boards resolved your input to. has_active_postings answers the yes or no question, active_jobs_by_site splits the employer-verified postings between Indeed and ZipRecruiter, and active_jobs_total_by_site.zip_recruiter reads the company's full open count from its ZipRecruiter page rather than the sample this run pulled. Indeed exposes no such total, so that slot is null. posted_last_48h_count tells you how many postings landed inside your recency window, which you set with hoursOld. Under profile sit industry, employee count, revenue band, headquarters, founding year and employee rating, with Indeed keys and ZipRecruiter keys side by side. latest_jobs nests the postings with title, location, salary fields, job_type, is_remote, job_level, posted_at and reposted_at. sites_with_errors names any board that failed for that company, and scraped_at stamps the run in UTC so each week's dataset sorts cleanly against the last.

  • Sample record in the README: 17 Indeed and 21 ZipRecruiter postings found, live ZipRecruiter total 593, 17 posted inside the window
  • Profile keys include company_industry, company_num_employees, company_revenue, zr_rating, zr_headquarters and zr_year_founded
  • Each nested job also lands as a flat record_type: job row for spreadsheet-friendly exports
  • posted_at is hour-precise on ZipRecruiter and day-precise on Indeed
  • sites_with_errors lets you rerun only the companies where one board failed

How do you schedule a weekly competitor scan on Apify?

Start with the companies input. Write each competitor as name and website, for example Kelly Services | kellyservices.com, because domain matching stops a generic name from pulling in unrelated employers; the normalized-name fallback is controlled by matchThreshold, default 0.82. Leave scanMode on companies and recencyMode on last48h, then set hoursOld to 168 so a run captures everything posted since the previous one; the field accepts 1 to 720 hours. maxJobsPerCompany caps the rows per company per board at 50 by default and 200 at most, which is where you control dataset size. Keep both sites enabled unless one board is irrelevant to your market, set countryIndeed if your competitors hire outside the US, and keep the residential proxy, since ZipRecruiter rate-limits datacenter IPs. Save the input, then create an Apify schedule with a weekly cron and attach a webhook that fires on run completion. A ten-company run with the default window finishes in a few minutes; split lists of several hundred names across schedules so each run stays inside the 60-minute default timeout.

  • companies: Name | website per line, or {name, website} objects through the API
  • hoursOld 168 with recencyMode last48h covers a full week between runs; latest10 returns the 10 newest postings regardless of age
  • maxJobsPerCompany: 50 default, 200 maximum, per company per board
  • countryIndeed accepts usa, uk, canada and others; ZipRecruiter covers the US and Canada
  • Turn includeDescription on only if you need full job text, since it adds latency on Indeed
  • Weekly schedule plus completion webhook makes the run hands-off after the first setup

How do you diff two weekly runs to spot a hiring shift?

Every run writes its own dataset, so the comparison is a join on company_query between this week's export and last week's. Three deltas carry most of the meaning. The change in active_jobs_total_by_site.zip_recruiter is the volume signal: a jump means new requisitions were approved, a drop means roles closed or a freeze started. posted_last_48h_count, with the window set to 168 hours, counts only what appeared since your last run, so it doubles as a velocity number without any date math on your side. The third delta is in the job rows: new title strings that did not exist last week, new locations, and new values in job_level. Filter out rows that carry reposted_at before counting, because a bumped ad is not a new opening and would inflate the week's total. Keep scraped_at on every row and append each run to one table; after a quarter you have a dated series per competitor that a pivot or a notebook can chart directly.

  • Join key: company_query, which echoes the exact name you submitted
  • Volume delta: active_jobs_total_by_site.zip_recruiter this week minus last week
  • Velocity: posted_last_48h_count with hoursOld at 168, no date parsing needed
  • Novelty: titles, locations and job_level values absent from the previous run
  • Exclude rows with reposted_at from new-opening counts; track them separately as hard-to-fill roles
  • Use the companies and jobs dataset views for pre-flattened tables before joining

Which week-over-week changes are worth acting on?

Counts on their own tell you that something moved; the job rows tell you what. A competitor whose first posting in a new country or metro shows up in location is opening a market, and it usually appears there before any announcement. A cluster of engineering titles that share a product name points to a build in progress, while a batch of account executive or customer success roles points to a go-to-market push. job_level shifting upward, with director and VP titles appearing where individual contributor roles used to be, suggests a new function being stood up. salary_min and salary_max on each row put a number on how aggressively they are bidding for talent against your own bands. On the negative side, a falling live total with no fresh postings for two consecutive runs reads as a freeze, and reposted_at stacking up on the same title reads as a role they cannot fill. The profile block adds the employee rating, which moves slowly but frames retention.

  • New location value for a competitor: market entry, so check which function leads the titles
  • Cluster of titles sharing a product or platform name: a build underway
  • Sales and customer success roles appearing together: a go-to-market push
  • job_level moving to director and above: a new function or leadership layer
  • Live total falling for two runs with zero fresh postings: freeze or restructuring
  • reposted_at repeating on one title: a role they are struggling to fill

How do you get the weekly diff into a dashboard, CRM or AI agent?

The run finishes, the webhook fires, and the dataset is ready. The REST endpoint for dataset items returns JSON, CSV or XLSX, and the jobs and companies views give you two pre-flattened tables ready for a warehouse or a Google Sheet. The Python and JavaScript clients wrap the same call for teams that want to run the diff in a notebook or a cron job. On the no-code side, the actor's Integrations tab on Apify lists Zapier, Make, Google Sheets, Airtable and Slack, so a Slack post per competitor whose live total jumped is a few clicks. For agents, the scraper is exposed as an MCP tool at mcp.apify.com, which lets a Claude or ChatGPT based assistant answer a question like which of these competitors opened new roles this week by running the scan and reading the rows. Pay-per-call flows over X402 and MPP are supported for agents without an Apify subscription. Combine it with the other job boards in the Datapika catalog at /scrape when a competitor hires mostly through LinkedIn or Glassdoor.

  • REST: dataset items endpoint with format=json, csv or xlsx, plus the jobs and companies views
  • Python: pip install apify-client, then actor('openclawai/indeed-ziprecruiter-scraper').call(run_input={...})
  • MCP: https://mcp.apify.com/?tools=fetch-actor-details,openclawai/indeed-ziprecruiter-scraper
  • No-code: Zapier, Make, Google Sheets, Airtable and Slack from the Integrations tab
  • Webhook on run completion triggers the diff step or the Slack alert
Weekly diff signals from each company record and the decision each one supports
FieldWeek-over-week changeWhat it usually meansDecision
active_jobs_total_by_site.zip_recruiterRisesNew requisitions approved; expansion underwayPull the new titles and locations, brief sales and product
active_jobs_total_by_site.zip_recruiterFalls with no fresh postingsFreeze, restructuring or roles filled in bulkWait for two consecutive runs before calling it a freeze
posted_last_48h_count (hoursOld 168)Moves from 0 to positiveHiring resumed or a new team spun upRead that week's titles; this is the earliest signal
latest_jobs[].locationNew city or country appearsMarket entry or new officeCheck whether sales or engineering leads the entry
latest_jobs[].titleCluster sharing a product or platform nameBuild in progressCompare against your own roadmap
latest_jobs[].job_levelDirector and above appearNew function or leadership layerTrack the executive hire for later announcements
latest_jobs[].salary_min / salary_maxRanges above your bands for the same titleAggressive bidding for talentFeed into compensation review
reposted_atSame title bumped across runsHard-to-fill role, not new demandExclude from new-opening counts; flag as a sourcing gap
has_active_postingstrue to falseNothing live on either boardConfirm sites_with_errors is empty before concluding they stopped hiring
profile.zr_ratingDrifts downEmployee sentiment fallingPair with rising repost counts as an attrition indicator

How to do it

  1. 1.Open https://apify.com/openclawai/indeed-ziprecruiter-scraper, keep scanMode on companies, and paste your competitor list one per line as Name | website so employer matching resolves to the right company.
  2. 2.Set hoursOld to 168 with recencyMode last48h so each run covers the week since the last one, leave maxJobsPerCompany at 50 or raise it toward 200 for heavy hirers, set countryIndeed for non-US competitors, and keep the residential proxy.
  3. 3.Run once to confirm matched_company_name is correct for every competitor, then create an Apify schedule with a weekly cron and a webhook on completion.
  4. 4.After each run, pull the companies and jobs dataset views, join on company_query to the previous week, and act on rises in the live ZipRecruiter total and on new titles and locations, excluding rows with reposted_at.
Scan your target companies

Questions, answered

How often should I scan competitor job postings?

Weekly is the practical cadence for competitor tracking. Indeed timestamps postings the same day, but ZipRecruiter ingests in daily batches, so its newest postings are typically 12 to 48 hours old; a 168-hour window on a weekly schedule catches everything from both boards without gaps. Daily runs make sense for a shortlist of two or three direct rivals where the extra rows are cheap. Whatever the cadence, keep hoursOld equal to the interval between runs.

Does Indeed have an official API for tracking competitor job postings?

No. Indeed retired its Publisher API, the read-side feed developers used to pull listings, in 2023, and the Publisher Program itself closed to new sign-ups in October 2022 with no reopening announced (Job Boardly, 2026; JobsPipe, 2026). What remains, Indeed Apply, Sponsored Jobs and the Job Sync API, is employer-side and gated behind partner approval at partners.indeed.com. There is no self-serve endpoint that returns Indeed search results or company pages, which is the gap this scan fills.

Does ZipRecruiter have an official API I could use instead?

Not for reading postings. ZipRecruiter shut down its ZipSearch publisher API on March 31, 2025, ending the program that let sites query its listings (Job Board Secrets, 2025; Job Boardly, 2026). The partner API it still runs is for job boards and publishers that syndicate listings to ZipRecruiter, not for querying jobs or company pages on demand. Neither path exposes an employer's live job total or hour-precise posting timestamps, which are the two fields the weekly diff depends on.

How much does a weekly competitor scan cost?

Every company record and every job row is one result event at $0.0005, so a scan returning 50 company records and 950 job rows costs $0.50 in results, plus a fraction of a cent for the run start. Apify platform usage (compute and proxy traffic) is billed separately and is typically a few cents for a run that size; every account includes free monthly platform credit. To keep a large list cheap, lower maxJobsPerCompany, turn emitJobRecords off, or use profiles mode when you only need the live total.

How do I stop a competitor with a common name from matching the wrong employer?

Put the website after the name with a pipe between them, such as Kelly Services | kellyservices.com, or pass {name, website} objects through the API. Jobs are matched by the employer's domain first and only fall back to a normalized-name comparison governed by matchThreshold, which defaults to 0.82 on a 0.5 to 1 scale. Check matched_company_name in the first run for every competitor; if a name resolved wrongly, raise the threshold or add the domain and rerun that company alone.

Can I track competitors that hire outside the United States?

Partly. Indeed supports country selection through countryIndeed, with values such as usa, uk and canada, so a UK or Canadian competitor list works by switching that field and setting location to match. ZipRecruiter covers the US and Canada only, so for other markets the ZipRecruiter live total will be empty and the signal rests on Indeed counts and job rows. For competitors hiring mainly in India or the Gulf, the Naukri and Bayt scrapers in the catalog cover those boards.

Keep reading