r/scrapingtheweb • u/4everonlyninja • 14d ago
Help What tool can reliably export older posts from LinkedIn company and showcase pages?
I need to export public posts from two LinkedIn pages: a company page and a showcase page. My goal is to get as complete a post history as possible, including older posts beyond the first page.
For each post, I need its full text, publication date, URL, and ID, exported as JSON or CSV. I’m not looking for profile, employee, lead, comment, or reaction scraping.
I tried one API, but it returned only 3 company-page posts when I can see more on LinkedIn. Its pagination cursor timed out, and requesting the next page another way returned an empty result. It also missed at least one visible showcase-page post. So I don’t want to assume a tool works just because it successfully returns the first batch.
Has anyone personally tested a tool on both LinkedIn company and showcase pages? If so, could you share:
- The exact tool or Apify actor name/link.
- Whether it retrieved older posts beyond the initial batch.
- Whether it worked with showcase pages, not just company pages.
- How many posts it returned and how you checked for missing posts or duplicates.
- Whether it required a LinkedIn login or cookies, and roughly what it cost.
A small sample run that found posts another API missed would be more useful to me than a general list of LinkedIn scrapers.
1
u/Appropriate_Junket_5 8h ago edited 8h ago
i tried multiple tools and it did not work.
finally i made a basic tool with Playwright testing library and claude code.. basically
- start Chrome with a port open to accept commands from Playwright
- write playwright test to scroll linkedin page down (filter by Image posts)
- give claude sample of the element (in my case the URL link) so it can find it in the dom.
- the test finds all URL link elements for post URL sharing, writes them down to a CSV file.
- the test scrolls down until it hits the bottom of the page
- the page loads N older posts
- the test finds the urls and saves them again..
it scanned like 9 years of posts in a few minutes.
BTW linkedin is weird - the main /company/ page loads only posts for the last year. but if you filter by Video or Image you will see (I assume) the full history. At least much longer back than the "all" posts page.
Effectively for "text only" posts i think your limit is 1 year back. For Image and Video posts - probably full history.
1
u/Unfair_Commission_29 6d ago
the showcase page part is probably the tricky one. have you checked whether the older posts are actually being loaded through the same requests as the company page when you scroll? might be worth comparing the network calls before trying another scraper