Claude Code can automate Instagram competitive research by scraping competitor profiles via Apify, downloading images locally, and building a swipe-style review app as a local HTML page. The app displays each post with its image, caption, like count, and comment count. Users vote yes/no on each post, and results are saved in the browser’s localStorage. Patterns in engagement, content type, and caption length emerge from reviewing the data systematically rather than scrolling manually.
If you do brand or marketing work, you’ve probably spent time manually scrolling through competitor Instagram accounts trying to figure out what’s landing. For a similar workflow on LinkedIn, see How to Ghostwrite LinkedIn Posts. You screenshot posts, copy captions into a doc, try to hold patterns in your head. It works, but it’s slow and it’s easy to miss things.
This tutorial shows you how to use Claude Code to automate that process: scrape competitor Instagram profiles, pull real posts with engagement data, and build a simple review app where you can swipe through posts and vote on what you like.
The end result is a local web page (a self-contained HTML file that runs in your browser, no server required) with actual images, captions, like counts, and comment counts, and a yes/no voting system so you can tag what resonates.
A note before you start: Instagram scraping tools break frequently because Instagram changes its structure and blocks scrapers. Before following these steps, check that the Apify actors mentioned here are still active and current. Links are in the Further Reading section.
What you need
- Claude Code installed
- An Apify account (more on cost below)
- A list of Instagram handles you want to research
What is scraping, and why Apify?
Scraping means automatically collecting data from a website. Instead of clicking through pages yourself, a script does it for you, visiting URLs and pulling out the information you care about, like post images, captions, and engagement numbers.
Instagram doesn’t offer a public API for this kind of research, so scraping is the practical alternative. The catch is that Instagram actively tries to block automated access, which means doing it yourself requires maintaining code that fights Instagram’s detection systems. That’s a lot of work to maintain.
Apify is a cloud platform that runs scrapers for you. Other developers build and maintain Instagram scrapers (called “actors” in Apify’s terminology), and you pay to run them. You don’t have to write or maintain the scraper itself. You just call it, pass it a list of accounts, and get back structured data.
Cost: Apify charges based on compute usage. Running a scraper on 5-10 Instagram accounts typically costs a few dollars at most. They have a free tier to start. Check their current pricing at apify.com/pricing before you begin.
The prompt
Here’s the prompt used to build this in Claude Code:
I need to do visual competitive research for [your brand]'s Instagram.
Build me a review app where I can swipe through posts from these accounts
and vote on which ones I like.
Accounts to research:
- @account1
- @account2
- @account3
That’s it. Claude Code takes it from there.
What Claude Code builds
Claude Code breaks this into several steps automatically:
1. Scrapes the accounts using Apify
Claude writes code that calls an Apify Instagram scraper actor. It passes in your list of accounts and collects each post’s image URL, caption, timestamp, like count, and comment count. Engagement metrics are the numbers that tell you how a post performed: likes show reach and resonance, comments indicate conversation, and the ratio between them tells you something about the type of content (broad reach vs. community engagement).
2. Downloads images locally
Instead of hotlinking to Instagram images (which can break when the URLs expire), Claude downloads the actual images to a local folder. This way your review app keeps working even after the source URLs go stale.
3. Builds the review app
Claude creates a single HTML file with CSS and JavaScript built in. The interface shows one post at a time as a card: the image, the caption, the account name, and the engagement numbers. You click yes or no (or use keyboard shortcuts) to vote. Your votes get saved to localStorage so they persist if you close the browser.
4. Debugs display issues
In practice, there are usually a few issues on first run: images not loading, layout problems on certain post types, API responses with slightly different shapes than expected. Claude Code works through these iteratively, running the app, identifying what’s broken, and fixing it.
What the output looks like
Before: a blank app with placeholder images and no data.
After: something like 84 real Instagram posts you can swipe through, each with the actual image, caption, like count, and comment count. You vote yes/no, and at the end you can see which accounts and post types got the most yes votes.
The patterns this surfaces are things like: this account gets high engagement on behind-the-scenes content but low engagement on polished product shots. Or: captions under 100 words consistently outperform longer ones for this audience. That’s the kind of insight that’s hard to see when you’re scrolling manually.
Running it yourself
Once Claude Code finishes building the app, you open the HTML file in your browser. No server, no deployment, nothing to install beyond what Claude already set up. The whole thing runs locally.
If you want to re-run the scrape with different accounts, you update the account list and run the data collection script again. Claude Code will have generated that as a separate script you can re-use.
What can go wrong
The scraper breaks. Instagram changes frequently. If the Apify actor fails, check the actor’s page for recent updates or look for a more recently maintained alternative actor. Before blaming the scraper, check your input. Seven scrapers failed because of one hyphen in a LinkedIn username.
Rate limiting. If you’re scraping a lot of accounts at once, Apify may hit rate limits from Instagram. Running smaller batches (5-10 accounts) is more reliable than trying to pull 30 accounts in one run.
Image URLs expire. Instagram image URLs include expiration tokens. If you’re coming back to a review app you built a week ago, some images may have expired. Re-running the scraper refreshes the URLs.
API key handling. Claude Code will ask for your Apify API key. Don’t paste it directly into a script that might get committed to a public repo. Store it as an environment variable instead. Claude Code knows to do this, but double-check the generated code before running it.
Further reading
- Apify Instagram scrapers for finding current actors that are actively maintained
- Apify documentation for understanding how to run actors and manage API keys
- Instagram Graph API for Meta’s official API (requires a business account and approved access, but worth knowing about)
- Apify pricing to understand cost before you run a large scrape
Common Questions
How do I scrape Instagram for brand research?
Use Claude Code with an Apify account. Provide a list of competitor Instagram handles, and Claude Code calls an Apify scraper to collect posts, images, captions, and engagement data. It then builds a local HTML review app where you can swipe through posts and vote on what resonates.
What does an Instagram scraping research app look like?
A single HTML file running in your browser. It shows one post at a time as a card with the image, caption, account name, like count, and comment count. You vote yes or no using buttons or keyboard shortcuts. Results persist in localStorage and can be analyzed afterward.
How much does Instagram scraping cost with Apify?
Apify charges based on compute usage. Scraping 5-10 Instagram accounts typically costs a few dollars. A free tier is available for getting started. Check current pricing at apify.com/pricing before running large batches.
What can go wrong with Instagram scraping?
The scraper may break when Instagram changes its structure. Rate limiting can occur with large batches (keep to 5-10 accounts). Image URLs expire after about a week. Always check your input handles are correct before blaming the scraper.
A note from Alex: hi i’m alex - i run code for creatives. i’m a writer so i feel that it is important to say - i had claude write this piece based on my ideas and ramblings, voice notes, and teachings. the concepts were mine but the words themselves aren’t. i want to say that because its important for me to distinguish, as a writer, what is written ‘by me’ and what’s not. maybe that idea will seem insane and antiquated in a year, i’m not sure, but for now it helps me feel okay about putting stuff out there like this that a) i know is helpful and b) is not MY voice but exists within the umbrella of my business and work. If you have any thoughts or musings on this, i’d genuinely love to hear them - its an open question, all of this stuff, and my guess is as good as yours.