From Copy - Pasting Usernames to One - Click Imports: Building CreatorLens AI

I wanted to store a lot of Instagram creators in a database, along with details I cared about, like their nationality and other attributes. The first version of the workflow was slow, and I spent the next few months making it faster. CreatorLens AI is the result: a platform to ingest, enrich and curate creator profiles, now managing 2,377 of them.

Step one: pasting usernames by hand

I started by pasting each creator's username into the app one at a time. That worked for ten profiles and became painful at a hundred.

So I added mass upload: collect usernames in a list, then add them all at once. It was faster, but I was still filling in every attribute myself, and that took even longer than collecting the names.

Step two: letting AI fill in the details

I connected the Gemini API so I no longer had to type attributes like nationality by hand. The AI reads the categories I have defined and fills them in for each profile. The part I like most is that if I add a new column, the AI fills that one in too, so the schema is not hard-coded.

Step three: a mobile app with a share button

The workflow was still slow because I had to open the web app and enter things. So I built a Flutter mobile app with a built-in share target. When I find a creator's reel, I share it to my app the same way I would send a reel to a friend, and it lands in the website's queue. The app signs in with Google, stores tokens securely on the phone, and sends the URL to the REST API with a bearer token.

It was better, but each share still took more than 5 seconds from tapping share until the app closed itself.

Step four: the Garbage Eater

While working on the app I had been saving reels, and I realised those saved reels were already a list of everything I wanted to add. So I used a scraper to export all my saved reels as a JSON file, then built a feature to paste that JSON in and import everything at once. I named it the Garbage Eater, because I throw a big messy file at it and it sorts everything out.

Now my whole job is to save a reel and keep scrolling. Everything else happens afterwards, with each reel taking under a second in my own use.

CreatorLens now has four ways to add profiles:

The staging pipeline

Raw URLs do not go straight into the main database. They pass through a staging queue, which is a two-step pipeline:

1. Queue. A shared reel (URL, username, thumbnail, caption) is saved in a staging collection and the request returns immediately.
2. Process. A separate request enriches that item: it fetches metadata such as followers, likes, comments and views, and runs the AI step. After that, I approve the item and it moves into the main profiles collection.

Splitting it this way means no single request has to do everything, and it lets me review items before they become real profiles.

Hitting Vercel's 10-second limit

I first hit Vercel's 10-second function limit with the Garbage Eater. Small files were fine, but a large JSON file took too long to process in one request, and the request was cut off.

The fix was to break the work into batches. The importer now sends items in chunks of at most 50 per request, and the staging pipeline processes them separately, so each request stays well inside the limit. The batches are throttled as well, so a big import does not flood the database.

Protecting a free-tier database

I run on Firestore's free tier, so every read and write counts against a daily quota. Three things keep usage down:

• A Bloom filter is a small in-memory structure that answers "have I seen this username before?" without touching the database. It can say "probably yes" by mistake, but it never says "no" for something it has seen, which makes it a safe quick check. Because it skips a database read, it is faster and saves quota.
• Per-route rate limiting caps how often each group of endpoints can be called. For example, mass upload allows 60 requests a minute, bulk actions 30, and the video refresh job only 5.
• Chunked, batched writes group many writes into transactions instead of sending one write per profile.

The Bloom filter has a limit I should be honest about: it lives in memory on each running server instance and is not loaded from the database. After a restart, or on another instance, it starts empty, so it reduces duplicates and database reads but does not guarantee zero duplicates.

A security detail worth sharing: column names

Because users can create custom columns, those column names end up in database queries. Firestore treats a dot in a field name as a path into nested data, so a malicious name could write to places it should not reach. A validation step now allows only letters, numbers, underscores and hyphens in column names, and I added middleware that blocks NoSQL and dot-notation injection before any input reaches a query.

The rest of the API follows the same careful approach:

• Short-lived JWTs with refresh-token rotation, so a leaked token expires quickly.
• AES-256-GCM encryption for sensitive fields.
• Google OAuth sign-in with per-user data isolation, so no one can see another user's profiles.

What I would improve next

• Share duplicate checks across server instances, so deduplication survives restarts.
• Add automated tests. The project currently has none, and the importer and staging logic would benefit most.
• Stream-parse very large JSON files instead of loading them whole.
• Move long-running enrichment to a proper background job queue.

Takeaways

Most of this project was removing the slowest step, one step at a time: first typing, then pasting, then AI, then the phone, and finally the importer. A serverless time limit does not have to block you if you split the work into small, separate requests.

• Live app: createrlens.vercel.app
• Code: github.com/NartanNaik/CreaterLens
• Stack: React, Node.js, Express, Firebase Firestore, Flutter, Vercel, Gemini API