Hacker Newsnew | past | comments | ask | show | jobs | submit | IWantAllTheData's commentslogin

Thanks for letting me know. I will choose a better wording.

Eng. Rate (%) is Engagement Rate. Which is a common metric marketers look into when hiring an influencer.


All the scrapers are in Python but im slowly transferring it to Golang, i find it much faster.

The website is NextJS.


Thank you. Yes proxy bill is quite high!

Yes you are right, the ID enumeration method doesn't work with TikTok.


I appreciate the positive comment. Thanks!

Fun fact: I did the UI Design, frontend/backend web development, database, servers, scraping and everything else by myself. It took me few months but i guess its worth it.


I use proxy.


All the data were scraped without creating a TikTok account, AKA public data. Google.com has scraped more TikTok profiles and videos than me.

In fact, one the way i did it is by scraping google search results itself.


It may not be quite that simple, see hiQ Labs vs LinkedIn[1].

[1] https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn


That's not about public data. Reference [7] talked about creating fake accounts to scrape.

> hiQ had prevailed on the Computer Fraud and Abuse Act (CFAA) “unauthorized access” issue related to public website data but was facing a ruling that it had breached LinkedIn’s User Agreement due to its scraping and creation of fake accounts (subject to its equitable defenses).

https://natlawreview.com/article/hiq-and-linkedin-reach-prop...


So many useful stuffs in the comments. Thank you, will give this a read as well.


Will check it out thanks. I'm always ready to take down the site if TikTok reach out me to.


There will be some backlash for this, it can seem nefarious when presented this way. That said, can we see your work?


I didn't even think this could be controversial before reading the comments. I always assume what I post on social media is public.

Do others have different assumptions? Why would people get angry at a random person collecting data they published?


Yes, I did receive some backlash, which is understandable. However, this is a very useful tool for content creators to find out what's trending on TikTok. It is also used by brands to analyze market sentiment about them.

I can't share the specifics, but i can explain the methodology in private. My contacts are on the website.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: