oliverandrich,
@oliverandrich@fosstodon.org avatar

Is there a better approach to scraping a SPA in than to use ?

webology,
@webology@mastodon.social avatar

@oliverandrich I haven't found one.

I even use Playwright as a requests/httpx replacement for some non-SPAs because it doesn't get outright blocked.

oliverandrich,
@oliverandrich@fosstodon.org avatar

@webology Thanks for the hint. I just tried it for one site, were I also was unable to create a result and it simply blocked anything what is not a browser. May be, I should use Playwright for all request in my little scraper.

webology,
@webology@mastodon.social avatar

@oliverandrich I like to keep track of my college basketball team and where they rank on the NBA draft boards. Most of these draft boards are compiled by gambling websites, and Playwright is the only tool that will work on them. They are pretty battle-hardened.

I tend to start with httpx and then I quickly pivot to Playwright if I need a real browser.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • python
  • GTA5RPClips
  • DreamBathrooms
  • InstantRegret
  • ethstaker
  • magazineikmin
  • Youngstown
  • thenastyranch
  • mdbf
  • slotface
  • rosin
  • modclub
  • kavyap
  • cisconetworking
  • osvaldo12
  • JUstTest
  • khanakhh
  • cubers
  • Durango
  • everett
  • ngwrru68w68
  • tester
  • normalnudes
  • tacticalgear
  • anitta
  • megavids
  • Leos
  • provamag3
  • lostlight
  • All magazines