It would be harsh to say that requests is a pile of poorly-written attractive nuisances on top of urllib3. But, unfortunately, it wouldn’t be wrong.
Many diligent devs have spent countless work hours trying to unfuck the project over the years, but there’s only so much you can do when:
“After receiving our first security disclosure, I was told that Requests wasn't a serious project but instead one person's art project and thus we shouldn't fix the vulnerability.” https://hachyderm.io/@sigmavirus24/111986425127558764
@hynek interesting insights, also in Ian’s blog post. What would you recommend to devs using Python today and looking for an HTTP client? I’ve used httpx a lot and love it but I have to admit I have never properly researched the “quality” of that library, in terms of security, test coverage, project governance etc.
So I’m curious what http client lib you recommend!
If you ever wondered why I’ve steered people away from it over the past years, read the linked article or just take it from Ian, who’s been one of the, if not the, longest-serving maintainers on the project:
“In short, the project feels dead.[…] It's hard to introduce new, necessary, and beneficial features. It's hard to fix gnarly bugs. It's hard to improve the user experience and it's consistently been because of one particular person over the years.”
I’ve personally never been involved with the project because I was horrified by the code base from day one.
But I consider many of those who navigated the bullshit necessary to keep the lights on my friends and had a direct insight over the years what it took to keep an art-project-turned-top-PyPI-download chugging along – just for the least-deserving person to achieve community sainthood.
For me Ian’s blog post is incredibly liberating because I’ve always carried a rage about the project with me, but I couldn’t articulate it without tangentially throwing the current maintainers under the bus, too. Now I just have to cite and point out that he’s still holding back.
@hynek I am happy to hear about what’s bad with urllib3, I won’t feel thrown under the bus! Maybe I won’t be able or willing to do anything about it, but as a maintainer, I am not urllib3 and urllib3 is not me. We’re all adults here :)
@quentinpradet I think you’re under/overestimating things here. :) One of the traditional Requests excuses for not having good test coverage (back then not even non paper), was that urllib3 is well-tested (yes, really). What’s going on in Requests goes way past urllib3’s problems. And I’m sure Ian has seen both too, so I trust his judgment. ;)
@quentinpradet Big picture-wise, you’re right of course: I don’t care that a package is shitty code, I would never shame anyone for that. The problem is the governance & the effect it has on the involved people until today. Him going away didn’t automagically undo the damage to both people and project and there’s a straight line from that to the technical problems Ian is enumerating (one doesn’t have to agree with all of them).
@hynek Your take (and rage) is still very much about the original author. I get it, I was involved in the async support mentioned in https://vorpus.org/blog/why-im-not-collaborating-with-kenneth-reitz/. But that was five years ago now and the original author no longer has any influence, thankfully.
@hynek At this point, I don’t feel requests is that different from other libraries that have been around for a decade or two, like urllib3, github3, dateutil, http.client, probably various web frameworks, etc. I could write the same post about urllib3, minus the drama: it has pools of pools! Redirects are implemented at the pool level but also at pool of pools level! The list goes on.
@quentinpradet @quentinpradet To me, urllib3 is actually despite all its flaws a model example of an ancient project that thru thoughtful governance was dragged kicking and screaming into 2024.
@hynek I think the reason this triggers me is that I have always thought the effort of working on projects like urllib3 or requests worth it simply because of the millions of downloads, and the pain it would be for the ecosystem to use a different client. And I am afraid to be wrong.
@quentinpradet man this got a therapy session real quick :) and I 100% understand your thinking and we’re aligned when it comes to urllib3. But this is exactly why toxic governance is such a problem even long-term. It’s all about people.
@hynek For those who may not have seen your previous comments, what do you typically recommend? Most of my projects typically use the equivalent of request.get/requests.post so I’ve typically not searched for anything else
Add comment