@webology@matt thanks, this post was … gross, but I didn’t feel qualified to comment.
I strongly believe developers should be doing stuff like webauthn and code review, but OpenSSF and similar orgs should be focused on providing resources to maintainers, not shaming people doing free work.
@glyph@webology@matt The scorecard really does feel like a tool for shaming and increasing burnout, which was exactly what resulted in adding that bad maintainer. Even if all its ideas are good, the problem isn't that I don't want to do them, it's that I have no time on top of all the other stuff, and they're things that can't be delegated to less experienced/trusted contributors.
@davidism@glyph@webology@matt the folks behind it were well-intentioned, but this sort of “we have built the hammer so every tool is a nail” post was an obvious outcome.
It's not lost on me that one of the possible factors in pushing the XZ maintainer into accepting any help offered was a number of probable sock puppet accounts concern trolling the maintainer about the "slow pace of development".
What is the chance this "scorecard" becomes another weapon for attackers to use on maintainers?
@michaelcoyote@davidism@glyph@webology@matt approximately 100%, though unfortunately the evidence here is that any request for improvement of any sort can now be weaponized. So I don’t think that’s a huge strike against Scorecard.
@Di4na@glyph yeah, I'm really hesitant to place blame on the mere act of "requesting improvement". It's true that here it was literally weaponized, and that there's a long, messy track record of it being abusive.
But asking questions is at the core of all user requirements gathering in open, and the line between asking questions, "just asking questions", and being a nation state is very hard to draw.
@glyph@webology@matt I just looked at their scorecard for Flask again and it's still just as inaccurate/unhelpful as it was last time someone told me to look at it.
It has been used to direct funding to critical open source projects.
Designed by security experts, and used successfully, it is probably the best dataset/tool we have to improve and identify projects in need.
In Django it detects valid issues. Dependencies not pinned, token permission issues, no code security scanning tool used, not transparent about security practices used.
They’re doing good work helping to direct funding and make things more secure. They deserve kudos.
@renedudfield Half of what is flagged, and you are calling out results from the tool not knowing how to read the metadata.
I don't prioritize a screenshot tool being loosely pinned over the framework itself having pinned dependencies. That scores them a 0 right off the bat.
@renedudfield Django did alot of things that pre-date the security community giving bonuses to doing things the GitHub way which haven't even been around for that long.
There are a few other tools that ding Django for not having a robust COC despite recommending two that are based on our COC as good starting points.
All because Django was doing this before the tools were written and no one wanted to jump through the hoop for a company who is not even donating to support them.
I'm not sure what you mean about a screenshot tool? But deps in the build scripts/actions are not pinned. The setup.cfg has dependencies unpinned. requirements files also.
This includes unpinned deps that depend on xz btw. eg. Pillow which pins xz.
It detects a security policy and gives points for it. btw, this isn't a GH specific tool or from them. It supports other systems.
I agree with the tool that Django needs funding to fix real issues. 7.2/10 is not bad or average though.
@renedudfield The screenshot tool is part of the many workflows that are deducted points for "0 out of 38 GitHub-owned GitHubAction dependencies pinned".
Those technically are pinned/versioned, but the tool is more pedantic and says to link each action to a sha instead of a version.
Again, these are "GitHub-owned" and I'm not going to argue about the irony of this but if the message is "Don't trust GitHub" then I'm happy to have a beverage over that topic.
@webology 💯 Too many times these reduce to “Does project use X feature of GitHub?” – I get the idea with scoreboards, but I’m a bit sceptical in practice: we don’t need to be dishing out sticks with which to beat maintainers.
There are several projects that are well maintained, but they probably don't fully buy into the GitHub security hoop Koolaid and that's a healthy reaction.
One final thought. I am away for the weekend, seeing family and not keeping up. I'm seeing several people bite on various aspects of xz that I probably won't read until Sunday night or Monday.
Add comment