fohrloop,
@fohrloop@fosstodon.org avatar

when creating a python project, and using some==2.2.0 to "pin" your requirements isn't actually pinning them, as the package owner (or anyone with access) may upload version 2.2.0-1, 2.2.0-2, etc. which will match the "==2.2.0".

https://www.youtube.com/watch?v=oGpyupM52IQ

diazona,
@diazona@techhub.social avatar

@fohrloop Huh. I did a quick experiment which suggests otherwise:

>>> from packaging.version import Version
>>> from packaging.specifiers import SpecifierSet
>>> s1 = SpecifierSet("==2.2.0")
>>> Version("2.2.0.post0") in s1
False
>>> Version("2.2.0.post1") in s1
False
>>> Version("2.2.0") in s1
True

(".post0" is the canonical way of writing "-0", and so on)

fohrloop,
@fohrloop@fosstodon.org avatar

@diazona oh, right! Should have added that this "-N" is added with --build-number=N. If you want to see a demo, check the video around 3:25.

fohrloop,
@fohrloop@fosstodon.org avatar

@diazona Or are the --build-number=1 and ".post1" also the same thing? For some reason pip installs the wheel with version "2.2.0-1" instead of "2.2.0" if that's available, even if you specify "package==2.2.0".

diazona,
@diazona@techhub.social avatar

@fohrloop Ahh, I see: the "1" is actually a "build tag" as per the wheel naming spec https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format.

So yeah, this is confusing: you can actually have a version 2.2.0-1, which is equivalent to 2.2.0.post1, and if that were the case, pip would be able to tell the difference. But the 2.2.0-1 mentioned in the video and your original toot is not a version, it's a part of the wheel filename, and it actually means version 2.2.0 with build tag 1, which is something different.

hugovk,
@hugovk@mastodon.social avatar

@diazona @fohrloop A wheel build tag is useful for fixing broken wheels, see "A wheel file wasn't compiled properly":
https://snarky.ca/what-to-do-when-you-botch-a-release-on-pypi/

diazona,
@diazona@techhub.social avatar

@hugovk @fohrloop Yeah I think that was mentioned somewhere earlier, but personally I'd rather just use a separate post release to solve that problem. IMO it's too unintuitive that you can install the same version of a package on the same system and get different code.

fohrloop,
@fohrloop@fosstodon.org avatar

@diazona @hugovk

I wonder if the build tags are the way to go if you use readthedocs and want to just update the documentation. Or would you rather move the git tag and force rebuild in ReadTheDocs..?

hugovk,
@hugovk@mastodon.social avatar

@fohrloop @diazona Read the Docs points to Git tags, not wheel build tags. Changing a wheel build tag on a release artifact doesn't change the code, and so won't trigger an RTD build.

You might want to point people to the /latest/ RTD page instead of a /stable/.

fohrloop,
@fohrloop@fosstodon.org avatar

@hugovk @diazona

What do you mean by /latest/ and /stable/?

I have https://wakepy.readthedocs.io/en/latest/ pointing to the latest released version (default if you just go to https://wakepy.readthedocs.io ) and then https://wakepy.readthedocs.io/en/dev/ which follows the dev branch which is the long living development branch of unreleased work (many call this master or main).

diazona,
@diazona@techhub.social avatar

@fohrloop @hugovk Oh I was assuming that /latest/ points to the documentation of the last commit on your main development branch, while /stable/ points to the documentation of the latest released version (i.e. with a version number). So, people who download the project from /pypi/ will more likely want the /stable/ version.

hugovk,
@hugovk@mastodon.social avatar

@diazona @fohrloop

For example:

https://black.readthedocs.io/en/stable/ is the last released version (24.3.0 tag)

https://black.readthedocs.io/en/latest/ is the current state of the unreleased main branch

I think these are the RTD defaults.

diazona,
@diazona@techhub.social avatar

@hugovk @fohrloop Agreed, except for pointing people to the /latest/ RTD page; I think it's very reasonable to decide not to do that in general.

Anyway, I would say updating the documentation is a perfect opportunity to make a post release.

fohrloop,
@fohrloop@fosstodon.org avatar

@hugovk
@diazona
For docs-only-change (typo, link fix, etc.) for version X.Y.Z I have just checked out with the git tag X.Y.Z, made changes, and moved the git tag X.Y.Z to the new commit, and forced doc rebuild for git tag X.Y.Z in the RTD web ui. But seems that giving the new commit tag X.Y.Z.post1 would be a better way. I wonder if that's still shown as X.Y.Z in RTD for the users..?

diazona,
@diazona@techhub.social avatar

@fohrloop @hugovk For a very small project this can work okay, but only if you change the tag quickly enough that nobody notices. In general, I think a better approach is to use release candidates: when you think you're ready to make a release, prepare it as an rc first and then after you've given yourself and others time to check it for possible issues, then retag the rc as the final version and release that. Then it should be quite rare that you need to change something after making the final release without incrementing the version number. But in the rare instances where you do, then I think having it show up as X.Y.Z.post1 in RTD is fine.

hugovk,
@hugovk@mastodon.social avatar

@diazona @fohrloop Yep, moving Git tags can cause confusion for others.

For CPython releases, the tag is made on the release manager's fork, and only pushed upstream once the release is ready, to avoid the whole world seeing a tag and thinking a new release is out.

There may be unforeseen build problems that delay the release for a few days!

# Make sure you are pushing to your GitHub fork, *not* to the main  

https://peps.python.org/pep-0101/#how-to-make-a-release

hugovk,
@hugovk@mastodon.social avatar

@diazona @fohrloop
There's a balance: if you have ~50 wheels, and just one needs updating, you may not want to ping everyone with a "new" release because a single wheel has been fixed for OS X 10.9 (EOL in 2016).

With @pillow, it uses a total of 1 day and 9.5 hours CI time to build all the wheels!

Plus we used to have to wait for someone on the other side of the world for Windows wheels.

See my comment here, and the discussion is about restricting this sort of thing:

https://discuss.python.org/t/restricting-open-ended-releases-on-pypi/43566/30

hugovk,
@hugovk@mastodon.social avatar

@diazona @fohrloop As it happens, I have just kicked off the wheel build process for this quarter's @pillow release, you can follow along with the fun at https://github.com/python-pillow/Pillow/actions/runs/8506382482 :)

fohrloop,
@fohrloop@fosstodon.org avatar

@hugovk @diazona @pillow

Thanks I think I needed something to do on Easter Monday! How long this will run?

hugovk,
@hugovk@mastodon.social avatar

@fohrloop @diazona @pillow

Just under three hours! Here's a previous run: https://github.com/python-pillow/Pillow/actions/runs/8500607792

The really slow bit is the QEMU-emulated part, cross-compiling stuff on a different architecture. But it means we don't need to use Travis CI any more, and this whole thing is automated including publishing sdist+wheels to PyPI at the end.

At least that's the plan! This is our first release using cibuildwheel + Trusted Publishers automation! 🤞

diazona,
@diazona@techhub.social avatar

@hugovk @fohrloop Well... I acknowledge your point, but given the way things are moving toward reproducibility these days, I think it's really valuable to have a deterministic mapping between (package, version, system) and wheel. That is, given the constraint that I want to install a particular version of a particular package on a particular system, there should be one specific wheel file that will be used for that particular installation. If a wheel file is broken or something, then I think making a new version needs to be the price of fixing that. (If that involves pinging people in some way that's annoying, then I would say it's the notification system that should be fixed.)

hugovk,
@hugovk@mastodon.social avatar

@diazona @fohrloop

Yeah, cibuildwheel has made it much easier to build a wide range of wheels on CI, and Trusted Publishers makes it easy to upload to PyPI.

I think we'll also see more projects start using Sigstore:
https://www.sigstore.dev/how-it-works

diazona,
@diazona@techhub.social avatar

@hugovk @fohrloop Indeed!

I never heard of Sigstore but I will have to check that out, thanks 😀

hugovk,
@hugovk@mastodon.social avatar
  • All
  • Subscribed
  • Moderated
  • Favorites
  • til
  • tacticalgear
  • DreamBathrooms
  • cisconetworking
  • khanakhh
  • mdbf
  • magazineikmin
  • modclub
  • InstantRegret
  • rosin
  • Youngstown
  • slotface
  • Durango
  • kavyap
  • ngwrru68w68
  • JUstTest
  • everett
  • tester
  • cubers
  • normalnudes
  • thenastyranch
  • osvaldo12
  • GTA5RPClips
  • ethstaker
  • Leos
  • provamag3
  • anitta
  • megavids
  • lostlight
  • All magazines