> Although Fedora 40 beta contained the 5.6 version of xz in an update, the build environment prevents the injection from correctly occurring, and has not been shown to be compromised. Fedora 40 has now reverted to the 5.4.x versions of xz.
> the apparent author of the backdoor was in communication with me over several weeks trying to get xz 5.6.x added to Fedora 40 & 41 because of it's "great new features". We even worked with him to fix the valgrind issue (which it turns out now was caused by the backdoor […]
> He has been part of the xz project for 2 years, adding all sorts of binary test files
> with this level of sophistication I would be suspicious of even older versions of xz
@scy The user (or whoever was behind that account) was seemingly also pretty active in other projects. The next couple of days are gonna be "interesting"... 😬
@scy Not really assuming anything. But as said "the cat is out of the bag" and it's important to notify people getting info via this channel, much more important than any denial or confirmation from the potential offender.
I'm not saying that it looks like someone has specifically targeted xz and played the long game by helping out a maintainer that was overworked and suffered from mental health issues
but it does look like someone has specifically targeted xz and played the long game by helping out a maintainer that was overworked and suffered from mental health issues
@m@scy Soweit ich das sehen kann ist die Checkliste: hast du 'nen Rechner der mit SSH aus'm Internet erreicht werden kann? Und hat dieser Rechner die xz Bibliothek installiert mit einer Version 5.6.0 oder 5.6.1 (xz --version auf dem Rechner sagt es dir!)? Dann bist du womöglich betroffen. Sieh nach ob es 'n Update gibt!
@positivedinosaur@m@scy Und, zumindest bislang die Erkenntnis, systemd-Systeme und glibc (wegen IFUNC). Homebrew hatte auch die Backdoor-Version, hatte aber, wie es aktuell scheint, keinen Trigger.
@ampoff@positivedinosaur Oh, ist der Schadcode inzwischen vollständig durchanalysiert? Weil heute Nacht war der Stand noch "systemd/OpenSSH ist der eine Trigger, den wir kennen, wer weiß welche es noch gibt".
@scy@positivedinosaur "wie es aktuell scheint" :) Diese Version, die ja wohl vom Adversary gepatcht wurde, hatte wohl eben diesen Trigger. Zu älteren weiß ich zumindest noch nichts.
@ampoff@positivedinosaur Es geht nicht nur um ältere Versionen, sondern auch um weitere Trigger in der 5.6er-Reihe, von denen wir noch nichts wissen.
Nur weil wir einen Trigger erschlagen haben heißt das nicht, dass es keine weiteren gibt. Das ist es, was ich an deiner Formulierung irreführend finde.
"Hatte keinen Trigger" und "hat den Trigger, den wir kennen, nicht" sind zwei komplett unterschiedliche Dinge.
@scy The original maintainer seems to have burned out and handed it over a couple of years ago. It's a lot of responsibility resting on individuals, and it's certainly a weak point in open source to exploit. https://www.mail-archive.com/xz-devel@tukaani.org/msg00567.html
@scy Is there any reason this couldn’t be the kind of pressure lots of open source maintainers receive from their users? I would apply Occam’s razor and prefer much simpler explanations. e.g. ”state actors forced/convinced the dev to help” or “talented rogue dev did it for fun/out of spite/to hack ex-partner”
The attack might be sophisticated, but IT security still is a garbage fire (again proofed by this).
Meanwhile, #Debian is considering rolling #xz back not only to the point before the backdoor was added, but to where the person who wrote the backdoor hadn't contributed any code to xz yet.
Which means considering creating patches to fix ABI breakage such a rollback would cause.
@scy It looks a bit excessive to me. I understand that none of the commits from this guy was actually malicious (i.e. the GitHub repository is itself clean).
Please do not advise people to run xz --version or similar to check whether they're affected or not.
Right now, as far as I know, the analysis of the obfuscated malware is far from complete. There may be other triggers. There may be malware in older versions, because the attacker had commit access for years.
By running xz and asking it for its version, you're running what could be more malware.
Instead, ask the system's package manager which version of xz is currently installed.
@scy I was wondering about that, too, but I don‘t think that‘s a very likely scenario here.
Malware infected binaries presenting a wrong version number would be very easy to stumble upon by accident - and the code changes would be, too.
One does not put that much effort into injecting malware just to give it away like this.
It's not that it might tell me the wrong version number.
It's that xz is not particularly dangerous as long as it's just a bunch of files on my disk. It's when you execute the code in these files that it can do harm.
@scy Ah, ok, in theory you are right.
But #xz has been executed many times since / if it was installed on our systems.
Not executing it now may feel good, but it does not make a difference, it’s still the same software that was executed a day ago, before we knew.
If I don’t execute it now for fear of some nefarious activity I must expect that this activity already happened.
So avoiding to execute it today only makes sense if I am about to reimage the system from scratch, immediately.
@scy I would agree and I see your point. However, the same would then apply to ssh. And that's a bit impossible then; how am I going to update my server if I can't ssh into it?
Fair enough though. Minimizing execution of the questionable library is not a bad idea.
Apparently the main #xz maintainer, Lasse Collin (Larhzu), who was assumed to have simply been offline for the past 24 hours, has now started working on a post-mortem overview page of the incident.
@scy@joeyh you can download debs with wget and unpack them with other tools. It's long, but not difficult, and definitely scriptable. some 3-5 commands per package, some 3-5 packages in total?
@mdione@joeyh And then you've extracted stuff on top of Apt-managed files without updating any of the associated data structures. It's not gonna like that.
My suggestion would be to install a safe version of Apt somewhere in /opt or something and use that. Or even do some chroot stuff from a rescue system.
Which is what I meant with "it gets hairy". It's not impossible, but not exactly trivial either.
@scy@joeyh right, but once you have versions you can trust, you can use them to download new version you can also trust. Of course, given that you can actually trust them.
@scy@joeyh You could even try to extract them not on top of your system but on a separate directory (/usr/local? /opt/emergency?) and play with $PATH and $LD_LIBRARY_PATH.
@scy it’s worse: systemd-notify support is a downstream patch applied by some distributors, because apparently openssh-portable doesn’t support it upstream
@scy Sorry, I am not buying this argument. Instead of using the official systemd library, developers should default to implementing their own version of a systemd-specific lowlevel socket protocol?
@scy But why do you expect people to know that? The page you linked lists a bunch of C functions at the top. And people should know that they should ignore those and rather lookup the protocol and implement it themselves?
@uncanny_static Let me give you a non black-or-white answer:
systemd could provide multiple smaller libraries instead of one monolith that links with everything.
Or prominently suggest users to implement sending the string themselves.
Or not provide a C function at all, having everyone call systemd-notify(1) in a subprocess.
The people who wrote that distro patch chose the easy way to add the feature to OpenSSH (a super high value target), instead of the "keep the attack surface small" way.
@scy I am not saying that this attack has been solely enabled by systemd. Far from that.
However, I think it was a contributing factor. When you are interfacing with another piece of software the standard approach is to look for the official libraries and use them, if they exist. In this case, however, this drastically increased the attack surface. 1/3
@scy Developers, in general, are not systemd experts and I do not think that they should be expected to know the inner workings of a systemd-specific protocol, even if it is that simple. Using the official library that implements such a basic functionality should not create a large attack surface. 2/3
@scy IMHO, one of the lessons to be learned here is that such a functionality should be provided by a library that is as simple and small as possible, and not expect people to implement the functionality themselves despite there being officially supported libraries for that. That is just not how people work and "roll your own" is usually considered bad practice. 3/3
@uncanny_static I don't think we really disagree. "Provide a simple and small library" was one of my suggestions, too, as was "prominently explain the simplicity of the protocol".
But you're right in that there's no use blaming anyone for linking to libsystemd now. We can just learn from it for the future.
@scy this does not affected OpenSSH.
It does affect #debian and some other linux distros who work hard on maintaining their track record of breakpatching fine software.
@bularator Look, there's only so many characters that fit in a toot. I can't explain why distros patch packages in 500 characters, and have some left to talk about the actual vulnerability.
For a normal user's understanding, OpenSSH is affected.
Also, if the software is so fine, why does it have to be patched in order to play nice with what's the default way to start daemons for years now?
Add comment