I've been working on a new app to use your #Linux phones and tablets as secondary wireless "side displays" (not mirrors). This is based on #GStreamer, x264, #Mutter APIs, and low-latency UDP streaming, and will be designed to work on both ARM and Intel, regardless of hardware acceleration support. (Testing on a #librem5)
This is still at an early stage, and will take some time to become usable. Thoughts and feedback?
@tuxdevices
I NEED THIS RIGHT NOW! 😀
No really, like 3-4 years back I was trying to do the same thing under X11 in hacky shell scripts - create a virtual screen, start the VNC server of that virtual screen and open it in VNC client on my laptop or phone, to extend the screen of my desktop. Don't remember why, but it didn't work for some reason.
With the ability to share a virtual screen under Wayland, I was thinking recently that this should be possible now.
@tuxdevices
I have an unused pine phone I would love to convert to a small heads up display of OS starts. I'll definitely be on a lookout to possibly use this app to finally achieve that 😁
@anty HUD of OS starts? If it is a console log you'd like to show on it, there could be way simpler approaches to achieve it (and not requiring a mutter session to run)
uses another protocol (miracast) and aims at supporting standardized streaming to tvs/commercial things
is not symmetric (aka: doesn't work as a sink)
does not focus on virtual screens but rather on mirroring the existing ones
Basically, Network Displays is the Wireless Display feature of e.g. Windows or Macs (and I think it could be integrated in GNOME more tightly in next releases), MirrorHall (this tool) is more like Sidecar
@tuxdevices I love everything about this and I encourage you to continue working on it.
Take a look at other codecs, including 8 bits per pixel greyscale for reading text or terminals, actually. What I'm trying to say is: Explore options that do not introduce compression artifacts. The encoding and decoding should be interchangable.
That being said, it's perfectly fine to start with H.264 only. Just plan ahead a little.
@tuxdevices@dylanvanassche I understand it could be a lot of extra work and won't happen any time soon, but would input support be technically feasible with this architecture?
@be@dylanvanassche in principle yes, but it needs another parallel stream "backwards", a Wayland simulated input method, and adding the logic to grab and transmit all input from the Gtk widget
@tuxdevices@be This serves also like a good example on how to grab and export the desktop to another screen if we ever want to have Android Auto support for example.
@be@Conan_Kudo it can be decoupled but there's no standard API and most compositors probably support it in a bit different way. Mutter so far had a really smooth casting and virtual sink API
@Conan_Kudo@be then support should be quite easy! There's a (bad) dbus library with support functions for Mutter, a compatible one for KWin could be added very quickly
@tuxdevices This looks amazing! Being able to use any Linux powered tablet or phone as a second screen is a very useful feature. Would like to see it work over USB Ethernet to a connected phone as I figure that would be the lowest latency.
@CalcProgrammer1 it should work the same over usb ethernet (also with much better bandwidth), perhaps apart from broken mDNS. That was one of the initial use cases, "plug in USB, click to share"
@alex Thanks for the feedback! It would work in principle, but this tool has no input support yet, as that is very hard to re-implement on a Gtk app :|
@tuxdevices I had that idea some years ago, it's so cool to see this! Never made an attempt as I just came up with it based on X11 functionality, which I was already on way to move away from.
Do you think this will be somewhat easily portable to termux (or even perhaps a dedicated Android app for F-Droid)?
@4censord There is no standardized virtual sink API sadly, and not all WMs support it. This works on all architectures (as gstreamer can fall back on software-based h264 even if no hardware support is enabled)
@guihn288 An Android release would be possible, but require a totally different codebase sadly (e.g. for MDNS APIs, streaming and so on). If you can copy the GStreamer pipeline to an Android client, however, you can use the "Manual Connection" button to stream to it
@tuxdevices I love the sidecar feature on Mac which sounds like the same thing as this. Since I work a lot while traveling it’s easy to bring my iPad and use it as a second display. It does lose connection a lot though, which is a hassle. Would love to see this on Linux.
@amalgam_ Thanks! Restarting a stream here is simply done by toggling the "connected device" button twice (I did it this way so far when it lost connection)
Add comment