[SOLVED] Ffmpeg makes choppy videos when recording audio

filister, 2 months ago (edited 2 months ago)

Here are my 2 cents. Try to use some codec with hardware acceleration like h264_qsv but make sure that your version supports it, otherwise you will need to re-compile it. Try to simplify your ffmpeg command and update your ffmpeg version if it is not the latest one. Example command:

<span style="color:#323232;">ffmpeg -f x11grab -draw_mouse 0 -framerate 30 -probesize 42M -video_size 1920x1080 -i :0 -f pulse -i default -c:v h264_qsv -preset:v ultrafast -r 30 -g 90 -b:v 4500k -c:a aac prueba.mkv
</span>

In this command:

-c:v h264_qsv specifies the H.264 video codec with Intel Quick Sync Video hardware acceleration.

-preset:v ultrafast - sets the video preset to ultrafast, meaning that ffmpeg will try to lower the quality of encoding and should potentially decrease your CPU utilisation. In this case I would also recommend you to increase the -b:v to 6-8-10M to compensate for the quality loss and if needed you can always offline transcode it to lower the size of your recording

-c:a aac specifies the AAC audio codec for audio encoding.Other parameters such as framerate, video size, input sources, and output file name remain the same as in your original command.

Make sure your version of ffmpeg is compiled with support for Intel Quick Sync Video. You can check this by running ffmpeg -encoders and looking for the h264_qsv encoder in the list. If it’s available, you should be able to use hardware encoding with Intel HD 4000.

Another idea is to try to run video grab only and then audio grab only and check the CPU utilisation. You can try to change the

<span style="color:#323232;">-f pulse -i default
</span>

and define the input manually, and not leaving it to default. Also make sure you are not using ALSA or pipewire, as the audio grab syntax is slightly different from what you posted

qjkxbmwvz, 2 months ago

Does top/htop/whatever show a large difference between encoding with and without audio? Do e.g. the pulse processes hog resources for some reason?

Do you get smooth recording if you reduce the screen resolution and do something that uses minimal resources (e.g., play Tetris at 640x480)?

Ignacio, 2 months ago

Depending exclusively on the codecs I use, htop shows a CPU usage of 90-100% or 10-20%. So, I use those that don't require a huge CPU usage. Memory and Swap are not affected, as far as I know. But despite those circumstances, the outcome is the same. Recording with no sound = smoothness, 60 fps, recording with sound = choppiness, 10 fp5s.

Something I also noticed is that it also happens before I open Doom 2, so it doesn't matter whether I'm playing or not. Even with just the terminal and the file manager open.

qjkxbmwvz, 2 months ago

Crazy idea, but what happens if you record audio and video with different ffmpeg processes? Just for debugging this might be useful to try. Perhaps there’s something funny about the timestamps you’re receiving from Pulse that causes weird behavior (or something along those lines).

Also, you mentioned this works better with other software — any idea if those settings are drastically different than your ffmpeg settings?

I have no idea what I’m talking about though, just throwing out ideas :)

Ignacio, 2 months ago

Crazy idea, but what happens if you record audio and video with different ffmpeg processes?

I also tried that with the -map parameter, if that's what you mean. Same outcome.

Also, you mentioned this works better with other software — any idea if those settings are drastically different than your ffmpeg settings?

SimpleScreenRecorder is very basic, but it works well, except for some random dropped frames here and there. No big deal, but sometimes it's noticeable enough for me to feel uneasy. 1920x1080, 60 fps, PulseAudio, H.264 video codec, ultrafast preset (skipping frames) and vorbis audio codec at 128 kbps. Just that.

qjkxbmwvz, 2 months ago

No, what I meant was to try recording video with ffmpeg, then open another terminal and record audio with ffmpeg. Does the video look ok when recorded like this?

Also, you said you tried various codecs, what happens if you dump the raw audio (PCM?) stream without reencoding at all?

Ignacio, 2 months ago

No, what I meant was to try recording video with ffmpeg, then open another terminal and record audio with ffmpeg. Does the video look ok when recorded like this?

Well, surprisingly, it works. Video runs smoothly, and I can merge the separate audio file to it witk Kdenlive. So, I guess the problem is solved. But I don't understand why it works this way and not with a single command in one terminal.

qjkxbmwvz, 2 months ago

Weird! Yeah I don’t understand either, but this info may help in further debugging.

Have you tried playing with the container format? I know .mkv is generally preferred but maybe try .mp4 or something else. My thinking here is that both audio and video work, but somehow muxing them is failing. Using a different container could maybe help since the muxing is different (???).

Other thing to try is of course making sure ffmpeg is up to date but it sounds like you’re on top of that.

Good luck!