[Request] Any Guides to FFMPEG, Transcoding, Codecs, and Metadata?

I have a collection of about ~110 4K Blu-Ray movies that I’ve ripped and I want to take the time to compress and store them for use on a future Jellyfin server.

I know some very basics about ffmpeg and general codec information, but I have a very specific set of goals in mind I’m hoping someone could point me in the right direction with:

  1. Smaller file size (obviously)
  2. Image quality good enough that I cannot spot the difference, even on a high-end TV or projector
  3. Preserved audio
  4. Preserved HDR metadata

In a perfect world, I would love to be able to convert the proprietary HDR into an open standard, and the Dolby Atmos audio into an open standard, but a good compromise is this.

Assuming that I have the hardware necessary to do the initial encoding, and my server will be powerful enough for transcoding in that format, any tips or pointers?

Coasting0942,

Consider online file converters. Just pay for a month or two, obviously need a good internet connection.

ancoraunamoka,

Are those your own blurays? Then share them before compressing.

Transcoding is hard. There is no way that your transcoding settings are going to be a one size fits all. I am currently encoding the famous iKaos Dragonball release and I did 48 samples before deciding what configuration to use.

You are better off downloading stuff from torrent, especially for newer media. You’ll find a community that put 100x your time collectively on transcoding. That will also save from your tremendous electricity costs.

Also look into vmaf for quality metrics. Consider that switching to uncompressed 1080 might bring you close to your goal with very very low effort.

Btw, can you share the title list?

fenndev,
@fenndev@leminal.space avatar

Yeah, sure thing.


<span style="color:#323232;">Abominable
</span><span style="color:#323232;">Ad Astra
</span><span style="color:#323232;">Aladdin (Live Action)
</span><span style="color:#323232;">Aladdin
</span><span style="color:#323232;">Alien - Covenant
</span><span style="color:#323232;">Alien
</span><span style="color:#323232;">Ant-Man and the Wasp
</span><span style="color:#323232;">Ant-Man
</span><span style="color:#323232;">Aquaman
</span><span style="color:#323232;">Arrival
</span><span style="color:#323232;">Austin Powers In Goldmember
</span><span style="color:#323232;">Austin Powers - International Man of Mystery
</span><span style="color:#323232;">Austin Powers - The Spy Who Shagged Me
</span><span style="color:#323232;">Avengers - Endgame
</span><span style="color:#323232;">Avengers - Infinity War
</span><span style="color:#323232;">Baby Driver
</span><span style="color:#323232;">Batman and Robin
</span><span style="color:#323232;">Batman Forever
</span><span style="color:#323232;">Batman
</span><span style="color:#323232;">Batman Returns
</span><span style="color:#323232;">Big Hero 6
</span><span style="color:#323232;">Black Panther
</span><span style="color:#323232;">Bohemian Rhapsody
</span><span style="color:#323232;">Brave
</span><span style="color:#323232;">Captain Amercia - Civil War
</span><span style="color:#323232;">Captain America - The First Avenger
</span><span style="color:#323232;">Captain America - The Winter Soldier
</span><span style="color:#323232;">Captain Marvel (2019)
</span><span style="color:#323232;">Cars 2
</span><span style="color:#323232;">Cars 3
</span><span style="color:#323232;">Cars
</span><span style="color:#323232;">Cinderella (Live Action)
</span><span style="color:#323232;">Close Encounters of the Third Kind 40th Anniversary Edition
</span><span style="color:#323232;">Deadpool 2
</span><span style="color:#323232;">Deadpool
</span><span style="color:#323232;">Detective Pikachu
</span><span style="color:#323232;">Doctor Sleep
</span><span style="color:#323232;">Doctor Strange
</span><span style="color:#323232;">Everything Everywhere All At Once
</span><span style="color:#323232;">Fantastic Beasts - The Crimes of Grindelwald
</span><span style="color:#323232;">Finding Dory
</span><span style="color:#323232;">Finding Nemo
</span><span style="color:#323232;">First Man
</span><span style="color:#323232;">From Up on Poppy Hill
</span><span style="color:#323232;">Frozen II
</span><span style="color:#323232;">Frozen
</span><span style="color:#323232;">Gemini Man
</span><span style="color:#323232;">Godzilla (1998)
</span><span style="color:#323232;">Godzilla - King of the Monsters
</span><span style="color:#323232;">Gremlins
</span><span style="color:#323232;">Guardians of the Galaxy Vol 1
</span><span style="color:#323232;">Guardians of the Galaxy Vol 2
</span><span style="color:#323232;">Halloween
</span><span style="color:#323232;">Harry Potter and the Chamber of Secrets
</span><span style="color:#323232;">Harry Potter and the Deathly Hallows - Part 1
</span><span style="color:#323232;">Harry Potter and the Deathly Hallows - Part 2
</span><span style="color:#323232;">Harry Potter and the Goblet of Fire
</span><span style="color:#323232;">Harry Potter and the Half-Blood Prince
</span><span style="color:#323232;">Harry Potter and the Order of the Phoenix
</span><span style="color:#323232;">Harry Potter and the Prisoner of Azkaban
</span><span style="color:#323232;">Harry Potter and the Sorcerer's Stone
</span><span style="color:#323232;">Howl's Moving Castle
</span><span style="color:#323232;">Incredibles 2
</span><span style="color:#323232;">Incredibles
</span><span style="color:#323232;">Inside Out
</span><span style="color:#323232;">Interstellar
</span><span style="color:#323232;">Iron Man 2
</span><span style="color:#323232;">Iron Man 3
</span><span style="color:#323232;">Iron Man
</span><span style="color:#323232;">IT (2018)
</span><span style="color:#323232;">John Wick 3 - Parabellum
</span><span style="color:#323232;">John Wick - Chapter 2
</span><span style="color:#323232;">John Wick
</span><span style="color:#323232;">Joker (2019)
</span><span style="color:#323232;">Jumanji (1995)
</span><span style="color:#323232;">Jurassic Park III
</span><span style="color:#323232;">Jurassic Park
</span><span style="color:#323232;">Jurassic Park - The Lost World
</span><span style="color:#323232;">Jurassic World - Fallen Kingdom
</span><span style="color:#323232;">Jurassic World
</span><span style="color:#323232;">Kiki's Delivery Service
</span><span style="color:#323232;">Kingsman - The Golden Circle
</span><span style="color:#323232;">Kingsman - The Secret Service
</span><span style="color:#323232;">Logan
</span><span style="color:#323232;">Maleficent - Mistress of Evil
</span><span style="color:#323232;">Maleficent
</span><span style="color:#323232;">Mary Poppins Returns
</span><span style="color:#323232;">Moana
</span><span style="color:#323232;">Monsters, Inc
</span><span style="color:#323232;">Monsters University
</span><span style="color:#323232;">My Neighbor Totoro
</span><span style="color:#323232;">Pacific Rim Uprising
</span><span style="color:#323232;">Paprika
</span><span style="color:#323232;">Ponyo
</span><span style="color:#323232;">Ratatouille
</span><span style="color:#323232;">Robin Hood
</span><span style="color:#323232;">Spider-Man - Homecoming
</span><span style="color:#323232;">Spider-Man - Into the Spider-Verse
</span><span style="color:#323232;">Spirited Away
</span><span style="color:#323232;">Starship Troopers
</span><span style="color:#323232;">Star Trek - Beyond
</span><span style="color:#323232;">Star Trek - Into Darkness
</span><span style="color:#323232;">Star Trek (2009)
</span><span style="color:#323232;">Tangled
</span><span style="color:#323232;">The Cat Returns
</span><span style="color:#323232;">The Expendables 2
</span><span style="color:#323232;">The Expendables 3
</span><span style="color:#323232;">The Expendables
</span><span style="color:#323232;">The Good Dinosaur
</span><span style="color:#323232;">The Hulk
</span><span style="color:#323232;">The Jungle Book
</span><span style="color:#323232;">The Karate Kid (1984)
</span><span style="color:#323232;">The Lion King (Live Action)
</span><span style="color:#323232;">The Lion King
</span><span style="color:#323232;">The Little Mermaid
</span><span style="color:#323232;">The Matrix
</span><span style="color:#323232;">The Matrix -  Reloaded
</span><span style="color:#323232;">The Matrix -  Revolutions
</span><span style="color:#323232;">The Predator
</span><span style="color:#323232;">The Princess and the Frog
</span><span style="color:#323232;">The Secret Life of Pets 2
</span><span style="color:#323232;">The Secret World of Arrietty
</span><span style="color:#323232;">The Shining
</span><span style="color:#323232;">The Wizard of Oz
</span><span style="color:#323232;">Thor
</span><span style="color:#323232;">Thor - The Dark World
</span><span style="color:#323232;">Toy Story 2
</span><span style="color:#323232;">Toy Story 3
</span><span style="color:#323232;">Toy Story 4
</span><span style="color:#323232;">Toy Story
</span><span style="color:#323232;">Turning Red
</span><span style="color:#323232;">Up
</span><span style="color:#323232;">Us
</span><span style="color:#323232;">Venom (2018)
</span><span style="color:#323232;">Wall-E
</span><span style="color:#323232;">Waterworld (2019)
</span><span style="color:#323232;">Wonder Woman
</span><span style="color:#323232;">Wreck-It Ralph
</span><span style="color:#323232;">Zootopia
</span>
ancoraunamoka,

yeah, I would redownload all of those instead of transcoding. They are all available with very good encodes publicly

authorinthedark, (edited )

oh hey I’ve actually done this recently so a lot of this is fresh for me. Now I had a slightly different use case than you, I had a bunch of AV1 files that wouldn’t run on my pi so I had to convert them to something less GPU intensive. I was finding x265 was indistinguishable for me from AV1, and had a HUGE file size drop, close to a half or even a third. x264 had a larger file size than 265 and looked worse so I don’t recommend it. I did not try VP9 once I was satisfied with 265 but you could try it out and see how it compares. My recommendation is to pick one (shorter) file and run a couple of different transforms on it till you’re satisfied before trying to transform your entire library

Preserving audio and metadata is trivial, just use -c:a copy for audio and -c:d copy for metadata

EDIT: I feel dishonest not mentioning the important caveats regarding my own experiments. My files were 1080p, so the difference between codecs might be less noticeable at that resolution. It was also anime, which is similarly going to be easier to compress and be less distinct between codecs. This is why I cannot recommend x264 because if you can ruin 1080p anime it’ll ruin whatever you’re working with. This is why I recommend picking a sample video and spend a day running a couple test transformations on it to see what you like

marci33,

Since you’re talking about anime, how did you solve the subtitles problem? My pi doesn’t transcode fast enough but I haven’t found a tdarr extension to do that automatically (nor a command)

authorinthedark,

I’m sorry my subtitles worked fine from the beginning so I don’t know what to tell you. I’ve got a pi4 running osmc as my media center if that makes any difference.

DaGeek247,
@DaGeek247@fedia.io avatar

Skipping the audio encode from a blu-ray will lose op out on a surprisingly large amount of space, especially with 110 source disks. I checked one of my two hour blu-ray backups. Audio will net you about nine audio tracks (english, french, etc). A single 5.1 448kbs audio track will take about 380MB of space per movie. Multiply that by nine (the number of different tracks in my sample choice) and you'll get 3420MB per disk. That means about 376GB of space is used on audio alone for ops collection. A third of a terabyte. You can save a lot of space by cutting out the languages you don't need, and also by compressing that source audio to ogg or similar.

By running the following ffmpeg command;
ffmpeg -i out-audio.ac3 -codec:a libvorbis -qscale:a 3 small-audio.ogv
I got my 382MB source audio track down to 200MB. Combine that with only keeping the language you need, and you end up dropping from 376GB down to 22GB total.

You can likely save even more space by skimping on subtitles. They're stored as images, so they take up a chunk of space too.

the_third,

Please use Opus instead of Vorbis. Vorbis is old and had been superseded by Opus.

sxan,
@sxan@midwest.social avatar

Just FYI, despite what media companies would like you to believe, making copies of media you own for your own use is not piracy. It’s allowed by law under fair use.

Lemmchen,

Depending on your jurisdiction. Big asterisk.

sxan,
@sxan@midwest.social avatar

True, good catch.

fenndev,
@fenndev@leminal.space avatar

Mhm, I’m aware. I just figured the nice folks here would likely have more experience with codecs and such than elsewhere!

(That, and, if I can build my own replacement Disney+, I would definitely want to share with friends.)

sxan,
@sxan@midwest.social avatar

I’m sure you’re right.

shrugal,

Best tip I can give is to use a tool that’s made for this task, like Tdarr/FileFlows/Unmanic. They take care of all the complicated issues like encoders, ffmpeg parameters and parallel processing on multiple nodes, so you only have to handle the things you actually care about.

dan,
@dan@upvote.au avatar

Unmanic is way easier to understand than Tdarr. I use it to transcode DVR recordings recorded using Plex and a HDHomeRun tuner. Digital TV uses MPEG2 which has pretty large file sizes.

RvTV95XBeo,

You’re about to embark on a massive journey and I wish you the best of luck :D

Compressing 110 4K movies is going to be a long encode time, but it will be so nice to digitize that collection.

First question: if you already have them ripped and stored, do you even need to re-encode them? If you have a powerful enough Jellyfin server for transcoding, that may be enough.

That said, if you’re looking to optimize for space and quality, there’s some questions to ask:

  • proprietary (but ubiquitous) HEVC or the emerging open standard AV1? It’s going to be a lot easier finding tips/guides for HEVC.
  • Constant Rate Factor (CRF), or Average Bit Rate (ABR). CRF tends to be more straightforward if you have a varied collection of content because it let’s the encoder choose settings to deliver consistent quality. 2-pass average bit rate is good if you have a target size or compatibility in mind - great for squeezing out the absolute optimal quality if you’re trying to fit on a Blu-Ray disc or meet certain streaming criteria, but 2x the encode time (hence “2-pass”) can be a tough sell.

Stick with software encoding, steer clear of hardware solutions like NVENC. They’re crazy fast, but inconsistent quality.

There’s some guides out there on converting DV to HDR 10+ etc, but I don’t have a lot of experience with that process.

fenndev,
@fenndev@leminal.space avatar

The issue is storage costs. Currently they (and some Blu-ray shows I ripped) are taking up just over 12TB. I bought all of these movies when I had money to spend on stuff like that, but money is short and times are tough. “Storage is cheap” but my wallet is cheaper right now, aha.

RvTV95XBeo,

Look, I definitely get it, but just a note, encoding 100+ 4k HEVC movies is going to take months of time at full PC loads. It’s a subtle cost, but depending on your electricity rate, running a 200W PC 24/7 for 3 months straight may hurt your bottom line more than it helps.

If you already have the storage, the cheapest option may be just to keep it. If you’re looking to increase the size of your collection it may be about even to save 6 TB via encode and to buy an extra 6TB disc, depending on your setup

jbrains,

I am definitely keeping an eye on my electricity bill right now, since I have Unmanic running full speed and it’s about 3 weeks into a 4-month job that might only save 1 TB.

DaGeek247,
@DaGeek247@fedia.io avatar

I did a comparison trying to find where my personal "good enough" and techinically indistinguishable crf levels were at a little while ago. It may be worth looking into as a start. I've never really touched hdr before though.

Comparing compression levels

reddthat,
@reddthat@reddthat.com avatar

If you want to do an open codec, use AV1 codec and Opus for audio.

github.com/master-of-zen/Av1an and use aomenc for the av1 codec.

Ffmpeg doesn’t have good av1 support.

Lemmchen,

With current hardware support I would advise against using AV1 or even H.265.

I’d recommend to experiment with H.264 and low CRF values to see what quality loss and file size OP is comfortable with: trac.ffmpeg.org/wiki/Encode/H.264

dan,
@dan@upvote.au avatar

What hardware doesn’t support H.265?

Lemmchen,

Old smartphones, old CPUs/GPUs, some SBCs.

dan, (edited )
@dan@upvote.au avatar

How many of those are you streaming video to, though?

Intel iGPUs have supported H265 since 7th gen, which is 8 years old now (released in 2016). Nvidia added support the same year, starting with the GTX1050. Even the Raspberry Pi 4 supports hardware-accelerated H265.

DaGeek247,
@DaGeek247@fedia.io avatar

God no. X264 is way worse than x265 is way worse than av1 for quality by size.

Yes, everything made in the past 15 years can do x264, but that does not mean it is a good idea. Only do x264 if you have a specific device that needs it. Otherwise, x265 is a better choice for long term storage.

Lemmchen,

You’re right, I forgot about the on-the-fly transcoding abilities of Jellyfin for a moment. But still, the server should be able to handle whatever codec they choose in hardware.

fenndev,
@fenndev@leminal.space avatar

AV1 is definitely what I’d like to do. I’m not aiming for maximum compatibility; small file size and high quality encodes are my goal. I can transcode if needed.

Lemmchen,

Keep in mind at least your server should be able to decode whatever you choose in hardware, so AV1 might still not be a good idea depending on your current hardware and upgrade plans.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • piracy@lemmy.dbzer0.com
  • rosin
  • DreamBathrooms
  • everett
  • magazineikmin
  • InstantRegret
  • ngwrru68w68
  • Youngstown
  • mdbf
  • slotface
  • vwfavf
  • tacticalgear
  • thenastyranch
  • kavyap
  • osvaldo12
  • provamag3
  • ethstaker
  • GTA5RPClips
  • khanakhh
  • Durango
  • tester
  • normalnudes
  • cisconetworking
  • modclub
  • cubers
  • anitta
  • Leos
  • megavids
  • JUstTest
  • All magazines
    •