x265

• ctu=32 for 1920x1080
• no-sao=1
• limit-sao=1
• b-pyramid=0
• aq-mode=3
• mcstf=1 Motion-compensated spatio-temporal filtering. Give it a try

x264

too many lookahead threads significantly degrades lookahead accuracy
https://code.videolan.org/videolan/x264/-/blob/master/encoder/encoder.c?ref_type=heads#L1294

x264 has a complex lookahead module designed to estimate the coding cost of frames that have not yet been
analyzed by the main encoder module. It uses these estimations to make a variety of decisions, such as adaptive B-frame
placement, explicit weighted prediction, and bit allocation for buffer-constrained ratecontrol. For performance reasons, it
operates on a half-resolution version of the frame and calculates SATD residuals only, doing no quantization or
reconstruction.

It is important to note that, in x264, the regular B-frame quantizer offsets (--pbratio) are disabled
when MBtree is on, since they serve the same role.

One might assume similarly that macroblock-tree can replace keyframe quantizer offsets. However, testing
suggested this was not the case

Benchmark Software Decoder

ffmpeg -benchmark -i H264.mkv -an -f null -

Important AMD encoder settings

PASceneChangeDetectionEnable=false PALookAheadBufferDepth=41 MaxNumRefFrames=4

• PASceneChangeDetectionEnable=false (default=true) - adaptive I-frames, Scenecut, requires Lookahead
• AdaptiveMiniGOP=true (default=true) - adaptive B-frames, B-adapt, requires Lookahead
• BReferenceEnable=true (default=true) - Use B-frames as Reference frames

MaxNumRefFrames=4 =4 for High 4.2 Profile (1920x1080@60)

https://github.com/obsproject/obs-studio/blob/master/plugins/obs-nvenc/nvenc.c

https://github.com/obsproject/obs-studio/blob/master/plugins/obs-ffmpeg/obs-ffmpeg-nvenc.c

obsproject/rfcs#7

Important NVenc settings

AVC
lookaheadDepth=32 useBFramesAsRef=1

HEVC
lookaheadLevel=3 lookaheadDepth=32

Note: NV_ENC_LOOKAHEAD_LEVEL_0 has the highest performance,
while NV_ENC_LOOKAHEAD_LEVEL_3 has the highest quality. It increases bitrate significantly and should not be used.

https://github.com/obsproject/obs-studio/blob/ef93ef6950df0a5435c2756c11ace92d15e00d52/plugins/obs-nvenc/nvenc-properties.c#L213

useBFramesAsRef=

=1 Any B-frame can be a reference frame

=2 Halved - Uses the following logic:

4 B-frames = 2 reference frames
3 B-frames = 1 reference frame
2 B-frames = 1 reference frame
1 B-frame = 1 reference frame <-- untested, but should be logical to clamp to 1

Show all supported encoders in FFMPEG:

ffmpeg -encoders

Show all encoder options:

ffmpeg -h encoder=hevc_nvenc

Lossless: NVenc

ffmpeg -y -benchmark -i INPUT.mkv -c:v hevc_nvenc -tune lossless -pix_fmt p010le -an OUTPUT.mkv

Almost Lossless: Apple ProRes 422 High Quality

ffmpeg -y -benchmark -i INPUT.mkv -c:v prores_ks -q:v 0 -profile:v 3 -bits_per_mb 1024 -vendor apl0 -pix_fmt yuv422p10le 1024.mov

Note:-bits_per_mb can go up to 8192. Higher = less lossy.

Lossless: UT Video

ffmpeg -y -benchmark -i INPUT.mkv -c:v utvideo -color_primaries bt709 -color_trc bt709 -colorspace bt709 -color_range tv OUTPUT.mkv

AV1 Basically lossless settings:

ffmpeg -y -benchmark -i INPUT.mkv -c:v libsvtav1 -svtav1-params "enable-tf=0:mbr=10M:tile-columns=2:tile-rows=0" -g 240 -qp 14 -preset 4 -pix_fmt yuv420p10le SVT_AV1.mkv

Libaom AV1 settings:

ffmpeg -y -benchmark -i TEST.mkv -c:v libaom-av1 -pix_fmt yuv420p10le -g 240 -cpu-used 4 -auto-alt-ref 0 -aq-mode 0 -tile-columns 2 -tile-rows 0 -row-mt 1 -crf 16 -pass 1 libaom-pass1.webm
ffmpeg -y -benchmark -i TEST.mkv -c:v libaom-av1 -pix_fmt yuv420p10le -g 240 -cpu-used 4 -auto-alt-ref 0 -aq-mode 0 -tile-columns 2 -tile-rows 0 -row-mt 1 -crf 16 -pass 2 libaom-pass2.webm

NVEnc by rigaya
Docs

High Quality Example:

NVEncC64.exe --codec hevc --cqp 18 --preset p7 --output-depth 10 --lookahead 32 --ref 8 --split-enc forced_2 --colormatrix bt709 --colorprim bt709 --transfer bt709 --colorrange limited -i INPUT.mkv -o OUTPUT.mkv

190 FPS with 2560x1440 input from the RTX 5070 Ti

Lossless:

NVEncC64.exe --lossless --colormatrix bt709 --colorprim bt709 --transfer bt709 --colorrange limited -i INPUT.mkv -o OUTPUT_Lossless-avc.mkv

ffmpeg -y -benchmark -i TEST.mkv -c:v libx264 -x264-params "ref=8:keyint=240:bframes=3:b-adapt=2:rc-lookahead=60:aq-mode=3:me=umh:merange=24:subme=9:trellis=2:lookahead-threads=2:threads=16" -crf 14 -pix_fmt yuv420p10le AVC_14.mkv

https://developer.nvidia.com/video-encode-decode-support-matrix