If you want to probe what encodings a VNC server supports, here's a rudimentory method. Run the below script (following the notes), subbing out the first encoding you pass in the array.

If it's not supported, you'll see an encoding log of 0 (the server fell back to raw). Otherwise you'll see the encoding you specified (or an error from rfb2 because it doesn't know how to deal with it, but that's fine).

https://chatgpt.com/share/692dfd34-f3ec-8013-a45a-4abb5506964c has some thoughts on "desirable" protocols. My sense is that ZRLE is the most modern standard for lossless. Tight might be a faster lossy alternative. H264 is probably awesome, but good luck finding a VNC server that supports it.

const rfb = require("rfb2");

// MODIFY: rfb2/rfbclient.js readFbUpdate() to log the rect.encoding when received

// https://datatracker.ietf.org/doc/html/rfc6143#section-7.7
const e = {
  Raw: 0,
  CopyRect: 1,
  RRE: 2,
  CoRRE: 4,
  Hextile: 5,
  Zlib: 6,
  Tight: 7,
  TightPng: 0xfffffefc,
  ZlibHex: 8,
  Ultra: 9,
  TRLE: 15,
  ZRLE: 16,
  ZYWRLE: 17,
  XZ: 18,
  XZW: 19,
  H264: 20,
  VAH264: 0x48323634,
};

// This pseudo-encoding may be required by the VNC server (VZVNCServer does)
const DesktopSize = -223;

const r = rfb.createConnection({
  host: "localhost",
  port: 5900,
  password: "passwd",
  encodings: [e.ZRLE, e.CopyRect, e.Raw, DesktopSize],
});

r.on("error", (error) => {
  console.log(error);
});