So, I have 2x RTX 3080's with 20GB VRAM each (modified), and I saw that ReBAR is entirely broken on these cards.
Fortunately, there is some sort of "fix/alternative" to atleast get some of that performance back, although it's largely untested, found on this reddit thread, reading the messages from u/Difficult_Tip_6669.
The repo mentioned on the reddit thread seems to have some PoC on how to get barlink working again, although the repository seems shallow and unmaintained. I do believe there is some sort of untapped potential here to get these cards some needed performance boost.
Update: I got it working and made a script for it to be installed easily and working on vLLM/SGLang:
https://github.com/fedes1to/nvidia-modded-p2p/