Well I have 3 x Pi 4’s and don’t have heatsinks on any of mine but there again all 3 have official PoE HAT’s! 
Currently my Pi’s are all in the loft running;
Pi: 40°C (This is a spare Pi doing nothing other than sitting there kept updated)
Pi-Hole: 39°C (Handles @120,000 queries a day)
Pi-Air 48°C (running an AirNav FlightStick feeding FA, FR24 & RB
The PoE fans run intermittently only spinning up when needed and on Pi-Air for example are usually @30s off/5s on and although very quiet and can’t be heard I would have preferred a fanless solution so I tried some back-2-back tests;
The official RPi Foundation fan-cooled PoE HAT consistently ran @10-15°C cooler than the Uctronics Mini PoE fanless HAT.
The large heatsink from Pi-Hut was pretty much useless with the bare Pi benching at 58.2°C and after application of the 15x15x15mm CPU heatsink showing 58.4°C an increase of 0.2°C temperature over the non-heatsink Pi although I concede that could have been down to different processes running at the time but showing no benefit whatsoever.
My own experience from testing was definitely worthwhile and showed the best solution for me by far was the official PoE HAT with the added advantage of the Pi not having to be near a power outlet and so can be hung remotely on the end of a very long single ethernet cable if so desired which was ideal for getting Pi-Air up in the far end of the loft as close as possible to the antenna.
YMMV of course…
Kind regards,
-=Glyn=-