I don’t know if this actually does what I hope it does, but if I happen to watch a video that I don’t want recommendations on in the future, I manually delete the video from my watch history.

I don’t know if that really helps, but at least as a placebo it feels good to do.