Surely you would let this go on even if you became a couple. He was even grateous enough to let you come and watch the next time he and his friends has their way with your "innocent" crush.
MY NEIGHBORS WAS NICE ENOUGH TO TAKE US WITH HIM ON VACATION. IF ALL HE ASKS IN RETURN IS FOR MY WIFE TO NOT WEAR CLOTHES ON HIS PARTYS, I JUST HAVE TO DEAL WITH IT.