The Olympic brand fallout may have only just begun