Their silence spelled trouble for Rep. Mike Johnson, who needed almost every single GOP vote in order to hold onto the gavel.