The former Union minister's remarks at an event in Delhi on Tuesday are being seen as a criticism of the party leadership at both the central and state levels.
Choudhary assumed office earlier this month, becoming the first BJP leader to head a government in the state after Nitish Kumar stepped down upon being elected to the Rajya Sabha.
Kumar had been away in Delhi a day ago when he was summoned back to Patna with the message that Nabin, who got elected to the Rajya Sabha two weeks ago, wanted to vacate the assembly seat.