Police said Ghosh was being taken to West Bengal by road. He is likely to be produced before the Barrackpore court in North 24 Parganas district on Friday.
Ghosh, the former MLA of Panihati, was arrested on charges of destruction of evidence, criminal conspiracy, and forcing someone to carry out an act against their will.
The new CBI investigating team on Thursday submitted a detailed status report on the ongoing inquiry and sought more time to complete the investigation.
The bench directed the CBI to submit all the materials collected in connection with the investigation before the court on the next date, in a sealed cover.