Zimbabwe was 158-4 and 233 runs behind Bangladesh's first-innings score of 391 at stumps on day two of the second test at Harare Sports Club on Friday.
Bangladesh dismissed Brendan Taylor for 36 late in the final session for an important breakthrough after the Zimbabwe skipper made two centuries in his team's strong win in the first test.
Zimbabwe was 34 short of the follow-on by the end of play, with Malcolm Waller 30 not out and Elton Chigumbura unbeaten after a rapid 45 off 49 balls with five fours.
The tourists have still fought back strongly over the first two days of the final test after a heavy 335-run defeat last weekend.
After posting a good total, Bangladesh removed Zimbabwe openers Regis Chakabva and Vusi Sibanda inside 13 overs and Taylor holed out to midwicket off Sohag Gazi after a typically watchful 104-ball innings.
Bangladesh earlier added 91 to its overnight score, with Nasir Hossain eventually out for 77. He was helped by lower-order contributions from Sohag Gazi, who made 21, and No. 11 Robiul Islam, who was 24 not out.
Hossain and Gazi put on 51 for the eighth wicket to push Bangladesh close to 400 after earlier half-centuries by Shakib Al Hasan and captain Mushfiqur Rahim on Thursday. Hossain clattered 11 boundaries in a belligerent 77 from 103 balls before legspinner Graeme Cremer bowled him round his legs.
Cremer (2-109) and Keegan Meth, with 2-41, backed up seamer Chigumbura's 3-75 but Bangladesh put itself in position to level the short series between the two lowest-ranked teams in test cricket after the Zimbabweans put the visitors in to bat.
Zimbabwe's reply was shaky as Chakabva and Sibanda edged behind to wicketkeeper Rahim off Robiul Islam to depart within three runs of each other. Hamilton Masakadza was bowled by spinner Shakib for 14 and Zimbabwe was 45-3.
Taylor temporarily steadied Zimbabwe, but he suddenly hit out at Gazi and was caught by Shakib in the deep. Chigumbura responded by racing to 45 and Waller dug in alongside him to take the home team to stumps.