The visitors were 66 for three in their second innings with an overall lead of 122 runs at the end of the fourth day at the Basin Reserve on Sunday.
Mominul Haque was on 10 while night watchman Mehedi Hasan was run out for one with three balls left of the day's play.
Mitchell Santner had started the collapse of three wickets for 16 when he bowled Tamim Iqbal for 25, before Neil Wagner had Mahmudullah caught by wicketkeeper BJ Watling for five.
Santner then ran Mehedi out with a direct hit to give his side confidence for the final day's play, with two of Bangladesh's top-order batsmen under injury clouds.
Opening batsman Imrul Kayes retired hurt on 24 after he appeared to suffer a pulled muscle while taking a quick single.
Captain Mushfiqur Rahim did not field in New Zealand's innings of 539 due to suspected broken fingers, though he was seen padded up to bat if required on Sunday.
Imrul took the gloves in Mushfiqur's absence and snared five catches, a record for a stand-in wicketkeeper, in New Zealand's innings that was anchored by Tom Latham's career-best 177.
Latham, whose previous best was 137 against Pakistan in 2014, had resumed on 119 with Henry Nicholls, who was on 35 and the hosts on 292 for three.
Both were untroubled as they took New Zealand to 347 for three before Nicholls was caught at leg slip by Mehedi off Shakib Al Hasan for 53.
Part-time off-spinner Mahmudullah then took two wickets in one over, the first from a wide long hop that Watling shaped to smash to bring up his 13th Test half century but succeeded only in nicking to Imrul.
Watling trudged off for 49 and was quickly followed by Tim Southee, who was trapped in front for one just four balls later.
Santner then made a career-best 73 before he was the last man out, bowled by Subashis Roy. It was his second Test half century.