But New Zealand seamer Tim Southee struck back early in dismissing debutant opener Dimuth Karunaratne lbw for a duck as the hosts were 9-1 at close of play.
Southee made good use of the new ball and had a delivery swing sharply in to the left-hander and hit the pad. Tharanga Paranavitana was yet to score batting with night watchman Suraj Randiv, who is on 3.
Brendon McCullum top scored for the Kiwis with 68 runs and shared a promising 90-run partnership for the fourth wicket with Daniel Flynn (53) after the tourists were reduced to 40-3 after the tourists' captain, Ross Taylor, elected to bat first after winning the toss.
It was a good toss to win for the visitors as the pitch at Galle has traditionally helped batting in the first three days and favored spin bowling in the fourth innings. However they wasted that chance.
"At Galle when you win the toss and bat first, you're after a total in excess of 400. We weren't able to do that today, but we have to address that," McCullum said.
"We're going to get an opportunity to bat again in this test match and we have to make sure we did a lot better than we did today. Looking forward to tomorrow, we've got a big first hour in the morning to try and expose the Sri Lankan middle order and if we can do that, today's misfortune will be a little bit easier to handle."
Herath said the pitch was still a good batting track despite the flurry of wickets.
"Not all my wickets came from turning balls. If we get a big score tomorrow we are in the game," he said.
Herath took three quick wickets in the middle session to hand back control to Sri Lanka and send New Zealand in to tea on 155 for six despite earlier being safely placed on 130-3. Herath took the wickets of Doug Bracewell (12) and Kruger van Wyk (28) in the final session to finish with the best bowling figures of 5-65.
Seam bowler Shaminda Eranga, who gave his team a good start, finished with three wickets for 51.
Earlier, McCullum and Flynn (53) lifted New Zealand from a poor start until Herath broke their partnership. The pair spent two hours together and faced 198 deliveries.
McCullum was beaten by a flighted delivery from Herath and was bowled. His 125-ball innings included two sixes and eight boundaries.
James Franklin struggled in scoring three off 43 balls before being trapped lbw to Herath. Flynn was out for 53, caught by wicketkeeper Prasanna Jayawardene off Herath with just three deliveries remaining before the scheduled tea break. His watchful innings came off 152 deliveries and included seven boundaries.
Eranga took two wickets in three deliveries in the first session to give the hosts a promising start. He halted New Zealand's progress when he removed Guptill for 11 and No. 3 batsman Kane Williamson for a duck, both caught at slip.
Taylor was bowled by seamer Nuwan Kulasekara for nine off an inside edge.
From Our Network
- Ayaz Promised He Wouldn't Speed, Says Azhar's Friend Jwala Gutta
- Watch: 'You're not happy India won', captain cool Dhoni fumes at journalist
- IPL 8: Glenn Maxwell has disappointed himself: George Bailey
- Sachin Tendulkar prays for flood victims, says Chennai is fighting back
- Ahmed Shehzad acted like Afridi's servant: Former PCB chief
- Why doesn’t farmers’ suicide trouble you as much as Pratyusha’s, asks Nana Patekar
- 'Find Your Beautiful': Radhika Apte explains real meaning of beauty
- Actor Harshvardhan Rane finds it challenging to romance three heroines in 'Maaya'
- Shah Rukh Khan is really impressed with his 'Raees' co-stars
- Kapil Sharma returns tonight: 5 reasons that make the first episode special