Carrick put United ahead in the seventh minute as Alex Ferguson's team looked determined to put last season's disappointing group-stage exit behind it.
Carrick received a pass in the area and was then tripped by Fernando Muslera as he rounded the Galatasaray goalkeeper, but maintained his focus and slotted the ball into the empty net while sitting down.
But Galatasaray pegged United back after that and were unlucky not to equalize.
The Turkish club hit the woodwork three times, had two penalty appeals rejected and forced a couple of good saves from David de Gea to leave Old Trafford scoreless.
"It's just great to get off to a winning start. Off the back of last year, not qualifying, it was vital we did," Carrick said.
"We defended well in the second half and we're delighted to get the three points and move on."
Nani could have made the game safe for United in the 52nd but missed a penalty when his weak spot kick was saved by Muslera.
"I think we expected a tough game. Galatasaray have a lot of experience," Ferguson said.
"There was a lack of concentration at times and it's unusual for us to be as frivolous with the ball. In the second half we were more solid."
Darren Fletcher came on as a substitute for United, marking his return a career-threatening bowel condition.
Fletcher came on for Paul Scholes in the 89th for his first appearance since Nov. 22, 2011, and was given a standing ovation by the Old Trafford crowd.
Robin van Persie and Shinji Kagawa made their European debuts for United and were both involved in setting up Carrick's goal.
But United's attack largely fizzled out after the opener, and De Gea had to rescue United with a superb double save late on, first denying Burak Yilmaz, then Emre Colak from the rebound.
From Our Network
- Alia Bhatt swims against the tide, defends Tanmay Bhat’s controversial video
- Epic! Kareena Kapoor cracks ‘high-end’ joke on ex-flame Shahid Kapoor
- Top 5 budget Reliance LYF smartphones
- Former cricketer Mohammad Azharuddin denies marrying Shannon Marie Talwar
- Know who said what on Tanmay Bhat’s controversial video mocking Tendulkar & Lataji