Carrick put United ahead in the seventh minute as Alex Ferguson's team looked determined to put last season's disappointing group-stage exit behind it.
Carrick received a pass in the area and was then tripped by Fernando Muslera as he rounded the Galatasaray goalkeeper, but maintained his focus and slotted the ball into the empty net while sitting down.
But Galatasaray pegged United back after that and were unlucky not to equalize.
The Turkish club hit the woodwork three times, had two penalty appeals rejected and forced a couple of good saves from David de Gea to leave Old Trafford scoreless.
"It's just great to get off to a winning start. Off the back of last year, not qualifying, it was vital we did," Carrick said.
"We defended well in the second half and we're delighted to get the three points and move on."
Nani could have made the game safe for United in the 52nd but missed a penalty when his weak spot kick was saved by Muslera.
"I think we expected a tough game. Galatasaray have a lot of experience," Ferguson said.
"There was a lack of concentration at times and it's unusual for us to be as frivolous with the ball. In the second half we were more solid."
Darren Fletcher came on as a substitute for United, marking his return a career-threatening bowel condition.
Fletcher came on for Paul Scholes in the 89th for his first appearance since Nov. 22, 2011, and was given a standing ovation by the Old Trafford crowd.
Robin van Persie and Shinji Kagawa made their European debuts for United and were both involved in setting up Carrick's goal.
But United's attack largely fizzled out after the opener, and De Gea had to rescue United with a superb double save late on, first denying Burak Yilmaz, then Emre Colak from the rebound.
From Our Network
- 5 interesting facts about Dinesh Karthik & Dipika Pallikal's wedding
- Sehwag trolls birthday boy Rohit Sharma in his inimitable style!
- Leicester City wins EPL title for first time in 132-year history
- Spectators sing the birthday tune for Virat Kohli in Mohali
- IPL 2016: KKR take on Pune, eye third consecutive win
- Raees Vs Kabil: Rakesh Roshan REACTS to Hrithik-SRK’s box-office clash!
- IPL: 8 hottest female anchors
- Will file witness list in National Herald case: Swamy to Delhi court
- AgustaWestland scam: How did Swamy get 'sensitive' documents, asks Congress
- West Bengal polls over, Mamata confident but high voter turnout gives hope to Congress-left alliance