The veteran Conservative MP said he met the president last week to reminisce about his father’s role as the last governor of Southern Rhodesia.
Sir Nicholas said they met for old times’ sake, adding: “It was purely for Auld Lang Syne and I’m glad I did it.”
But Labour MP Kate Hoey called the visit a “body blow” to people who had suffered under the Mugabe regime.
The 93-year-old president has been quoted in Zimbabwean press reports saying Sir Nicholas’s visit showed the British wanted dialogue with his country.
Ms Hoey, who chairs the All-Party Parliamentary Group on Zimbabwe, said it had given him a “huge fillip”.
“You can’t have a private visit which is then front page of the newspapers in Zimbabwe,” she said.
She said she would be urging ministers to make clear the Mid Sussex MP had not been visiting on their behalf.
Speaking on BBC Radio 4’s Today programme, Sir Nicholas said his father Christopher Soames would not have forgiven him if he had not tried to “call on” the president.
Sir Nicholas said the UK had “terrible unresolved issues” with Zimbabwe, but described the meeting as “a historic moment for me and my family”.
Responding to Ms Hoey’s criticism that the meeting would “pander to the vanity of a wily and ruthless dictator”, he added: “I met him purely and entirely in a personal capacity – there was no-one present, it was just him and me.”-BBC