Richey said it was a joy to feel Scottish rain on his face
Richey, who was freed from prison in the US last week following a plea deal, is now back in Edinburgh.
He told newspapers he was devastated that Cynthia Collins died in the Ohio fire but insisted it was not his fault.
He told The People: "Nothing can bring back that little girl but I've been rotting in my own hell for 21 years."
The 43-year-old's conviction for the arson attack was overturned last year.
At the Putnam County Common Pleas Court in Ottawa, Ohio, on Monday, he pleaded no contest to charges of attempted involuntary manslaughter, child endangering, and breaking and entering.
At the hearing, Cynthia's aunt Valerie Binklay said he had fooled no-one and that he would "burn in hell".
In an interview with the People newspaper, Richey insisted: "I did not kill that little girl.
"I'm devastated she died but it wasn't my fault."
Richey showed little sympathy for Cynthia's family.
He said: "I'm an innocent man, so if Valerie Binklay and the rest of them want me to fry for something I never did, why should I feel sorry for them?"
Speaking to the paper about the moment he was reunited with his mother Eileen, he said: "I held her so tightly I thought I would crush her.
"Even though I'm a man I felt I was her little boy again. In many ways she's suffered the most. Not knowing if she'd ever see her son again must have been agony for her."
Richey is already said to be finding things vastly different since he was locked up - including the price of a pint rising from about 60p in the mid-1980s to £3.50 today.
He said he felt like he was "stuck in a time warp" - citing his pin-ups as the Nolan Sisters and Sam Fox.
During his time on death row he said he would often spend up to 23 hours in his 6ft by 10ft cell.
Lack of exercise and a poor diet caused him to put on weight and he claimed "caused havoc" with his health - leading to heart problems.
But he admitted to the paper to making his own "hooch" in jail.
He also said at times he had felt like cutting his own wrists or throat.
Richey said he wanted to remarry ex-wife Wendy Amerud, 50, - who divorced him before the fire and start a new life in Scotland with son Shaun, 22.
He now hopes to be able to live a quiet life.
"I will never be able to forget my past but all I want is to enjoy the simple things.
"Just feeling the Scottish rain on my face this week has been a joy," he said.
In a separate interview with the Mail on Sunday newspaper, Richey said he could "never intentionally harm a child".
He said he knew and "adored" Cynthia Collins and would have done anything to save her.
Richey said: "It has been a very bitter experience finding myself in a situation where I was going to lose my life because I had been held responsible for the loss of hers - a child I played with as I had other kids in her street and looked after her."
In 1994 Richey was one hour away from the electric chair but was granted a stay of execution.
He told how he had ordered the Scottish dish of haggis and neeps as a last meal but was given steak instead.
Richey, a former US marine, revealed that prison officials had given him a choice on how he wanted to die - the electric chair or lethal injection.
"I said if they were going to kill me that was a decision they would have to make on their own," he said.
On the day of the fire, Richey admitted he was on a "high" after taking prescription drugs and collapsed into a nearby bush.
He claimed he attempted to save the child when he awoke to discover a fire in her flat.
"I tried to climb up on the balcony but was stopped by a neighbour," he said.
"All I could think of was Cynthia and trying to save her at that point."