Dutch actor Robert Wolders, who captivatedAudrey Hepburn‘s heart and was with her until her death, died Thursday. He was 81.

Prior to being with Hepburn, he was married toWuthering Heightsactress Merle Oberon for about four years until her death in 1979 at age 68.

The actor recently talked to PEOPLE about his great love with Hepburn and clarified that he never proposed because they never felt the need.

“I felt she had two unhappy marriages, it was wonderful the way it was,” he said. “When Audrey would be asked, she’d also say, ‘Why mess with a good thing?’ I remember her saying to one interviewer it’s more romantic this way because it’s not another piece of paper, but out of loyalty to each other that binds us together. Had we been younger and wanted to have children, it might have been different, but that’s just not the case.”

Hepburn and Wolders in 1991.Ron Galella/Getty

F:PHOTOMediaFactory ActionsRequests DropBox47966#Getty Images2GettyImages-104078268.jpg

Wolders added: “She viewed us as being married. There were times in order to avoid confusion that she would present me as, ‘My husband, Rob,’ but never as, ‘my partner,’ or, ‘my life partner,’ or anything like that. What always pleased me or always sounded so nice was, ‘Have you met my Rob?’ Or, ‘This is my Rob.’”

Wolders also explained how they instantly connected when they met at the perfect time.

“We were ready for each other,” Wolders said of their near-instant chemistry. “At the time in our lives that we met, we had both made our mistakes. If chance would have had it that we would have met at an earlier stage, we might not have had the discoveries together that we did have and found those things in life together that were valuable to us at a later point in life when we were both more mature.”

After Hepburn’s death, Wolders datedGigistar Leslie Caron and then had a two-decade relationship with Shirlee Fonda, the fifth wife of late actor Henry Fonda, according toThe Hollywood Reporter.

source: people.com