Unfortunately one can get a type of cancer that to the pathologist looking at tissue under the microscope appears indistinguishable from ovarian cancer despite prior removal of the ovaries. This is known as primary peritoneal carcinomatosis, but it is thankfully quite rare. This affects a very small percentage of women and appears to be more common in women with certain risk factors such as the genetic mutations known as BRCA1 and BRCA2. However, this affects only about 5-10% of ovarian cancer patients.
It turns out that the lining of the abdominal and pelvic cavity, known as the peritoneum, shares some of its embryologic or developmental origin with the cells that make up the capsule (the "skin") of the ovaries. In some cases they have found disseminated tumor implants throughout the abdomen and pelvis, which look pathologically like ovarian cancer, in patients with small and benign appearing ovaries. Some have referred to this as the "small ovary syndrome."
Regardless, this is treated similarly to ovarian cancer.