A Genetic Test Led Seven Women in One Family to Have Major Surgery. Then the Odds Changed.

When she was in her early 30s, Katy Mathes decided to check her cancer risk. A genetic test showed a mutation on a BRCA gene, which significantly raises a person’s lifetime risk of developing hereditary breast or ovarian cancer.

Thirteen people in the family got tested—her mother, her sister, cousins and aunts. Eleven had the mutation. Almost all did their testing with Myriad Genetics Inc., which introduced the first BRCA tests in 1996.

“I treated my test results like the Bible,” said Ms. Mathes, now 37, an elementary-school art teacher in Colorado. “There was no questioning the report.”

Ms. Mathes, who has one child, decided she would have no more. To reduce her cancer risk, she underwent surgery to remove her ovaries and fallopian tubes. So did her younger sister, their mother and four other relatives. Ms. Mathes and her sister also had double mastectomies.

This year, their mother sat them down at the table of their parents’ winter home in Florida. Two weeks earlier, her genetic counselor had called. The lab was no longer sure the variant is a...