(Health.com) -- Heart-attack patients who don't talk to their doctors about when it's safe to have sex again are likely to see a drop-off in their sex lives, new research suggests.

Doctors don't seem to be having "the talk" with most of their patients--especially if the patients are women. Only 46 percent of men and 35 percent of women discussed sex with their doctors when they left the hospital, the study found. Even fewer--about 40 percent of men and just 18 percent of women--ever discussed it during the following year.

Many heart-attack survivors who refrain from sex may be doing so out of fear, says the lead author of the study, Dr. Stacy Lindau, a professor of obstetrics and gynecology at the University of Chicago.

"They're not sure whether it's safe to have sex after a heart attack, and they worry it might trigger another heart attack or result in death," Lindau says. (In fact, sexual activity is a factor in less than 1 percent of heart attacks, by some estimates.)

Generally speaking, it's safe to resume sexual activity--or any mild-to-moderate physical activity--a few weeks after an uncomplicated heart attack, says cardiologist Dr. Nieca Goldberg, the medical director of New York University's Women's Heart Program, who was not involved in the study. "When I talk about it, I include it in a discussion about physical activity," she says. "That's what people are worried about."

Health.com: 28 days to a healthier relationship

But many cardiologists give inadequate instructions when it comes to sex and exercise, Goldberg adds. The study "clearly shows the need for us to do better," she says.

Lindau and her colleagues evaluated data from a long-running study of heart attack survivors. As part of that study, about 1,200 men and 600 women were surveyed and asked to recall their sexual activity before their heart attack, and one year after. The average age of the participants was about 60.

Compared to the patients who did talk to their doctor about sex, the men and women who did not were 30 percent and 40 percent more likely, respectively, to be having less sex than usual one year after their heart attack.

Health.com: What to do when heart disease hurts your sex life

Overall, men were more likely to be having sex than women. Nearly 70 percent of men reported some sexual activity during the year after a heart attack, compared to just 40 percent of women, the study found. (The women were also less likely to be married.)

Health.com: 9 surprising heart attack risks

"Older adults generally value sexuality, they see it as relevant, and they feel it's appropriate for doctors to address the topic," says Lindau. "But patients repeatedly tell us they think doctors should initiate the discussion. Doctors need to open the door for conversation."

It's not clear from the study why this isn't happening as often as it should. Some doctors may feel uncomfortable discussing the topic with their patients, especially if it's a male doctor and a female patient, Lindau says.

"It seems women doctors are better than men doctors at raising these issues," she says. "The distribution of the workforce of cardiologists is still predominantly male, and we still need more information on why male doctors may not be as good with these issues."

Health.com: 30 days to more satisfying sex

Doctors may also feel more comfortable discussing sex with men because there are proven FDA-approved treatments (such as Viagra) for erectile dysfunction and other sexual problems in men, Lindau adds, while similar treatments for female sexual dysfunction are limited.

The doctors shouldn't take all the blame, however. As Goldberg points out, it takes two to tango. Although they may be reluctant to raise the issue, patients who are concerned about sexual activity have to speak up, she says. "If your doctor doesn't bring it up, you need to be very specific and ask that question."

Health.com: 6 steps to rev up your marriage

The bottom line? Having a heart attack doesn't mean your sex life has to come to an end. "It is the norm to be sexually active before and after a heart attack," says Lindau. "I hope these findings empower patients to know they're not alone if they're having [sexual] problems, and to raise the topic with their doctor."

Lindau presented her research at the American Heart Association's Scientific Forum on Quality of Care and Outcomes Research in Cardiovascular Disease and Stroke. The study was funded by the National Institute on Aging and the National Heart, Lung, and Blood Institute.