Researchers described ultrasound as a "promising candidate" in contraception.
However, far more tests are required before it could be used.
The concept was first proposed in the 1970s, but is now being pursued by researchers at the University of North Carolina who won a grant from the Bill & Melinda Gates Foundation.
They found that two, 15-minute doses "significantly reduced" the number of sperm-producing cells and sperm levels.
It was most effective when delivered two days apart and through warm salt water.
In humans, the researchers said men were considered to be "sub-fertile" when sperm counts dropped below 15 million sperm per millilitre.
The sperm count in rats dropped to below 10 million sperm per millilitre.
Lead researcher Dr James Tsuruta said: "Further studies are required to determine how long the contraceptive effect lasts and if it is safe to use multiple times."
The team needs to ensure that the ultrasound produces a reversible effect, contraception not sterilisation. As well as investigate whether there would be cumulative damage from repeated doses.
Dr Allan Pacey, senior lecturer in andrology at the University of Sheffield, said: "It's a nice idea, but a lot more work is needed."
He said that it was likely that there would be recovery of sperm production, but the "sperm might be damaged and any baby might be damaged" when sperm production resumed.
"The last thing we want is a lingering damage to sperm," he said.