Geidam is the hometown of Yobe state governor Ibrahim Geidam.
Witnesses said people were trapped in their homes during the attack. It was unclear whether there were casualties.
State police commissioner Sulaiman Lawal confirmed the city was under attack on Saturday night but did not provide any further details, the Associated Press reports.
The men, armed with Kalashnikov rifles according to Agence France Presse, fired into the air before making off with money taken from the vaults of the bank.
Boko Haram: Timeline of terror
- 2002: Founded
- 2009: Hundreds killed when Maiduguri police stations stormed
- 2009: Boko Haram leader Mohammed Yusuf killed
- Dec 2010: Bombed Jos, killing 80
- 2010-2011: Dozens killed in Maiduguri shootings
- May 2011: Bombed several states after president's inauguration
- Aug 2011: UN HQ bombed in Abuja
- 4 Nov 2011: Dozens killed in north-eastern town of Damaturu
He said the men were on their way to the town's prison.
Earlier this month Boko Haram carried out an attack in Damaturu which killed more than 100 people.
In August it bombed the UN headquarters in Abuja, killing 24 people.
Boko Haram means "Western education is forbidden". Parts of the group say they want Sharia, Islamic law, more widely applied across Nigeria, but correspondents say most factions are focused on local issues.