A suicide bomber killed six persons at Redeem Christian Church of God (RCCG) on the outskirts of Potiskum on Sunday morning.
A Red Cross official and a witness confirmed the bombing.
“People were just going to the church when the bomber entered, otherwise the casualty figure would have been higher,” said Red Cross official Hassan Alhaji Muhammad, who visited the scene of the blast.
A hospital in Potiskum said it had received the bodies.
Suspected Islamist militant gunmen and bombers killed more than 200 people last week and 650 since 29 May, in a spate of bloodletting that prompted renewed outrage and calls for an international meeting to coordinate the offensive against Boko Haram.
French President Francois Hollande said the Boko Haram threat was getting stronger and that he was ready to hold a summit, following one in Paris in May 2014, to gather the leaders of countries fighting the insurgents.
New Nigerian President Muhammadu Buhari is due to visit his counterpart in Cameroon after the Muslim holy month of Ramadan and meet U.S. President Barack Obama on July 20, when the fight against Boko Haram is likely to be high on the agenda.
Potiskum is in Yobe state next to Borno, a remote state which is Boko Haram’s stronghold in Nigeria and borders Cameroon, Chad and Niger.