The vigilantes were following a call by the Third Way, a far right-party with suspected links to neo-Nazi groups, for its members to stop illegal crossings near the town of Guben on the German-Polish border.
Police seized the weapons carried by the 50 suspects and made them leave the Guben area late on Saturday and in the early hours of Sunday, a spokesperson said. Some of the suspects had travelled to the Polish border from other parts of Germany.
On Saturday, dozens of people held a vigil in Guben to show their opposition to the planned far-right patrols.
Germany has stationed an extra 800 police officers on the Polish border to control the flow of migrants trying to enter the European Union from Belarus, the interior minister was quoted as saying on Sunday.
"Hundreds of officers are currently on duty there day and night. If necessary, I am prepared to reinforce them even further," Horst Seehofer told the Bild am Sonntag newspaper.
Seehofer said there had already been 6,162 unauthorised entries into Germany from Belarus and Poland this year.
Last week, Seehofer said Germany did not intend to close the border with Poland, but on Sunday he said the country might have to consider reintroducing controls.
"If the situation on the German-Polish border does not ease, we will also have to consider whether this step needs to be taken in coordination with Poland and the state of Brandenburg. This decision will come to the next government," he said.
The three German parties working to form a coalition government say they aim to wrap up talks by the end of November and elect Social Democrat Olaf Scholz chancellor in December.
Many EU states accuse Minsk of sending illegal migrants across the border into the EU to put pressure on the bloc, which imposed sanctions on Belarus after President Alexander Lukashenko's disputed re-election in August 2020.
Lukashenko denies this and has blamed the West for what he says is a looming humanitarian catastrophe this winter after migrants were left stranded on the Belarusian-Polish border.