MADISON, Wis. -

The federal office that helps refugees settle in the United States has placed 50 unaccompanied children in Wisconsin since the beginning of the year, but the government has asked both Milwaukee and Madison if there are locations that additional children could be housed on a temporary basis.

But four republican lawmakers sent a letter to the federal government this week asking them to withdraw their request to house the children in Wisconsin.

"We're being told by the federal government this is a temporary situation but placing them so far north in a state like Wisconsin makes it appear much less temporary," said Rep. Joel Kleefisch, R-Oconomowoc, one of the four lawmakers to send the letter.

Kleefisch, along with Rep. John Nygren, R-Marinette, Rep. Dave Craig, R-Big Bend and Assembly Speaker Robin Vos, R-Burlington, sent the letter Thursday to the Department of Homeland Security.

They said they were concerned about the drain on local and state budgets.

"I believe Wisconsin's priorities should be Wisconsin children," said Kleefisch. "And the priority of the children who have come here to the state from southern countries should be making sure they are with their parents."

The City of Madison has been asked to potentially house children and offered up a former grocery store and soon-to-be closed Shopko as potential locations. 

Democratic Rep. Chris Taylor, D-Madison, says state legislators should stay out of the situation.

"It's not my role as a state legislator to dictate to local communities humanitarian efforts they may want to provide to people in need," said Taylor.  "It's up to the community to decide and I'm very proud that my community here in Madison is looking at ways we can assist needy children who are alone and afraid at our border."

Governor Scott Walker said Friday that he wasn't entirely opposed to providing the temporary shelter for children, but was concerned that housing them for very long could eventually "drain the entire system."

The Office of Refugee Resettlement released a state-by-state breakdown Thursday showing where children had been placed. It did not provide specifics on their locations.

The office said it tries to place children with a parent, relative or family friend when possible.

Immigration officials have been overwhelmed by tens of thousands of unaccompanied children fleeing from gang violence in Honduras, Guatemala and El Salvador.