MADISON, Wis. -

The charges against two men accused of assaulting a man downtown were upgraded to hate crimes two days after the alleged incident, police said.

The Madison Police Department said a 22-year-old Madison man was attacked downtown on Tuesday night by a pair of men who officers allegedly overheard saying they were doing "white boy work."

On Thursday morning, police said they had changed charges from disorderly conduct and trespassing citations to attempted robbery with a hate crime modifier.

In a report, police had said the man was attacked at a State Street bus stop near the intersection with Fairchild Street at about 11:20 p.m.

The victim told police he was talking on his cellphone with a friend and speaking in his native African language when two strangers approached and ordered him to "stop talking that way."

The victim told officers one man punched him in the face. Afraid, the victim ran and called police, the report said.

A sergeant out on routine foot patrol spotted the described suspects as they ducked down near a residence in the 400 block of West Mifflin Street. Police noted that area is marked with no trespassing signs.

Police said Chad E. Grubbs, 28, of Madison, was charged with attempted robbery party to a crime with a hate crime modifier and bail jumping.

Rodney A. Michaelis, 42, with no permanent address, was charged with attempted robbery and battery, both with hate crime modifiers.

Police had said officers overheard one man acknowledging the confrontation with the victim, but claimed the victim started it by saying derogatory things about "white people."

According to the report, later as the men were being taken to jail, the same suspect told his friend: "We did work tonight. We did white boy work tonight."

Police said the victim was shaken up but not injured in the attack, which he also said was unprovoked. The victim told detectives that the suspects also attempted to steal his phone during the assault, according to police.