Pedestrians battled strong gusts and rain whipping around high-rise buildings on Federal Street in the financial district at noontime Wednesday.