Hi mate, in terms of Sydney Oxford st is where you want to head quite a few gear shops. Daily Jocks has a pop up store on Oxford St which has quite a bit of gear, Bang and SOS are also good shops. Budgy Smuggler has a store in Manly where you can pick up for swimmers.
High street store really don;t have much choice, perhaps some standard issue speedos and undies.
Good luck and enjoy your trip!