The Australian was taking charge of his new club for the first time at the Emirates