Leaves form the vine and then when Zuko turns against him, but also when he tells Zuko he thinks of him as his own son. I think these were the saddest, but not the only sad moments.
The short of him in Ba Sing Sei when he begins to sing the song to the crying child and then finishes it by his son’s grave. That was a depressing scene.