The rest refers to the remaining people of Israel who survived when Jerusalem was attacked (see Zechariah 13:8).