Because they can remember your face. Also apparently they hold grudges and will plan and conspire with one another in order to get revenge. According to this article crows:
1. Can remember your face.
2. They conspire with one another.
3. The have a incredible memory.
4. They use tools for problem solving.
5. They plan (or plot if you are more of a paranoid type).
6. They have adaptive behaviour.
So I realized two things from this article. First, it seems crows and I have a lot in common. Secondly, I am not being paranoid when I have thought a crow was watching me......it really was.