Seems illegal. You should really go to your principal with that, OP!

Sue for defamation

in my school some company came around to take the school pictures. it had nothing to do with yearbook and i would know... i was in yearbook for 2 years so either it was the company who used the picture (which you should sue) (also make sure you didnt sign a contract some where along the way saying they are allowed to use your picture for ads) or it was someone bullying you (which i would talk to someone to take care of it) to find out if it was a bully i would look if the ad is legit. either way i would so something about it. i would be uber pissed if that was my picture for a picture retake.

That was really rude of your school to do

I would tell your parents. If they don't help try talking to a teacher you trust. That can be really damaging to self esteem and everyone knows in high school it's the most fragile.

You can sue as they have not asked your permission to use your picture.

They're legally required to ask your permission to use your picture. Especially if you're a minor.

You should not have signed the release. If you or your parents didn't (if you're under 18) then you have a case. But most school pictures have a release to sign with it. Source: I am a photographer and did school pictures for a few years.

You could probably sue for slander.

Jokingly ask them for a free retake, they might take you up on it.