do they make their first appearance in "The Reunion" episode? which would be episode 4
They made an appearance in the first episode, and yesterday Jerry and Elaine made an appearance.
That wasn't the first episode, the poster had it right... their first appearance is in Episode 4, "Reunion".