General Rildo continuously grows stronger throughout the Baby Saga, and is able to triumph over Goku, Pan, and Trunks, but this happens before Goku achieves Super Saiyan 4. When Rildo is killed ...

Super Saiyan 4 was first introduced in Dragon Ball GT when ... but he gains his senses once his granddaughter, Pan, arrives and reminds him of who he is through her pleas; he returned to his ...

and it's only after Pan is able to remind him who he is that Goku is able to gain control of his new power, and shape it into Super Saiyan 4. Combined with how dire the circumstances presented by ...