The wait sweetens the drink
Timing is everything when toasting in Japan. The toast is the official start to the party, rather than simply an acknowledgement of something or someone. Parties in Japan tend to be well-planned, and often the “wonderful and lucky opportunity” for everyone to be gathered despite busy schedules, etc, is stressed to the point of corniness. However, the system ought to be respected, and even if you are waiting for more people to show up to the event or party, you probably shouldn’t start drinking before the official “toast (kanpai)” has taken place. It may be painstaking at times, but it’s proper manners to wait it out, and be “one” with the party through the ceremonial toast. Then you can proceed to get wild, if you choose to do so with your fellow Japanese attendees.