Falling leaves, not always falling temperatures
Fall in Japan can still be quite warm. Of course, it all depends on where you are. If you are in Tokyo expect dayttime temperatures in the high 60s (F) or even higher in the early part of fall and even into late October. Nights usually drop to the lower 50s, sometimes a bit cooler.
Bring along a light jacket and a sweater. Remember, layers always work best.
If you're going into the mountain regions expect colder temperatures calling for heavy coats - a good quality down jacket won't bulk up your suitcase, but will keep you warm - scarves, hats, gloves.
Stop by any Uniqlo store and pick up some "heat tech" - this is very light weight, but very warm, long underwear. Available for both men and women. The price is very reasonable, around for top and bottom both, and it won't take up much room in your suitcase for your return trip, yet will keep you warm in Japan.