Go to local supermarkets or try pick-your-own farms
Just keep in mind that fruits in Japan are relatively more expensive than other countries. So avoid buying them from department stores, supermarkets for expats, or Seijo Ishii that imports foreign foods, and go to one of your local supermarkets.
Or you can join a bus tour for PYO experience. You can buy fruits in bulk from local farmers at lower prices.
http://www.kudamononavi.com/picking/
(Japanese only)
Natchan