Bike crime in Tokyo is certainly present, so locking up your bike is always a good choice. Because so many people ride bikes in Tokyo, the registration system is somewhat strict, and police will enforce the system by checking bike registration at random, to make sure that the bike belongs to you.
At best, this system acts as a deterrent to prevent bike theft. Someone who is thinking of stealing a bike now has an extra thing to consider: the registration sticker on the bike. The sticker can be removed, however it will make it difficult to register again in some cases.
Despite having a decent registration system to protect your bike, you should always lock up also. Most people just lock the back wheel of the bike, or simply lock the frame to a pole. This is very different than places in America for example where you must lock your bike frame and both wheels to a secure pole with a heavy, cut-resistant lock. Even with a very expensive roadbike in Tokyo, just make sure it is registered and locked in some small way.
I have been riding my bike everyday for 3 1/2 years with only a very simple combination lock.