Key is to greet in Japanese and with a sincere humble intent

Japanese greet often amongst themselves though it is not like in US or other countries where even if you dont know anybody people say "Hey hello - whats up" or something. But typically there are daily greetings like
Good Morning (Ohayo Gozaimasu) or Good Night (Oyasumi Nasai) etc. - which perhaps you can get a list on web. Then are different greetings based on work front or other situational aspects. Some greetings that I would want my foreigner friends to use while visiting Japan -
- Good day - Konnichiwa
- Thanks a lot - Doumo Arigato Gozaimashita
(eg. when somebody shows you direction or gives you something or wherever possible)

- It was a great help or You really helped me - Taihen osewa ni narimashita
(eg. you asked for some direction, the person tried to guide you and then before you move on you can politey thank him and say this greeting)

- Sorry / my apologies - Gomen nasai
(eg. you didnt see people standing in a line - and just came in between - then realized that people are standing behind you - quickly say Gomen nasai and try to move behind)

Key is to say it in Japanese and then with a sincere humble intent.
Firstly Japanese will definitely recognize the the fact that you have taken the efforts to talk in Japanese. Then if the intent is sincere and you sound humble and polite, then they will definitely go out of the way to help you out.
Initially you can use basic Japanese -as it might be difficult to switch between local colloquial Japanese and polite formal Japanese. Key is to speak slow and be polite.