Gather supporting documents that prove how long you've lived in the country, your intention to remain in Japan for a long time (or at least long enough to repay the loan), and financial stability in the form of a steady paycheck or proof of working at the same company for many years. There's no single method of gathering these documents, but you can use anything from pay stubs to a written statement from your company verifying the length of time you've worked for them. While not proof itself, it helps if you have a spouse or child who is a Japanese native to show you have a reason to remain in Japan for a long time. None of this guarantees you'll get a loan, but the more supporting documents you have when you walk in the greater your chances will be.
Establish a good credit history in Japan. This step is optional, but very helpful. However, it may be very difficult getting a credit card in Japan for the exact same reasons it's difficult to obtain a loan: lack of an already-established credit history. This seems like a double standard, but credit companies in Japan are often reluctant to extend credit to non-citizens regardless of the amount of time you've spent in the country.
Buy a registered "hanko", or rubber seal bearing your name, from a licensed hanko carver. This seal is what you use for any financial services requiring a signature, such as opening a bank account or applying for a loan. A handwritten signature is not considered valid in these cases, and neither is an unregistered hanko.
Apply for the loan at your business of choice. Your local bank is normally the first and most obvious choice for this process, but it's possible they would turn you down since you're not a Japanese citizen. There are alternatives—a foreign-owned bank (such as Shinsei) that's used to international clients may give you the option to use your credit history from your native country as proof. This option may help you if you've had trouble establishing credit within Japan. There are also independent lending institutions who conduct business with non-Japanese citizens every day, although their interest rates are usually higher.