Hire a designer to create a custom logo for your company. There is no substitute.
Business cards are essential, and you should carry them with you always. I just ordered 5000 business cards for $115.00 from www.platinumprinters.com, so as you can see, they are cheap. Stock up.
Fictitious Name / Registration*
If you want people to be able to write checks to your company name, you need to register with your state. This will allow you to go in and open a checking account in your company's name. With this, you will use your social security number for the business registration and pay taxes as a sole proprietor.
Going a step beyond the fictitous name / registration, incorporating gives you legal protection from the company. You will obviously also be able to get a business checking account, plus, an EIN# will give your company a seprate tax ID# than your social security number.
With my company, I incorporated about a year after filing for the fictitious name, so my LLC ended up taking over the ownership of the fictitious name.
Business Checking Account & Checks
For the actual bank that you do business with, I'm guessing that most people will choose based on convenience and location, so I'm not going to try to plug any particular bank, here. I go to the bank that has a branch closest to my house.
Regarding checks, I order my checks from Walmart (online), you can't beat the price. $50 for 500 checks, the kind you stick through an inkjet printer and each check gives you 2 tear-offs.
My preference here is Quickbooks. In 2009, they had a $150 mail-in rebate, which actually made the software free. It was a one-day special through Staples.
Insurance: General Liability, Equipment, Professional*
Consult an attorney if you're not sure what type of insurance you should have.
*Disclaimer: The information you obtain on this site is not, nor is it intended to be, legal advice. You should consult an attorney for individual advice regarding your own situation.