You need to be clear who is your target audience. If you focus on 'anyone who needs a bookkeeper' you'll just be another bookkeeper and may not be memorable. That means you may not get referrlas and thesea re an importnat source of work. As are testimonials which you need to collect.
You'll want to build an advocate network - that is people who have the confidence to tell others about you. No one will come beating a path to your door - eevn if you spend a fortune on a website. Indeed, you don't need to. A simple one page site will do to start with. You could even create your own, quite simply, using a wordpress multi page blog.
What I would say is critical is a decent quality business card and a professional email address - not hotmail or gmail. They make you look like an amteur to most poeple.
Before you finalise what you'll put on the card, look at some of the others you've seen. Think about what works well and what's a waste of time. Can you use the back constructively for example? Don't bother spending time and effort on a 'logo'. When you're comfortably established you can decide on what your branding should promote. And your log will be just one part of that.
Here's a link so some other comments I've made about accountants' and bookkeepers' websites recently - with various tips and ideas that may be useful.
You might also want to consider linking up with one of the new style bookkeeping networks (as distinct from the professional bodies). These may be useful routes to source work.
Again, let me know if you'd like details.