Consider scale (the styling ‘rule’ is that the scale of your handbag should mirror your frame, i.e don’t wear a chunky, short-handled bag if you’re thin and willowy, and vice versa. Of course also remember that rules can be bent – even broken – successfully), what level of formality you’re trying to present (material of the bag, shape, stiff versus slouchy and so on… and if you’re going for an interview obviously don’t use that dog-eared, beaten up handbag which might look great with your boho outfit), colour (do you want it to contrast and stand out or blend and tone?), detailing (buckles and clasps – again the general styling rule is that more angular detailing will suit angular people, and more rounded detailing, curvier people. And it goes without saying that muddling up gold handbag detailing with silver jewellery, belt buckles, zips on clothing and so on just looks sloppy). Bear in mind the practicality factor too (i.e. clutches look great but they’re useless if you’re going to also have to hold a cocktail and a plate of canapés, and a tiny bag may look very chic but the whole look will be ruined if you add a tote bag as you actually need stuff with you).
I’m a big fan of using a handbag to bring some interest to an outfit. My two current favourites are a muted gold Jerome Dreyfuss grown up female Indiana Jones-esque slouchy long strapped-shoulder bag (called, slightly strangely for me as it’s also the name of my brother-in-law, Igor), and a boho Toureg desert bag like this one with metal and beady detailing, from Tosca&Salome. But you also need a ‘sensible’ / plainer one (or, obviously, multiple) in your wardrobe. Hopefully the collage will give you a bit of inspiration, my favourite is the vintage one, carried by Plum.