I think you need a good hat for one. I don't know names of styles of mens hats very well. But I am sure you will know them when you see them. Layers and proper fit. Don't forget accessories like one of a kind cuff links, the perfect handkerchiefs, etc.
What may help is that there are many, many Victorian sites around. Many that specialize in mens fashion as well with a lot of historical reference and sketches from old sources, etc. If you look at real Victorian clothing enough you may be able to see some of the base elements pretty well and be able to add your own darker style to that general.
Stuff like
http://www.victoriana.com/Mens-Cloth...lothing-1.html
here is a late Victorian hat (although I was looking for one a bit earlier)
http://www.victoriana.com/Mens-Clothing/mens-hats.html