I think I like the one with leaves better than flowers as it suggests more of an organic/plant-like approach. You mentioned you guys also do veggies and all sorts of plants, which is a market not to be ignored.
The logo is at the heart of your brand. It's a small, visual representation of what it is you do. When a customer thinks of you, they'll remember "flower" because that's what was in your logo (and people do remember visual cues more so than textual/verbal ones... we're just visual beings). If you want to brand yourself as a flower nursery, then only use flowers. If you want to brand yourself as an all-around nursery, you have to go the "plant" route and use something more leafy.
That's just my two cents though.