Will Haley be mispronounced all the time?

My husband and I are considering the name "Hailey" for a girl, but I prefer the spelling Haley, which is not common but also not unheard of. I prefer the simplicity of taking out the "i", but also our last name starts with H and has some repetitive letters from "Hailey". So I thought it would be less matchy matchy to spell it "Haley".

My husband is concerned that people will see Haley and pronounce it Hallie or Holly. What are your thoughts? Is this less common spelling going to create frustration for our daughter for her whole life, or do you think it is an acceptable alternate spelling?