Well,....actually the home owner is responsible for the property right up to the curb. I suppose it depends in what state you live, but my experience has been the OWNER of the house is responsible. If someone trips and FALLS on the sidewalk in front of your house you are also responsible if it can be shown there was some negligence on the property owners part. He/she even has to keep the grass mowed. Thats just the way it is. Most home owners have insurance to cover such things. Its possible that this particular owner felt threatened, regardless of your explanation. Just let it go and move on.
RR