Not speaking on any legal authority or anything but I would think we have the expectation of a right to detect on public property as long as we are not being destructive and leave the area in as good if not better condition when we leave. That is why, unless it is specifically posted, I just go ahead and search at will. I feel that I use good respect and common sense with my detecting and all and I try not to leave any reason for anyone to question my activities. I don't go during school hours or when children are using the park etc. I don't search in manicured areas, gardens etc. Personally, I only search areas where there is an expectation of digging anyways. The chips or sand in a play lot or a schoolyard sand box etc. Those are areas where, when I am done, I can refill my holes, tamp them down with my foot an a few hours later when the dampness from the fresh dug hole dries out or the wind has drifted the area over, you don't even know that I was ever there.
However, there can be official rules, laws or ordinances that might prohibit or restrict that right or access. To know for sure, you would need to contact your city or village parks district, the school district, maybe police department or other authority. Usually, unless clearly posted, if you fall into one of those situations, you will be told to leave. That is why your response was a proper one. You say you didn't challenge the authority of the custodian or create a confrontations. Now, if it were me, I would inquire with the school district offices or the principal of that school as to whether there is, in fact, a law, ordinance or something. This may be an issue that has never officially come up before and someone was making rules on the fly. They may just need the assurance that you are not some hack that is going to leave gopher holes all over their soccer field or leave holes in the tot lot for some kindergarterner to trip in and break an ankle. There could be outright bans in some areas, Especially around special facilities or areas like athletic fields, stadiums, village offices and facilities.... For example, detecting the front lawn of village hall or the police or fire department might raise a few unsympathetic eyebrows. If a law doesn't exist, you can be sure that one soon will. Some schools might have similar restrictions. Note also that some public parks can be specifically posted also.
One last point, as I think about it, I wouldn't jump too high on the list of authorities. Start with the specific school and check with the principal or other authority. The higher you go, the more likelyhood that you will cause some curiocity that may lead to an over reaction on a higher, more wide spread level. If you go to the police, you might just be told to check with the school district or school anyway. Start with the school and see if it may be an issue of their own policy. Avoid rocking too many boats, I guess.