I wrote a lovely, semi-long reply and when I hit send the internet ate it all! That probably works out best for you though, because now I'm going to paraphrase it and save you time, I'm sure, haha:
Unfortunately when we hear 'treasure hunter' or 'metal detectorist,' we immediately think of sites we know of that have been looted, and we have a generally negative reaction to the very idea of such individuals. Having spent time in England, I'm aware the relationship could be better, and I maintain hope that it will improve. And coming from a small, very historic (and very prehistoric) town, I'm well aware of or acquainted with several local treasure hunters, and I know the different types and their motivations. Most are pretty good guys.
As NJ mentions below, archaeologists are primarily more accepting of treasure hunters that work alongside us to help uncover and preserve history. Any time a single person just pockets an artifact without recording it's exact location or depth in the ground, it loses some informational value. If they take it home and put it in a box with other artifacts from other sites, it loses all or almost all of its informational value. Archaeologists are constantly aware that when we excavate a site, we are destroying it. We can't put it back, and the next generation of archaeologists can't excavate it a second time. We also realize that we could be as careful and precise as currently possible, and we're still going to miss and destroy information. For instance, we use a 1/4" screen (generally) to catch artifacts. But I've worked with microartifacts (those smaller than 1/4"), so I've seen what we usually leave behind. It's amazing what a few thousand artifacts only 1 to 2 mm in size can tell you that the larger artifacts cannot.
A site becomes more significant if it (1) is the location of a significant event(s)/has the potential to generate high public interest, and/or (2) if it is an undisturbed, relatively rare example of a type of site from any point in our history. We might cringe if we hear of looters taking projectile points from plowed fields, but we fume to hear of detectorists digging through several inches of soil to get at a hit in the yard of a pioneer homestead, only to ignore the depth and specific location of whatever prize they've gotten (not to mention not reporting it to a historian or archaeologist), and discarding any glass, nails, or ceramics sherds they go through as unimportant. Of course, our disdain skyrockets when we learn they've done it just to make a profit, and we purely seethe when it's done illegally.
Yep... that was the condensed version.