That's an old fashioned shirt neck button. They were quite popular in the late 1800s and early 1900s. I have my great-grandfather's. At least, that's what it looks like to me. It allowed a little flexibility in the collar spread. It was used to close the top "button" area of the shirt, around the collar.
It if was a cuff link, it would have a lever on one side to open up to prevent its slipping out of the cuff slit.
By the way, that's a great find! Keep it! -Noodle