Yes, you need the plug to keep out dirt and water. Seems like you can put some plastic and a rubber band over it to protect it temporarily, but not for underwater. Otherwise keep the headphones connected.
Purchase a replacement from Garret asap. There are tapered plastic plugs that you should be able to find at hardware stores, all you need is one small enough to go in and large enough to stay in - these are tapered plastic plugs. Or putty inside a plastic bag, bubble gum.....let your mind run wild But do keep water and dirt out of the hole. Good Luck..........63bkpkr