Click on the object you want to keep. Similar colors get selected — click more spots to expand the selection, then remove the rest.
Click the object to select it. Click again elsewhere to add more to the selection.