Can the responder delete their submission

No, they cannot. But you can provide respondents with submission edit links, so they will be able to edit their submitted entries.

Also, you as the account owner can delete the submissions:

How to Delete Form Submission Data