When I delete a field from our survey, does it delete previously submitted data associated with that field?

For example, we have previous form submissions where clients have answered the question in a field that I want to delete. Does deleting this field alter the submissions that have previously been made in some way?

Yes, deleting a field from a form will delete the data associated with that field.  There is a warning pop-up that states this whenever a field with submission data is deleted.  If you would like to retain the data and remove the field, simply hide the field in the form: