Can I create a column on the form that is only visible to me when a form is submitted but not to the person submitting the application?

Yes, any column like the configurable list widget or the matrix field. You can then just hide it in the form through its properties:

When the form is submitted, you can see this hidden field using the edit link:

You can also edit a submission in the form submissions page: https://www.jotform.com/help/269-How-to-view-Submissions.