How can I recieve notifications that I have received a submission?

Also, if a person chooses to pay via cash or check, and not via paypal (or any other online payment system) will JotForm consider it a submission?  Will I still be able to see the registration form?

You should receive email notifications no matter if yo have integrated your form with a payment integration or not, the difference with PayPal is that the payment may be cataloged as incomplete, this happens due to different factors that could be a bit hard to get control; however, you're still able to see submissions, if it has been cataloged as incomplete you can follow this guide in order to view those submissions: How-to-view-incomplete-payments.

But you can get notified each time there is an incomplete payment in your form, if your submitter has been redirected to PayPal, I would suggest you to enable the Incomplete Payment Notification in your form so you get notified about incomplete payments: How-to-Setup-Incomplete-Payment-Notification-E-Mail

Also, we do recommend to enable the Instant Payment Notification in your PayPal account so you also get notified when there's a payment: How-to-Enable-IPN-for-Paypal

Sometimes, emails are not sent because the email address is in our bounce list, this is not the case with your email address, but I can see there is some missing info, I would suggest you to properly set up your email notifications so you avoid emails not being received because of any security filter, here is also a guide that will help you to properly set up your email notifications: How-to-Prevent-Email-Bouncing-Related-Issues

Hope this helps.