Duplicate data after data import

Hi,

I have use data import on one of my form and it turns out that all data imported have been duplicated, is there any way to delete duplicate data. I tried the corresponding App but it request a jotform login and the it hangs without any login request.

The form with imported data is: http://form.jotformpro.com/form/51244921438959

Data was coming from form http://form.jotformpro.com/form/52494421787969

Data has been imported though CSV file, test was initially performed by importing data into http://form.jotformpro.com/form/52494421787969 for which we didn't experienced any problem.

Import was made on Sept. 8 at 5:10pm Paris time.

When you look at submissions and search for a name that was imported the result is incorrect.

Thanks for your help.

Georges

 is there any way to delete duplicate data

You can also delete the dupicates manually in the form View Submission page.

I understand it can be tedioius if there are a lot of data, and having to do it manually one by one.. You can use the Search option in the view submission page grid to search the specific submission.

OR

an alternative option is to use external application like Excel or other spreadsheet to do the deduplication process.

#1. Download the form's submission data to Excel or CSV

#2. Open the downloaded file in Excel or Spreadsheet

#3 Use the function of the Spreadsheet to auto filter and clear any duplicated data. Using the Excel will make the deduplication process easier since it can be done in 1-2 steps only.

#4 You can upload/import back the deduplicated data to the form using Import Data App again.

#5 Make sure the form previous existing submission data were removed in the form so that it will have a clean slate when you import the data back.

Hope this help. Let us know how we can be of further assistance.

Hi Jonathan,

I thought you could perform an SQL instruction on the database as I already planned the tedious way on 190 duplicated records. Further more the search function in jotform does not work properly on data of duplicated records.

Thanks anyhow but if you could perform the SQL delete I would be greatfull.

Regards,

Georges

You can get and delete your submissions via our API.  For example you can get all submissions using the GET call: http://api.jotform.com/docs/#submission-id

Once you get the submissions, you can always limit them to certain time frame or other conditions in your code and then remove them using the DELETE command: http://api.jotform.com/docs/#delete-submission-id

This would require a fair bit of setup on your end but it can be done.

Hi David,

Thanks for the tip, I have finally decided to do it the tedious way, added to my problem the record count for this form when I am in the page "my forms" is incorrect it was displaying 512 records when in excel I have 710 lines including header line.

Coudl you check the data integrity on your end ?

Thanks,

Georges

I ran our script that adjusts your submissions counter to the actual count and the number did increase by quite a bit.  The true count should now be shown.

Thanks for your help David.

I am done with the tedious work, actually I have to import another 92 records set, but I am nervous about it.

Regards,

Georges

Hi Georges,

Can you describe how you the import process? Perhas we can provide you with a suggestion to make things less tedious..

As I have mentioned previously, if you can open your raw data in Excel, you can do the deduplication process more simple. After that you can import the same Excel data file to the form submission data.

Hope this help. Please don't hesitate to let us know how we can be of further assistance.

Best

Hi Jonathan,

I imported the data using csv file in order to properly define the character set and get the accent properly in imported data. I ha no errors during the import process.

I think the import process is not very robust as I tried few times the import on a clone of the form without any duplicated data, the clone was empty. Then when I imported into the already populated original form I got these duplicates.

I may try you suggestion with full data reload coming from Excel but I'll do it on an empty form.

Best Regards,

Georges

Do let us know how it turns out.  I am not sure where the duplicates are coming from in the first place to be honest.  If they are not present in the sheet, they should not be imported to the form.

Hi David,

I eventually found the way to avoid the data duplication by playing the magic "cache clear", to be on the safe side I do it on both JotForm and Web browser.

By performing this preventive task I never encountered any more data duplication.

Thanks, for your help.

Regards,

Georges