- sudsportAsked on September 08, 2015 at 05:03 PM
I have use data import on one of my form and it turns out that all data imported have been duplicated, is there any way to delete duplicate data. I tried the corresponding App but it request a jotform login and the it hangs without any login request.
The form with imported data is: http://form.jotformpro.com/form/51244921438959
Data was coming from form http://form.jotformpro.com/form/52494421787969
Data has been imported though CSV file, test was initially performed by importing data into http://form.jotformpro.com/form/52494421787969 for which we didn't experienced any problem.
Import was made on Sept. 8 at 5:10pm Paris time.
When you look at submissions and search for a name that was imported the result is incorrect.
Thanks for your help.
- JotForm SupportjonathanAnswered on September 08, 2015 at 08:45 PM
is there any way to delete duplicate data
You can also delete the dupicates manually in the form View Submission page.
I understand it can be tedioius if there are a lot of data, and having to do it manually one by one.. You can use the Search option in the view submission page grid to search the specific submission.
an alternative option is to use external application like Excel or other spreadsheet to do the deduplication process.
#1. Download the form's submission data to Excel or CSV
#2. Open the downloaded file in Excel or Spreadsheet
#3 Use the function of the Spreadsheet to auto filter and clear any duplicated data. Using the Excel will make the deduplication process easier since it can be done in 1-2 steps only.
#4 You can upload/import back the deduplicated data to the form using Import Data App again.
#5 Make sure the form previous existing submission data were removed in the form so that it will have a clean slate when you import the data back.
Hope this help. Let us know how we can be of further assistance.
- sudsportAnswered on September 09, 2015 at 04:03 AM
I thought you could perform an SQL instruction on the database as I already planned the tedious way on 190 duplicated records. Further more the search function in jotform does not work properly on data of duplicated records.
Thanks anyhow but if you could perform the SQL delete I would be greatfull.
- JotForm SupportdavidAnswered on September 09, 2015 at 11:43 AM
You can get and delete your submissions via our API. For example you can get all submissions using the GET call: http://api.jotform.com/docs/#submission-id
Once you get the submissions, you can always limit them to certain time frame or other conditions in your code and then remove them using the DELETE command: http://api.jotform.com/docs/#delete-submission-id
This would require a fair bit of setup on your end but it can be done.
- sudsportAnswered on September 09, 2015 at 04:23 PM
Thanks for the tip, I have finally decided to do it the tedious way, added to my problem the record count for this form when I am in the page "my forms" is incorrect it was displaying 512 records when in excel I have 710 lines including header line.
Coudl you check the data integrity on your end ?
- JotForm SupportdavidAnswered on September 09, 2015 at 06:06 PM
I ran our script that adjusts your submissions counter to the actual count and the number did increase by quite a bit. The true count should now be shown.
- sudsportAnswered on September 09, 2015 at 06:40 PM
Thanks for your help David.
I am done with the tedious work, actually I have to import another 92 records set, but I am nervous about it.
- JotForm SupportjonathanAnswered on September 09, 2015 at 10:25 PM
Can you describe how you the import process? Perhas we can provide you with a suggestion to make things less tedious..
As I have mentioned previously, if you can open your raw data in Excel, you can do the deduplication process more simple. After that you can import the same Excel data file to the form submission data.
Hope this help. Please don't hesitate to let us know how we can be of further assistance.
- sudsportAnswered on September 10, 2015 at 02:20 PM
I imported the data using csv file in order to properly define the character set and get the accent properly in imported data. I ha no errors during the import process.
I think the import process is not very robust as I tried few times the import on a clone of the form without any duplicated data, the clone was empty. Then when I imported into the already populated original form I got these duplicates.
I may try you suggestion with full data reload coming from Excel but I'll do it on an empty form.
- JotForm SupportdavidAnswered on September 10, 2015 at 03:31 PM
Do let us know how it turns out. I am not sure where the duplicates are coming from in the first place to be honest. If they are not present in the sheet, they should not be imported to the form.
- sudsportAnswered on September 25, 2015 at 05:24 AM
I eventually found the way to avoid the data duplication by playing the magic "cache clear", to be on the safe side I do it on both JotForm and Web browser.
By performing this preventive task I never encountered any more data duplication.
Thanks, for your help.