Getting Issues using API when getting form submission

Good morning, I'm trying to run a script that I have within your API to bring submissions from one of my forms, as I use them within my system. But when I'm executing it at some point in the search it gives the following error:

at connResetException (node:internal/errors:691:14)

at TLSSocket.socketCloseListener (node:_http_client:402:19)

at TLSSocket.emit (node:events:532:35)

at TLSSocket.emit (node:domain:475:12)

at node:net:687:12

at TCP.done (node:_tls_wrap:580:7) {

code: 'ECONNRESET',

config: {

url: 'https://api.jotform.com/user/submissions?limit=1000&offset=114000&filter={"formIDs":["211504399072656"]}',

method: 'get',

headers: {

Accept: 'application/json, text/plain, */*',

APIKEY: '75ee86d75b0b4dbddf18c1a0a7',

'User-Agent': 'axios/0.21.4'

}...

I would like you to test whether the API for getting submissions has a problem. I await feedback as soon as possible to complete the API get successfully.

Hi laeciasps,

Thanks for reaching out to Jotform Support. Are you trying to get a list of all submissions for all forms on your account? If yes, can you please try to use this URL as you see below:

https://api.jotform.com/user/submissions?apiKey={apiKey}

Note that you'll need to add your API key to the field that is between the curly brace. You can visit this guide on How to Create Jotform API Keys for more detailed instruction. 

You may also want to visit this Jotform API documentation using /user/submissions for more detailed information, and click Show Optional Fields to view more options.

Give it a try and let us know if there’s anything else we can help you with.

I am using an id from a specific form to bring the questions from that form, because based on the id from that form I validate a visit. I have a system that schedules visits and I search for forms to synchronize this visit schedule with this form.

Formulário ID: 211504399072656

I use the limitation of every thousand forms brought he makes the sum so I can keep track of how much he brought. I use the limitation of every thousand forms brought he makes the sum so I can keep track of how much he brought. Today alone I have run the search within the API three times to bring up all the submissions made within the form: "Survey of the socio-family situation of the CMIC family" and it always results in the same error that I showed in my first response.

Hello Laeciasps,

Thanks for getting back to us. It seems like you're encountering a connection reset error (`ECONNRESET`) while trying to retrieve submissions from a specific form using the Jotform API. This error typically occurs when the connection to the server is unexpectedly closed while your script is trying to communicate with it. To troubleshoot:

• Retry Mechanism: Implement a retry mechanism in your script to retry the API request a few times with a delay between retries. This can help mitigate transient network or server issues.
• Check API Status: Check if there are any ongoing issues or maintenance activities on Jotform's API status page.
• Review API Usage: Ensure you are not hitting any rate limits or quota limits imposed by Jotform's API.
• Logging and Debugging: Add detailed logging to your script to capture more information about the request and response, which might provide insights into the issue.

Implementing these steps should help you identify and address the connection reset issue you are encountering with the Jotform API.

I'm still having the same problem, this isn't the first time, I had the same problem before, unable to update my submission base because I simply couldn't get it to the Jotform platform. I need it to be checked, because I haven't been able to get it since Tuesday and I desperately need this information.

Hi Laecia,

Thanks for getting back to us. Can you try creating a new API key and use that? Let me show you how to do that:

1. On your account's API settings page, click on the Create New Key button.
   
2. Next, enter the new API key label.
   
3. Next, set access accordingly and that's it.
   

If the same thing happens again, let us know together with the error log you received. We also have a guide on How to Create Jotform API Keys that you can check out.

Give it a try and let us know if you need any help.

My API key has full access. Why would I make a new read-only one?



I re-ran the script and it processed up to 127000 and aborted as shown in the image below.

Any news about my request? I need this problem to be resolved. Could someone read my request and pass it on to the developer team?

Hi Laeciasps,

Thanks for getting back to us. I'll reach out to the technical Team right now with the new information. Although we don’t have a solution yet, rest assured, we’ll keep working on this until it's resolved. We’ll get back to you as soon as we have an update.

Thanks for your patience and understanding. We appreciate it.

Hi laeciasps,

I see you are trying to retrieve submissions based on the form ID. I will suggest a faster and more accurate method for this. You can use the following endpoint:

curl -X GET "https://api.jotform.com/form/{formID}/submissions?apiKey={apiKey}"

Using this endpoint, you can retrieve submissions based on the form ID. This method is more accurate. Please try it and let us know if you encounter any issues.

Thanks for your patience and understanding. We appreciate it.

@Zahir Olmez or Jotform developer team.

Should I use this endpoint instead of: 'https://api.jotform.com/user/submissions?limit=1000&offset=114000&filter={"formIDs":["211504399072656"]}? Do I have a script for this in case I only have to change from "/user/submission..." to the endpoint you indicated? Can I do the limiting with limit and offset as is already done?



I await further clarification, thank you.

Hi laeciasps,

Thanks for getting back to us.

You can use features like limit and offset in the same way as the previous endpoint. You can use it like this: /form/{formID}/submissions?apiKey={{api_key}}&limit=1&offset=1. This endpoint is the most accurate way to retrieve submissions for a form.

Thanks for your patience and understanding. We appreciate it.

@Zahir Olmez or Jotform Developers Team

Good afternoon,



I changed it to the url you gave me and it started to carry out the procedure, but it still doesn't work. ECONRESET follows the excerpt below showing the resulting problem as well as information such as the url used and everything else.





Error: aborted

  at connResetException (node:internal/errors:720:14)

  at TLSSocket.socketCloseListener (node:_http_client:455:19)

  at TLSSocket.emit (node:events:526:35)

  at TLSSocket.emit (node:domain:489:12)

  at node:net:337:12

  at TCP.done (node:_tls_wrap:657:7) {

 code: 'ECONNRESET',

 config: {

    url: 'https://api.jotform.com/form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=1000&offset=37000',

  method: 'get',

  headers: {

   Accept: 'application/json, text/plain, */*',

   APIKEY: '75ee86d75b0b4dbddf18c1a0a7',

   'User-Agent': 'axios/0.21.4'

  },

Hi Laeciasps,

Thanks for getting back to us. I'll reach out to the technical Team right now with the new information. As soon as we have an update, we'll let you know.

Thanks for your patience and understanding. We appreciate it.

Hello Laeciasps,

Thank you for getting back to us. We suspect that your request might be timing out. To better understand this issue, could you please provide us with the full error message returned from your request?

Specifically, we are interested in:

• The complete text of the error message

If you can provide this information, we can investigate the issue more thoroughly and offer a quicker solution.

Thank you for your patience and understanding.

Best regards,

Hello, @Zahir Olmez or Jotform Developer Team

Below is a description of the error as requested, yesterday it was successful to bring all submissions, but today there was an error again when processing 24000 of the forms.

Error: aborted

  at connResetException (node:internal/errors:720:14)

  at TLSSocket.socketCloseListener (node:_http_client:455:19)

  at TLSSocket.emit (node:events:526:35)

  at TLSSocket.emit (node:domain:489:12)

  at node:net:337:12

  at TCP.done (node:_tls_wrap:657:7) {

 code: 'ECONNRESET',

 config: {

    url: 'https://api.jotform.com/form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=1000&offset=24000',

  method: 'get',

  headers: {

   Accept: 'application/json, text/plain, */*',

   APIKEY: '75ee86d75b0b4dbddf18c1a0a7',

   'User-Agent': 'axios/0.21.4'

  },

  transformRequest: [ [Function: transformRequest] ],

  transformResponse: [ [Function: transformResponse] ],

  timeout: 0,

  adapter: [Function: httpAdapter],

  xsrfCookieName: 'XSRF-TOKEN',

  xsrfHeaderName: 'X-XSRF-TOKEN',

  maxContentLength: -1,

  maxBodyLength: -1,

  validateStatus: [Function: validateStatus],

  transitional: {

   silentJSONParsing: true,

   forcedJSONParsing: true,

   clarifyTimeoutError: false

  },

  data: undefined

 },

 request: <ref *1> ClientRequest {

  _events: [Object: null prototype] {

   abort: [Function (anonymous)],

   aborted: [Function (anonymous)],

   connect: [Function (anonymous)],

   error: [Function (anonymous)],

   socket: [Function (anonymous)],

   timeout: [Function (anonymous)],

   finish: [Function: requestOnFinish]

  },

  _eventsCount: 7,

  _maxListeners: undefined,

  outputData: [],

  outputSize: 0,

  writable: true,

  destroyed: true,

  _last: true,

  chunkedEncoding: false,

  shouldKeepAlive: true,

  maxRequestsOnConnectionReached: false,

  _defaultKeepAlive: true,

  useChunkedEncodingByDefault: false,

  sendDate: false,

  _removedConnection: false,

  _removedContLen: false,

  _removedTE: false,

  strictContentLength: false,

  _contentLength: 0,

  _hasBody: true,

  _trailer: '',

  finished: true,

  _headerSent: true,

  _closed: true,

  socket: TLSSocket {

   _tlsOptions: [Object],

   _secureEstablished: true,

   _securePending: false,

   _newSessionPending: false,

   _controlReleased: true,

   secureConnecting: false,

   _SNICallback: null,

   servername: 'api.jotform.com',

   alpnProtocol: false,

   authorized: true,

   authorizationError: null,

   encrypted: true,

   _events: [Object: null prototype],

   _eventsCount: 11,

   connecting: false,

   _hadError: false,

   _parent: null,

   _host: 'api.jotform.com',

   _closeAfterHandlingError: false,

   _readableState: [ReadableState],

   _maxListeners: undefined,

   _writableState: [WritableState],

   allowHalfOpen: false,

   _sockname: null,

   _pendingData: null,

   _pendingEncoding: '',

   server: undefined,

   _server: null,

   ssl: null,

   _requestCert: true,

   _rejectUnauthorized: true,

   timeout: 5000,

   parser: null,

   _httpMessage: [Circular *1],

   autoSelectFamilyAttemptedAddresses: [Array],

   write: [Function: writeAfterFIN],

   [Symbol(alpncallback)]: null,

   [Symbol(res)]: null,

   [Symbol(verified)]: true,

   [Symbol(pendingSession)]: null,

   [Symbol(async_id_symbol)]: 174498,

   [Symbol(kHandle)]: null,

   [Symbol(lastWriteQueueSize)]: 0,

   [Symbol(timeout)]: [Timeout],

   [Symbol(kBuffer)]: null,

   [Symbol(kBufferCb)]: null,

   [Symbol(kBufferGen)]: null,

   [Symbol(kCapture)]: false,

   [Symbol(kSetNoDelay)]: false,

   [Symbol(kSetKeepAlive)]: true,

   [Symbol(kSetKeepAliveInitialDelay)]: 1,

   [Symbol(kBytesRead)]: 1143684070,

   [Symbol(kBytesWritten)]: 6487,

   [Symbol(connect-options)]: [Object]

  },

  _header: 'GET /form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=1000&offset=24000 HTTP/1.1\r\n' +

   'Accept: application/json, text/plain, */*\r\n' +

   'APIKEY: 75ee86d75b0b4dbddf18c1a0a7\r\n' +

   'User-Agent: axios/0.21.4\r\n' +

   'Host: api.jotform.com\r\n' +

   'Connection: keep-alive\r\n' +

   '\r\n',

  _keepAliveTimeout: 0,

  _onPendingData: [Function: nop],

  agent: Agent {

   _events: [Object: null prototype],

   _eventsCount: 2,

   _maxListeners: undefined,

   defaultPort: 443,

   protocol: 'https:',

   options: [Object: null prototype],

   requests: [Object: null prototype] {},

   sockets: [Object: null prototype] {},

   freeSockets: [Object: null prototype] {},

   keepAliveMsecs: 1000,

   keepAlive: true,

   maxSockets: Infinity,

   maxFreeSockets: 256,

   scheduling: 'lifo',

   maxTotalSockets: Infinity,

   totalSocketCount: 0,

   maxCachedSessions: 100,

   _sessionCache: [Object],

   [Symbol(kCapture)]: false

  },

  socketPath: undefined,

  method: 'GET',

  maxHeaderSize: undefined,

  insecureHTTPParser: undefined,

  joinDuplicateHeaders: undefined,

  path: '/form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=1000&offset=24000',

  _ended: false,

  res: IncomingMessage {

   _readableState: [ReadableState],

   _events: [Object: null prototype],

   _eventsCount: 3,

   _maxListeners: undefined,

   socket: [TLSSocket],

   httpVersionMajor: 1,

   httpVersionMinor: 1,

   httpVersion: '1.1',

   complete: false,

   rawHeaders: [Array],

   rawTrailers: [],

   joinDuplicateHeaders: undefined,

   aborted: true,

   upgrade: false,

   url: '',

   method: null,

   statusCode: 200,

   statusMessage: 'OK',

   client: [TLSSocket],

   _consuming: true,

   _dumped: false,

   req: [Circular *1],

      responseUrl: 'https://api.jotform.com/form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=1000&offset=24000',

   redirects: [],

   [Symbol(kCapture)]: false,

   [Symbol(kHeaders)]: [Object],

   [Symbol(kHeadersCount)]: 42,

   [Symbol(kTrailers)]: null,

   [Symbol(kTrailersCount)]: 0

  },

  aborted: false,

  timeoutCb: [Function: emitRequestTimeout],

  upgradeOrConnect: false,

  parser: null,

  maxHeadersCount: null,

  reusedSocket: true,

  host: 'api.jotform.com',

  protocol: 'https:',

  _redirectable: Writable {

   _writableState: [WritableState],

   _events: [Object: null prototype],

   _eventsCount: 2,

   _maxListeners: undefined,

   _options: [Object],

   _ended: true,

   _ending: true,

   _redirectCount: 0,

   _redirects: [],

   _requestBodyLength: 0,

   _requestBodyBuffers: [],

   _onNativeResponse: [Function (anonymous)],

   _currentRequest: [Circular *1],

      _currentUrl: 'https://api.jotform.com/form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=1000&offset=24000',

   [Symbol(kCapture)]: false

  },

  [Symbol(kCapture)]: false,

  [Symbol(kBytesWritten)]: 0,

  [Symbol(kNeedDrain)]: false,

  [Symbol(corked)]: 0,

  [Symbol(kOutHeaders)]: [Object: null prototype] {

   accept: [Array],

   apikey: [Array],

   'user-agent': [Array],

   host: [Array]

  },

  [Symbol(errored)]: null,

  [Symbol(kHighWaterMark)]: 16384,

  [Symbol(kRejectNonStandardBodyWrites)]: false,

  [Symbol(kUniqueHeaders)]: null

 },

 response: undefined,

 isAxiosError: true,

 toJSON: [Function: toJSON]

}

Hello Laeciasps,

Thank you for reaching out to us. We suspect that your request might be timing out due to the large amount of data being fetched (around 44 MB in your example). To address this issue while allowing you to continue using the current structure, we have a few suggestions:

1. Fetch Data in Smaller Chunks:

Try reducing the limit value to a smaller number, such as 500. This will decrease the amount of data fetched in each request and reduce the likelihood of a timeout. For example:

https://api.jotform.com/form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=500&offset=37000

2. Implement a Retry Mechanism:

Add a retry mechanism in your script to retry the API request after a delay if it fails. This can help mitigate transient network or server issues.

3. Enable Data Compression:

Use gzip compression to reduce the size of the data being transferred. You can add the Accept-Encoding: gzip header to your requests:

headers: {

 'Accept': 'application/json, text/plain, */*',

 'APIKEY': '75ee86d75b0b4dbddf18c1a0a7',

 'User-Agent': 'axios/0.21.4',

 'Accept-Encoding': 'gzip'

}

4. Increase Timeout Duration:

Increase the timeout duration in your API client (e.g., Axios) to allow more time for the request to complete:

const axios = require('axios');



axios.get('https://api.jotform.com/form/211504399072656/submissions?apiKey=75ee86d75b0b4dbddf18c1a0a7&limit=1000&offset=37000', {

 timeout: 60000 // Set timeout to 60 seconds

})

.then(response => {

 console.log(response.data);

})

.catch(error => {

 console.error('Error:', error);

});

Please try these suggestions and let us know if you need further assistance.

Thank you for your patience and understanding.

Best regards,

I did three of the four points, I'm still going to test it by adding point 2, which is the return mechanism, but until then the process was going well, but it reached a certain point where it seemed like it took too long to progress, then it didn't give an error, but simply ended the process. See photo below.

@Zahir Olmez or Jotform Developer Team,

I forgot mark you, but response above.

Hello, somebody can help me, again. I sended yesterday my questions and didn't receive a response.

Hello Laeciasps,

I understand that you are experiencing delays or connection issues while fetching large data sets from the Jotform API. These issues can sometimes be caused by server load, leading to response delays. Implementing a retry mechanism can help mitigate these issues by retrying the API request after a failure.

What is a Retry Mechanism?

A retry mechanism attempts to resend an API request after it fails, waiting a certain period between retries. This approach helps to overcome temporary network issues or server load problems.

How to Implement It?

Below is an example of how to implement a retry mechanism using Node.js and Axios:

const axios = require('axios');

const axiosRetry = require('axios-retry');



// Enable retry mechanism for Axios

axiosRetry(axios, {

 retries: 5, // Number of retry attempts

 retryDelay: (retryCount) => {

  console.log(`Retrying request, attempt number: ${retryCount}`);

  return retryCount * 2000; // Wait 2 seconds between retries

 },

 retryCondition: (error) => {

  return error.code === 'ECONNRESET' || error.response.status === 503;

 },

});



const fetchData = async () => {

 try {

    const response = await axios.get('https://api.jotform.com/form/211504399072656/submissions', {

   params: {

    apiKey: '75ee86d75b0b4dbddf18c1a0a7',

    limit: 500,

    offset: 24000

   },

   timeout: 60000, // Set timeout to 60 seconds

   headers: {

    'Accept-Encoding': 'gzip'

   }

  });

  console.log(response.data);

 } catch (error) {

  console.error('Error fetching data:', error);

 }

};



fetchData();

Explanation:

1. axios-retry: This library works with Axios to automatically retry requests after a failure.

2. retries: Sets the maximum number of retry attempts.

3. retryDelay: Specifies the delay between retries in milliseconds. In this example, it waits 2 seconds between each retry.

4. retryCondition: Determines the conditions under which a retry is attempted, such as ECONNRESET errors or 503 (Service Unavailable) status codes.

By using this mechanism, you can handle temporary network or server issues more gracefully, ensuring your data retrieval process is more robust.

If you have any further questions or need additional assistance, please don’t hesitate to reach out.

Thank you,

@Zahir Olmez or Jotform Developer Team,

Do we have any news regarding the last question I asked you?? 

Hello Laeciasps,

I understand that implementing the retry mechanism has been beneficial to some extent, but you are still facing issues with the process stalling after a certain point.

Regarding your last question, if you have any specific issues or further questions, please feel free to share them, and I will be happy to assist you. If the retry mechanism is not fully resolving the issue, there might be other factors at play that we can investigate together.

Thank you for your patience and understanding.

Best regards,