My GenerateEmails form is working. I can successfully send an email with a report attached.
When I accidentally tried to send a somewhat large report, it thought for a while then came back with a server timeout message.
Is this something that can be fixed, and how long of a process will cause this message?
Thanks…
Hi Ron,
Thank you for bringing this scenario to our attention.
Could you please upload an updated version of your application to the shared folder and send me an email at elton@five.co with the steps so I can try to reproduce your scenario?
Thank you
Elton S
Hi Ron,
After investigating this scenario, I have identified two possible issues in your report design:
1 - Timeout when selecting multiple study groups:
The timeout occurs when multiple study groups are selected. This happens because Five has an internal limit of 3 minutes to generate a report. If the report is not completed within this timeframe, the process fails due to a timeout.
2 - Email not sent due to attachment size limitations:
Depending on the number of study groups selected, the email may not be sent because email providers (such as Gmail) have limits on attachment size.
For example:
When generating the report with the first 10 study groups selected, the PDF size is approximately 8 MB.
When selecting the first 20 study groups, the file size increases to around 15 MB.
An improvement can be implemented by adding a function to the “Study Group Roster” report to the event “On Run”. With this function, you can pass a configuration object to the report to control settings such as image quality and format.
For example, when selecting the first 10 study groups, the PDF size can be reduced to approximately 2 MB, compared to the previous 8 MB.
Below is an example of this function using an options object:
function SetPDFQuality(five, context, result) {
result.update('ErrErrorOk', '', '');
result.setOptions({
ImageQuality: 0.60,
ImageType: 'jpeg',
});
return result;
}
Regards,
Elton S
Thanks for this answer, Elton.
This is a major disappointment, that there is a hard timeout after 3 minutes. I will try your code in the OnRun event. While that may help, I don’t know if it will totally solve the issue.
Can you please point me to documentation regarding this function you are suggesting? It’s not clear what the 0.60 means. I guess the lower the quality number the smaller the file?
Also, I’m confused by the ImageType: ‘jpeg’ item. Why is a PDF file having anything to do with a jpeg image?
Finally, is there ANY way that 3-minute limit can be overridden? This seems like a big show-stopper for anyone who has large reports to generate in the background.
Thanks…
Hi Ron,
I have notified the team regarding increasing the timeout. I will keep you informed of any updates.
Meanwhile, and I believe you may need it, you can generate a PDF for each studyGroup, then merge them into a single PDF when you send the email.
This is an example of how to achieve it, which I am regenerating 4 reports, combining and merging them into 3 PDF files:
function DoAddAndMergeAttachments(five, context, result) {
const reportResult0 = five.executeAction('StaffExtension', {});
if (reportResult0.isOk() === false) {
return five.createError(reportResult0);
}
const reportResult1 = five.executeAction('WebProductsReport', {});
if (reportResult1.isOk() === false) {
return five.createError(reportResult1);
}
const reportResult2 = five.executeAction('PositionReport', {});
if (reportResult2.isOk() === false) {
return five.createError(reportResult2);
}
const reportResult3 = five.executeAction('StaffReport', {});
if (reportResult3.isOk() === false) {
return five.createError(reportResult3);
}
const pdfs1 = \[\];
const pdfs2 = \[\];
// Array pdfs1 with reports (StaffExtension and WebProductsReport)
if (reportResult0.report) {
pdfs1.push(reportResult0.report);
}
if (reportResult1.report) {
pdfs1.push(reportResult1.report);
}
// Array pdfs2 with reports (PositionReport, and StaffReport)
if (reportResult2.report) {
pdfs2.push(reportResult2.report);
}
if (reportResult3.report) {
pdfs2.push(reportResult3.report);
}
// Array pdfs3 with reports (StaffExtension, WebProductsReport, PositionReport, and StaffReport)
const pdfs3 = \[\];
pdfs3.push(...pdfs1, ...pdfs2);
//Store the PDF utility method.
const pdfUtils = five.pdfUtils();
//Merge reports (StaffExtension and WebProductsReport) into PDF pdf1
const mergedPDF1 = pdfUtils.merge(pdfs1);
if (mergedPDF1.isOk && mergedPDF1.isOk() === false) {
return five.createError(mergedPDF1, 'Failed to merge pdf1 files for attaching');
}
let attachResult1 = five.addAttachment(mergedPDF1, "pdf1.pdf");
if (attachResult1.isOk() == false) {
return five.createError(attachResult1);
}
//Merge reports (PositionReport, and StaffReport) into PDF pdf2
const mergedPDF2 = pdfUtils.merge(pdfs2);
if (mergedPDF2.isOk && mergedPDF2.isOk() === false) {
return five.createError(mergedPDF2, 'Failed to merge pdf2 files for attaching');
}
const attachResult2 = five.addAttachment(mergedPDF2, "pdf2.pdf");
if (attachResult2.isOk() == false) {
return five.createError(attachResult2);
}
//Merge all reports (StaffExtension, WebProductsReport, PositionReport, and StaffReport) into PDF pdf3
const mergedPDF3 = pdfUtils.merge(pdfs3);
if (mergedPDF3.isOk && mergedPDF3.isOk() === false) {
return five.createError(mergedPDF3, 'Failed to merge pdf3 files for attaching');
}
const attachResult3 = five.addAttachment(mergedPDF3, "pdf3.pdf");
if (attachResult3.isOk() == false) {
return five.createError(attachResult3);
}
return five.success(result);
}
Below is the definition for the SetPDFQuality function (we are in the process of updating our documentation). The properties definition is:
ImageQuality: Set the quality of the pdf content. Example: default is 1, meaning 100%, 0.6 means 60%.
ImageType: The report generates an image for each page during PDF creation, which is then compiled and converted into a final PDF.
The supported types are ‘png’ and ‘jpeg’
Regards,
Elton S
Thanks for the suggestions, Elton.
You may be right about needing to generate in smaller batches (one roster at a time), then put them together. That would also have an advantage of each roster having “page 1 of 2”, “page 2 of 2” instead of “page 1 of 40” which happens when I generate a single report for many groups.
It’s unclear if this is a huge advantage or not. The main thing is avoiding the timeout. where does the timeout occur? is it when the executeAction(‘reportName’, {}) is happening? If so, that would explain the timeout.
It looks like this is a server function, only because the name starts with Do. But it’s not the server function itself that is timing out, but the executeAction, right?
Hi Ron,
Indeed, my example was executed on the Do event (Server-side).
The timeout is set on the report internally (by default), and not in the executeAction function, as an action can also be Process, Mailmerge, etc.
Regards,
Elton S
I’m curious about your example of generating reports individually then combining them. your example function begins with the word “Do”, so I’m inferring that this is a server function. Please correct me if I’m wrong.
With on-demand reports, the server function is called to do the staging if needed, then either way, the report is actually generated inside the callback function. On demand report does not necessarily time out if I’m generating the full (largest) report. But this doesn’t guarantee that it won’t time out in the future as data grows.
Definitely this is an issue with generating emails with reports attached. In that case, the actual generating is controlled in a server function, and the executeAction is also called in the server function instead of from the client.
So now I’m wondering if I should create a new server function dedicated to generating a single report, then I can either call it from inside the callback function (if doing on-demand report). The problem is that we can’t call a server function from another server function. So I could make the final function to generate the single report into a library, that could be called from either client or server, called GenerateSingleReport.
This library could decide whether to iterate groups and generate a single report at a time then combine, or generate a full report. This decision can be based on a variable set from my ReportSettings table.
To do this, I would either need to create 2 different queries (one joining to SelectedGroups table, which is done currently, and one exactly the same but without using the join to SelectedGroups), then decide which one to use based on some metadata. But I believe that would require me to have 2 different report definitions, since the parameters are hard-coded in the report definition.
Alternatively, do you think I can somehow make the join in the query optional, based on a variable or something? That way I still only need a single report/query. But how, in five, would I do that?
Thanks for taking the time to read this post. I feel like we are getting closer…
UPDATE:
I figured out that I don’t need to change the query at all. When calling the report from client-side, just include all records based on the join with the SelectedGroups table.
When running the reports to attach to the emails, this is done server-side. There, I can control which of the originally selected records in the SelectedGroups table are selected. I’ve already written code that will snapshot the groups where IsSelected = true, then reset so only the desired group is selected, then run the report with the original query, then reset the SelectedGroups to where it was from the snapshot when I start the process. I’ve already tested that code, and it should work.
This lets me call the report from either client-side or server-side.
I need to verify the following:
Thanks…
Hi Ron,
Thank you for the update and for teh extra questions, here are the answers:
1 - The timeout is always on the server-side, which is where the reports are executed.
2 - I cannot guarantee it, as it depends on how long it takes to generate the report. That’s why I mentioned that splitting the report would help you, so that each report execution has a timeout to finish (in the context of a group of many studies).
3 - This is still under investigation. The team works across different demands, including improvements, new features, and issues. I will keep you posted about the issue.
4 - In my example, I used the function that attached the reports to an email. It means that when I call the email, I do not pass the attachment property to it (as everything is generated inside).
Yes, this function is attached to the event “Do Merge Record“ server-side,
Observation: this was only one example; you could also generate all reports in the main server-side function and pass the array with many attachments, then the function that attaches the email can merge the reports.
Hope this information can help you.
Regards,
Elton S
Thanks Elton. This does help somewhat, but I need some clarification.
I’m not sure I stated my questions properly.
1:
The documentation doesn’t address this issue. I’ve been able in the past to generate the report from the client-side, inside of a callback function after doing some prep work by calling a server function. I’ve also been able to generate reports directly from a server function (before that stopped working). I wanted to know if the timeout issue you talked about may be there regardless of whether the executeAction to generate the report comes from client-side or server-side.
2:
The report I’m referring to is the GroupRoster. Since no groups have more than 20-25 members or so, this is NOT a large report. However, doing one for all groups does get large. Is the report generation via executeAction subject to the same possible timeouts even if called from client-side?
4:
I am working toward a similar technique as your observation, where I generate reports on the server-side, and if they are too large (like the GroupRoster), I can generate them in smaller chunks and combine. If not too large, just generate them normally. This question is mainly around whether I should change my technique and do this also when doing on-demand reports (which don’t involve emailing or attaching).
I think I should always do the reports server-side, because of the timeout issue, but then I don’t know how to render them client-side if they are on-demand reports. I only know how to generate them client-side (in the callback function). In fact, I suspect that on-demand reports to be displayed in the browser must be called client-side. True?