Batch output

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Batch output

Gonzalo Vasquez
One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.

Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.

Several questions arise:

  1. On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
  2. In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?

Any other ideas for such batch processing?

Any comments will be of great help!


Regards,




Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  

Reply | Threaded
Open this post in threaded view
|

Re: Batch output

Fatih Piristine
hi Gonzalo

I had similar problems earlier but couldn't find a way around rather than generating each in separate context. for 2nd part, implementing processor to generate these docs from queue might work.




On Thu, Nov 21, 2013 at 11:11 PM, Gonzalo Vasquez <[hidden email]> wrote:
One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.

Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.

Several questions arise:

  1. On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
  2. In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?

Any other ideas for such batch processing?

Any comments will be of great help!


Regards,




Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  




--

Fatih

Reply | Threaded
Open this post in threaded view
|

Re: Batch output

Gonzalo Vasquez
Thanks Fatih for such a fast response!! Can you provide further details and/or references to the processor you mention?

Regards,
Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  

El 21-11-2013, a las 18:47, Fatih P. <[hidden email]> escribió:

hi Gonzalo

I had similar problems earlier but couldn't find a way around rather than generating each in separate context. for 2nd part, implementing processor to generate these docs from queue might work.




On Thu, Nov 21, 2013 at 11:11 PM, Gonzalo Vasquez <[hidden email]> wrote:
One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.

Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.

Several questions arise:

  1. On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
  2. In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?

Any other ideas for such batch processing?

Any comments will be of great help!


Regards,




Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  




--
Fatih

Reply | Threaded
Open this post in threaded view
|

Re: Batch output

Fatih Piristine
I already mentioned there, queue might solve your problem to process your data with template-s.


On Thu, Nov 21, 2013 at 11:49 PM, Gonzalo Vasquez <[hidden email]> wrote:
Thanks Fatih for such a fast response!! Can you provide further details and/or references to the processor you mention?

Regards,
Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  

El 21-11-2013, a las 18:47, Fatih P. <[hidden email]> escribió:

hi Gonzalo

I had similar problems earlier but couldn't find a way around rather than generating each in separate context. for 2nd part, implementing processor to generate these docs from queue might work.




On Thu, Nov 21, 2013 at 11:11 PM, Gonzalo Vasquez <[hidden email]> wrote:
One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.

Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.

Several questions arise:

  1. On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
  2. In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?

Any other ideas for such batch processing?

Any comments will be of great help!


Regards,




Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  




--
Fatih




--

Fatih Piristine

address: Riekontie 237, 43250 Kolkanlahti, Finland
mobile: +358 (46) 538 2774

mail: [hidden email]

skype: v-fpiris

linkedIn: fi.linkedin.com/in/fatihpiristine

Reply | Threaded
Open this post in threaded view
|

Re: Batch output

rsargent
In reply to this post by Gonzalo Vasquez
I had good results stitching them together with iText.

rjs

On 11/21/2013 02:11 PM, Gonzalo Vasquez wrote:
One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.

Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.

Several questions arise:

  1. On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
  2. In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?

Any other ideas for such batch processing?

Any comments will be of great help!


Regards,




Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  


Reply | Threaded
Open this post in threaded view
|

Re: Batch output

Alexios Giotis
For the first case, each document should be rendered separately but you may reuse the FopFactory to speed it up.

For the 2nd case, I assume that the output file of the printing process has thousand of pages and if so, my recommendation is to generate FOP intermediate format for each document (a FOP specific XML format) and then concatenate the FOP intermediate format files to final PDF,AFP or PS output file(s). Although you mentioned a single output file, if you exceed 50 thousand pages, you might need to split it. The number depends on the complexity of the documents. By splitting, the memory requirements are reduced and the concatenation can also be executed in parallel. A small example for generating the intermediate format and then concatenating exists at [1].

HTH,
Alex Giotis


[1] http://svn.apache.org/viewvc/xmlgraphics/fop/trunk/examples/embedding/java/embedding/intermediate/ExampleConcat.java?view=markup


>
> On 11/21/2013 02:11 PM, Gonzalo Vasquez wrote:
>> One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.
>>
>> Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.
>>
>> Several questions arise:
>>
>> • On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
>> • In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?
>>
>> Any other ideas for such batch processing?
>>
>> Any comments will be of great help!
>>
>>
>> Regards,
>>
>>
>>
>>
>> Gonzalo Vásquez Sáez
>> Gerente Investigación y Desarrollo (R&D)
>> Altiuz Soluciones Tecnológicas de Negocios Ltda.
>> Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
>> +56 2 335 2461
>> [hidden email]
>> http://www.altiuz.cl
>> http://www.altiuzreports.com
>>  
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Batch output

Gonzalo Vasquez
Sounds good the idea of concatenating FO files, but how can that actually be achieved if using several page masters?

Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
 


El 22-11-2013, a las 6:02, Alexios Giotis <[hidden email]> escribió:

> For the first case, each document should be rendered separately but you may reuse the FopFactory to speed it up.
>
> For the 2nd case, I assume that the output file of the printing process has thousand of pages and if so, my recommendation is to generate FOP intermediate format for each document (a FOP specific XML format) and then concatenate the FOP intermediate format files to final PDF,AFP or PS output file(s). Although you mentioned a single output file, if you exceed 50 thousand pages, you might need to split it. The number depends on the complexity of the documents. By splitting, the memory requirements are reduced and the concatenation can also be executed in parallel. A small example for generating the intermediate format and then concatenating exists at [1].
>
> HTH,
> Alex Giotis
>
>
> [1] http://svn.apache.org/viewvc/xmlgraphics/fop/trunk/examples/embedding/java/embedding/intermediate/ExampleConcat.java?view=markup
>
>
>>
>> On 11/21/2013 02:11 PM, Gonzalo Vasquez wrote:
>>> One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.
>>>
>>> Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.
>>>
>>> Several questions arise:
>>>
>>> • On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
>>> • In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?
>>>
>>> Any other ideas for such batch processing?
>>>
>>> Any comments will be of great help!
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>>
>>> Gonzalo Vásquez Sáez
>>> Gerente Investigación y Desarrollo (R&D)
>>> Altiuz Soluciones Tecnológicas de Negocios Ltda.
>>> Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
>>> +56 2 335 2461
>>> [hidden email]
>>> http://www.altiuz.cl
>>> http://www.altiuzreports.com
>>>  
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Batch output

Gonzalo Vasquez
In reply to this post by rsargent
That's what we are doing right now...generating each pdf on it's own, and the "concatenating" them with iText, but I think the overhead might be reduced somehow.
Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  

El 21-11-2013, a las 19:38, Rob Sargent <[hidden email]> escribió:

I had good results stitching them together with iText.

rjs

On 11/21/2013 02:11 PM, Gonzalo Vasquez wrote:
One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.

Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.

Several questions arise:

  1. On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
  2. In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?

Any other ideas for such batch processing?

Any comments will be of great help!


Regards,




Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
+56 2 335 2461
[hidden email]
http://www.altiuz.cl
http://www.altiuzreports.com
  



Reply | Threaded
Open this post in threaded view
|

Re: Batch output

Alexios Giotis
In reply to this post by Gonzalo Vasquez
Hi Gonzalo,

The 2nd point is about concatenating files in FOP intermediate format, not XSL:FO (see [1]). Each document can have its own page masters, since this is defined in the XSL:FO.

[1] http://xmlgraphics.apache.org/fop/1.1/intermediate.html

Alex Giotis


On 22 Nov 2013, at 16:33, Gonzalo Vasquez <[hidden email]> wrote:

> Sounds good the idea of concatenating FO files, but how can that actually be achieved if using several page masters?
>
> Gonzalo Vásquez Sáez
> Gerente Investigación y Desarrollo (R&D)
> Altiuz Soluciones Tecnológicas de Negocios Ltda.
> Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
> +56 2 335 2461
> [hidden email]
> http://www.altiuz.cl
> http://www.altiuzreports.com
>
>
>
> El 22-11-2013, a las 6:02, Alexios Giotis <[hidden email]> escribió:
>
>> For the first case, each document should be rendered separately but you may reuse the FopFactory to speed it up.
>>
>> For the 2nd case, I assume that the output file of the printing process has thousand of pages and if so, my recommendation is to generate FOP intermediate format for each document (a FOP specific XML format) and then concatenate the FOP intermediate format files to final PDF,AFP or PS output file(s). Although you mentioned a single output file, if you exceed 50 thousand pages, you might need to split it. The number depends on the complexity of the documents. By splitting, the memory requirements are reduced and the concatenation can also be executed in parallel. A small example for generating the intermediate format and then concatenating exists at [1].
>>
>> HTH,
>> Alex Giotis
>>
>>
>> [1] http://svn.apache.org/viewvc/xmlgraphics/fop/trunk/examples/embedding/java/embedding/intermediate/ExampleConcat.java?view=markup
>>
>>
>>>
>>> On 11/21/2013 02:11 PM, Gonzalo Vasquez wrote:
>>>> One of our processes has to deal with thousands of documents in a batch process, either for email sending, or for printing.
>>>>
>>>> Depending on our customers' needs, we choose from PDF, AFP and PS as output formats. Now we are trying to move everything to FO generated documents, so we can you a single template for all output formats.
>>>>
>>>> Several questions arise:
>>>>
>>>> • On the email process (many files as output), do I have to deal with every document in a separate context, or is there any fo-related trick to achieve this in a better way?
>>>> • In the printing process (single file output), is there anyway to feed the template just once for all documents and the "add" the data to get a huge document with all the subdocuments in it?
>>>>
>>>> Any other ideas for such batch processing?
>>>>
>>>> Any comments will be of great help!
>>>>
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>>>
>>>> Gonzalo Vásquez Sáez
>>>> Gerente Investigación y Desarrollo (R&D)
>>>> Altiuz Soluciones Tecnológicas de Negocios Ltda.
>>>> Av. Nueva Tajamar 555 Of. 802, Las Condes - CP 7550099
>>>> +56 2 335 2461
>>>> [hidden email]
>>>> http://www.altiuz.cl
>>>> http://www.altiuzreports.com
>>>>
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]