[jira] [Commented] (FOP-2211) [PATCH] Fix & improve the handling of temporary files using the new URI resource resolvers

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (FOP-2211) [PATCH] Fix & improve the handling of temporary files using the new URI resource resolvers

Andreas Joseph Krogh (Jira)

    [ https://issues.apache.org/jira/browse/FOP-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13593313#comment-13593313 ]

Alexios Giotis commented on FOP-2211:
-------------------------------------

Hi Simon,

Thank you for the patch. I looked at it and it does not resolve the issue of keeping on disk many and big files. In a test we did last week, the temp file was 100GB and for sure we don't wish to keep such files until the JVM is normally terminated. For us, this is several months or a year.

Also, I don't think that backwards compatibility is an issue here. This is trunk, there are many changes since 1.1 that do affect users embedding FOP in their apps and this is not one of them. I am sure it will be easy to change your code or to create an adapter.

I will try to find some time to submit updated versions of the patches that take into account the comments above.

               

> [PATCH] Fix & improve the handling of temporary files using the new URI resource resolvers
> ------------------------------------------------------------------------------------------
>
>                 Key: FOP-2211
>                 URL: https://issues.apache.org/jira/browse/FOP-2211
>             Project: Fop
>          Issue Type: Bug
>          Components: general
>    Affects Versions: trunk
>            Reporter: Alexios Giotis
>             Fix For: trunk
>
>         Attachments: fop.patch, tempurisimple.patch, xgc.patch
>
>
> As written in http://markmail.org/message/zelumstxxsdyvkcz , after the merge of the Temp_URI_Resolution branch (Sept 2012), the actual pattern of using temp files has changed from:
> {code}
> File tmpFile = File.createTempFile(....);
> // Write and read from the file
> tmpFile.delete();
> {code}
> to:
> {code}
> File tmpFile = new File(System.getProperty("java.io.tmpdir"), counterStartingFrom1AsString);
> tmpFile.deleteOnExit();
> // Write and read from the file
> {code}
> This is fine when FOP is executed from the command line (which I guess this is how most people use it) but it introduces a number of bad side effects for long running processes that use FOP embedded.
>  
> 1. Different FOP processes can't be executed in parallel on the same system because creating the same temp file fails.
> 2. If the JVM is not normally terminated, the temp files are never deleted and the next invocation of the JVM fails to run.
> 3. deleteOnExit() keeps for the life of the JVM an unknown number of temp files both on disk and a reference in memory.
> There should not be a need to implement a custom resource resolver when using FOP embedded in order to fix those issues. The default implementation should work at least as good as it worked in FOP 1.1 or earlier.
> Attached are 2 patches, one for XGC and one for FOP that should fix and improve the handling of at least the temporary files.
> For reference, [1] lists some reasons for implementing the new URI resource resolvers.
> [1] http://wiki.apache.org/xmlgraphics-fop/URIResolution

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Commented] (FOP-2211) [PATCH] Fix & improve the handling of temporary files using the new URI resource resolvers

Peter Hancock-2
Hi Alexios,

I take it you are you reading from the temp files more than once,
otherwise you could reluctantly hook into the close method of Resource
to do the file deletion.  I am interested to know how you are
benefiting from reusing the temp file.

Thanks,

Peter

On Tue, Mar 5, 2013 at 10:55 AM, Alexios Giotis (JIRA) <[hidden email]> wrote:

>
>     [ https://issues.apache.org/jira/browse/FOP-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13593313#comment-13593313 ]
>
> Alexios Giotis commented on FOP-2211:
> -------------------------------------
>
> Hi Simon,
>
> Thank you for the patch. I looked at it and it does not resolve the issue of keeping on disk many and big files. In a test we did last week, the temp file was 100GB and for sure we don't wish to keep such files until the JVM is normally terminated. For us, this is several months or a year.
>
> Also, I don't think that backwards compatibility is an issue here. This is trunk, there are many changes since 1.1 that do affect users embedding FOP in their apps and this is not one of them. I am sure it will be easy to change your code or to create an adapter.
>
> I will try to find some time to submit updated versions of the patches that take into account the comments above.
>
>
>> [PATCH] Fix & improve the handling of temporary files using the new URI resource resolvers
>> ------------------------------------------------------------------------------------------
>>
>>                 Key: FOP-2211
>>                 URL: https://issues.apache.org/jira/browse/FOP-2211
>>             Project: Fop
>>          Issue Type: Bug
>>          Components: general
>>    Affects Versions: trunk
>>            Reporter: Alexios Giotis
>>             Fix For: trunk
>>
>>         Attachments: fop.patch, tempurisimple.patch, xgc.patch
>>
>>
>> As written in http://markmail.org/message/zelumstxxsdyvkcz , after the merge of the Temp_URI_Resolution branch (Sept 2012), the actual pattern of using temp files has changed from:
>> {code}
>> File tmpFile = File.createTempFile(....);
>> // Write and read from the file
>> tmpFile.delete();
>> {code}
>> to:
>> {code}
>> File tmpFile = new File(System.getProperty("java.io.tmpdir"), counterStartingFrom1AsString);
>> tmpFile.deleteOnExit();
>> // Write and read from the file
>> {code}
>> This is fine when FOP is executed from the command line (which I guess this is how most people use it) but it introduces a number of bad side effects for long running processes that use FOP embedded.
>>
>> 1. Different FOP processes can't be executed in parallel on the same system because creating the same temp file fails.
>> 2. If the JVM is not normally terminated, the temp files are never deleted and the next invocation of the JVM fails to run.
>> 3. deleteOnExit() keeps for the life of the JVM an unknown number of temp files both on disk and a reference in memory.
>> There should not be a need to implement a custom resource resolver when using FOP embedded in order to fix those issues. The default implementation should work at least as good as it worked in FOP 1.1 or earlier.
>> Attached are 2 patches, one for XGC and one for FOP that should fix and improve the handling of at least the temporary files.
>> For reference, [1] lists some reasons for implementing the new URI resource resolvers.
>> [1] http://wiki.apache.org/xmlgraphics-fop/URIResolution
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Commented] (FOP-2211) [PATCH] Fix & improve the handling of temporary files using the new URI resource resolvers

Alexios Giotis
Hi Peter,

I am not reusing in any way the temp files. Since FOP creates them, I believe FOP should also delete them. This should happen when temp files are no longer required and not when the JVM will terminate. This is what FOP does in version 1.1 or before. I could setup my own resource handler and override the close method of the Resource to delete the temp file, as you suggested. But fixing this in FOP is cleaner and benefits other people which use FOP embedded in their Java application.

Is it clear ? Do you agree ?


Thanks,
Alexios



On 7 Mar 2013, at 12:21, Peter Hancock <[hidden email]> wrote:

> Hi Alexios,
>
> I take it you are you reading from the temp files more than once,
> otherwise you could reluctantly hook into the close method of Resource
> to do the file deletion.  I am interested to know how you are
> benefiting from reusing the temp file.
>
> Thanks,
>
> Peter
>
> On Tue, Mar 5, 2013 at 10:55 AM, Alexios Giotis (JIRA) <[hidden email]> wrote:
>>
>>    [ https://issues.apache.org/jira/browse/FOP-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13593313#comment-13593313 ]
>>
>> Alexios Giotis commented on FOP-2211:
>> -------------------------------------
>>
>> Hi Simon,
>>
>> Thank you for the patch. I looked at it and it does not resolve the issue of keeping on disk many and big files. In a test we did last week, the temp file was 100GB and for sure we don't wish to keep such files until the JVM is normally terminated. For us, this is several months or a year.
>>
>> Also, I don't think that backwards compatibility is an issue here. This is trunk, there are many changes since 1.1 that do affect users embedding FOP in their apps and this is not one of them. I am sure it will be easy to change your code or to create an adapter.
>>
>> I will try to find some time to submit updated versions of the patches that take into account the comments above.
>>
>>
>>> [PATCH] Fix & improve the handling of temporary files using the new URI resource resolvers
>>> ------------------------------------------------------------------------------------------
>>>
>>>                Key: FOP-2211
>>>                URL: https://issues.apache.org/jira/browse/FOP-2211
>>>            Project: Fop
>>>         Issue Type: Bug
>>>         Components: general
>>>   Affects Versions: trunk
>>>           Reporter: Alexios Giotis
>>>            Fix For: trunk
>>>
>>>        Attachments: fop.patch, tempurisimple.patch, xgc.patch
>>>
>>>
>>> As written in http://markmail.org/message/zelumstxxsdyvkcz , after the merge of the Temp_URI_Resolution branch (Sept 2012), the actual pattern of using temp files has changed from:
>>> {code}
>>> File tmpFile = File.createTempFile(....);
>>> // Write and read from the file
>>> tmpFile.delete();
>>> {code}
>>> to:
>>> {code}
>>> File tmpFile = new File(System.getProperty("java.io.tmpdir"), counterStartingFrom1AsString);
>>> tmpFile.deleteOnExit();
>>> // Write and read from the file
>>> {code}
>>> This is fine when FOP is executed from the command line (which I guess this is how most people use it) but it introduces a number of bad side effects for long running processes that use FOP embedded.
>>>
>>> 1. Different FOP processes can't be executed in parallel on the same system because creating the same temp file fails.
>>> 2. If the JVM is not normally terminated, the temp files are never deleted and the next invocation of the JVM fails to run.
>>> 3. deleteOnExit() keeps for the life of the JVM an unknown number of temp files both on disk and a reference in memory.
>>> There should not be a need to implement a custom resource resolver when using FOP embedded in order to fix those issues. The default implementation should work at least as good as it worked in FOP 1.1 or earlier.
>>> Attached are 2 patches, one for XGC and one for FOP that should fix and improve the handling of at least the temporary files.
>>> For reference, [1] lists some reasons for implementing the new URI resource resolvers.
>>> [1] http://wiki.apache.org/xmlgraphics-fop/URIResolution
>>
>> --
>> This message is automatically generated by JIRA.
>> If you think it was sent incorrectly, please contact your JIRA administrators
>> For more information on JIRA, see: http://www.atlassian.com/software/jira