Re: Forcefully stop org.apache.xalan.transformer.TransformerImpl.transform()

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Forcefully stop org.apache.xalan.transformer.TransformerImpl.transform()

Alexios Giotis
Hi Normen,

The Java Thread Model does not support forcefully stopping a thread executing arbitrary code. Thread.stop() is deprecated as it unlocks all the monitors it has locked and other threads may view an inconsistent state of protected objects. Thread.interrupt() does not guarantee when or if the thread will be interrupted. There are some tricks what work in certain cases but not in general.

To implement reliably such functionality, you may spawn a new process (e.g. Process p = processBuilder.start()) and keep a reference to the Process p. The new process will execute the transformation using Xalan or any other code. When needed, you may kill it (p.destroy()).

But this is not enough. You will need to choose a way for interprocess communication (sockets ? files ? or better some library) and implement it. 

It is much easier and better to find what is causing the memory leak. There are plenty of tools in Java to help with this task. Xalan and FOP work quite well and I am not aware of any memory leak. I suggest to let the application run for some time and when the memory usage is getting high, connect to it using Java Mission Control or jVisualVM or even JConsole or jmap to take a heap dump. Then analyze the heap dump to find out where memory is retained. If the heap dump is several GBs, I suggest to use Eclipse MAT which works with huge heap dumps. 


HTH,
Alexios Giotis







On 13 Dec 2017, at 14:52, Normen Ruhrus <[hidden email]> wrote:

Hey there,
 
using FOP for quite some time now, thanks everyone for all the fine development.
 
Iam running FOP 2.0 embedded into a Servlet on Tomcat 8.0.x with Java8 on a Windows Machine (server 2012 R2).
Usually we have no problems but from time to time during the transformation org.apache.xalan.transformer.TransformerImpl.transform() process an Out of Memory Error occurs.
 
When trying to recreate the problem with the same XML and same XSL we have no problems; the process takes a very short time (<100ms) and uses not much memory (<50MB).
The application is configured so it can cosume up to 12GB(!) of RAM so it looks like some indefinite looping is going on, but unfortunately i have no further  logs at this time to help me with the investigation.
 
This is why i wanted to put the transformation into a seperate Thread and supervise it and kill it off if a certain timeout is reached, however iam not able to. 
So the question is can you think of any way to stop this processing? I have implemented the multithreading here with FutureTasks but can switch to just Threads easily, too. It would be better to kill off the Thread before it throws an Out of Memory error  and with this affecting the whole Servlet so that subsequent requests cannot be processed anymore.
 
Naturally it would be best to find the source of the problem, of course, but that can take me some time and i would like to have a fallback solution instead with forcefully stopping the Transformation.
Hope someone is able to help me out with this; thanks in advance and best Greetings
 
Normen

Reply | Threaded
Open this post in threaded view
|

Re: Forcefully stop org.apache.xalan.transformer.TransformerImpl.transform()

Alexios Giotis
To avoid waiting until the memory usage gets high, start the JVM with the following options which will write the heap dump when an out of memory error occurs. I use them in production systems, there is no overhead.

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/diskWithFreeSpace/dumps

Note that the heap dump size will be equal to the memory size (12GB in your case).


On 14 Dec 2017, at 00:53, Alexios Giotis <[hidden email]> wrote:

Hi Normen,

The Java Thread Model does not support forcefully stopping a thread executing arbitrary code. Thread.stop() is deprecated as it unlocks all the monitors it has locked and other threads may view an inconsistent state of protected objects. Thread.interrupt() does not guarantee when or if the thread will be interrupted. There are some tricks what work in certain cases but not in general.

To implement reliably such functionality, you may spawn a new process (e.g. Process p = processBuilder.start()) and keep a reference to the Process p. The new process will execute the transformation using Xalan or any other code. When needed, you may kill it (p.destroy()).

But this is not enough. You will need to choose a way for interprocess communication (sockets ? files ? or better some library) and implement it. 

It is much easier and better to find what is causing the memory leak. There are plenty of tools in Java to help with this task. Xalan and FOP work quite well and I am not aware of any memory leak. I suggest to let the application run for some time and when the memory usage is getting high, connect to it using Java Mission Control or jVisualVM or even JConsole or jmap to take a heap dump. Then analyze the heap dump to find out where memory is retained. If the heap dump is several GBs, I suggest to use Eclipse MAT which works with huge heap dumps. 


HTH,
Alexios Giotis







On 13 Dec 2017, at 14:52, Normen Ruhrus <[hidden email]> wrote:

Hey there,
 
using FOP for quite some time now, thanks everyone for all the fine development.
 
Iam running FOP 2.0 embedded into a Servlet on Tomcat 8.0.x with Java8 on a Windows Machine (server 2012 R2).
Usually we have no problems but from time to time during the transformation org.apache.xalan.transformer.TransformerImpl.transform() process an Out of Memory Error occurs.
 
When trying to recreate the problem with the same XML and same XSL we have no problems; the process takes a very short time (<100ms) and uses not much memory (<50MB).
The application is configured so it can cosume up to 12GB(!) of RAM so it looks like some indefinite looping is going on, but unfortunately i have no further  logs at this time to help me with the investigation.
 
This is why i wanted to put the transformation into a seperate Thread and supervise it and kill it off if a certain timeout is reached, however iam not able to. 
So the question is can you think of any way to stop this processing? I have implemented the multithreading here with FutureTasks but can switch to just Threads easily, too. It would be better to kill off the Thread before it throws an Out of Memory error  and with this affecting the whole Servlet so that subsequent requests cannot be processed anymore.
 
Naturally it would be best to find the source of the problem, of course, but that can take me some time and i would like to have a fallback solution instead with forcefully stopping the Transformation.
Hope someone is able to help me out with this; thanks in advance and best Greetings
 
Normen


Reply | Threaded
Open this post in threaded view
|

AW: Forcefully stop org.apache.xalan.transformer.TransformerImpl.transform()

Normen

Hey Alexios,

 

thanks a lot for your help. I read some time yesterday and also came to the conclusion to use the ProcessBuilder; i just wanted to test how much time it will take to spawn a new process for every transformation and use the Object In/Out Streams for handling the communication.

 

But of course you may be right that there is need for a permanent process (or more than one depending on the hardware) and communicate with it.

 

I was not aware of the capabilities to create Heap Dumps; i will check it out, too.

 

Best Regards and have a nice christmas  time

 

Normen

 

 

-------------------------------------------------------------------------------------------

Alphasoft Software Dienstleistungs- und Datenverarbeitungs GmbH

 

Prinz-Regent-Str. 68a, 44795 Bochum

fon: +49(0) 234 609 37 118

e-mail: [hidden email]

 

Amtsgericht - Registergericht - Bochum

HRB 1183, Sitz Bochum

USt.ID-Nr.: DE127057789

Geschäftsführer: Jens Plasa, Normen Ruhrus, Axel Freimuth

 

Von: Alexios Giotis [mailto:[hidden email]]
Gesendet: Donnerstag, 14. Dezember 2017 00:10
An: [hidden email]
Betreff: Re: Forcefully stop org.apache.xalan.transformer.TransformerImpl.transform()

 

To avoid waiting until the memory usage gets high, start the JVM with the following options which will write the heap dump when an out of memory error occurs. I use them in production systems, there is no overhead.

 

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/diskWithFreeSpace/dumps



Note that the heap dump size will be equal to the memory size (12GB in your case).

 



On 14 Dec 2017, at 00:53, Alexios Giotis <[hidden email]> wrote:

 

Hi Normen,

 

The Java Thread Model does not support forcefully stopping a thread executing arbitrary code. Thread.stop() is deprecated as it unlocks all the monitors it has locked and other threads may view an inconsistent state of protected objects. Thread.interrupt() does not guarantee when or if the thread will be interrupted. There are some tricks what work in certain cases but not in general.

 

To implement reliably such functionality, you may spawn a new process (e.g. Process p = processBuilder.start()) and keep a reference to the Process p. The new process will execute the transformation using Xalan or any other code. When needed, you may kill it (p.destroy()).

 

But this is not enough. You will need to choose a way for interprocess communication (sockets ? files ? or better some library) and implement it. 

 

It is much easier and better to find what is causing the memory leak. There are plenty of tools in Java to help with this task. Xalan and FOP work quite well and I am not aware of any memory leak. I suggest to let the application run for some time and when the memory usage is getting high, connect to it using Java Mission Control or jVisualVM or even JConsole or jmap to take a heap dump. Then analyze the heap dump to find out where memory is retained. If the heap dump is several GBs, I suggest to use Eclipse MAT which works with huge heap dumps. 

 

 

HTH,

Alexios Giotis

 

 

 

 

 

 



On 13 Dec 2017, at 14:52, Normen Ruhrus <[hidden email]> wrote:

 

Hey there,

 

using FOP for quite some time now, thanks everyone for all the fine development.

 

Iam running FOP 2.0 embedded into a Servlet on Tomcat 8.0.x with Java8 on a Windows Machine (server 2012 R2).

Usually we have no problems but from time to time during the transformation org.apache.xalan.transformer.TransformerImpl.transform() process an Out of Memory Error occurs.

 

When trying to recreate the problem with the same XML and same XSL we have no problems; the process takes a very short time (<100ms) and uses not much memory (<50MB).

The application is configured so it can cosume up to 12GB(!) of RAM so it looks like some indefinite looping is going on, but unfortunately i have no further  logs at this time to help me with the investigation.

 

This is why i wanted to put the transformation into a seperate Thread and supervise it and kill it off if a certain timeout is reached, however iam not able to. 

So the question is can you think of any way to stop this processing? I have implemented the multithreading here with FutureTasks but can switch to just Threads easily, too. It would be better to kill off the Thread before it throws an Out of Memory error  and with this affecting the whole Servlet so that subsequent requests cannot be processed anymore.

 

Naturally it would be best to find the source of the problem, of course, but that can take me some time and i would like to have a fallback solution instead with forcefully stopping the Transformation.

Hope someone is able to help me out with this; thanks in advance and best Greetings

 

Normen