Memory issue  
Author Message
Frank Brouwer





PostPosted: 2003-10-7 0:47:00 Top

java-programmer, Memory issue Hi All,

I have a problem with memory when I use a connection to a database. I have
also posted the question in the database group but as I suspect that it has
to do with garbage collection I also post the question here.

I have a problem with memory using SQLserver on a server running WIN2K
Advanced Server with 2 GB! of internal memory. We limmited the SQLserver
memory to 1 GB and 512 MB for JVM (version 1.4.2_01). The OutOfMemory error
is always at the same place in the jdbc driver as shown below.

java.lang.OutOfMemoryError
at com.microsoft.util.UtilPagedTempBuffer.compressBlockList(Unknown Source)
at com.microsoft.util.UtilPagedTempBuffer.getBlock(Unknown Source)
at com.microsoft.util.UtilPagedTempBuffer.write(Unknown Source)
at com.microsoft.util.UtilPagedTempBuffer.write(Unknown Source)
at com.microsoft.util.UtilByteArrayDataProvider.receive(Unknown Source)
at com.microsoft.util.UtilByteOrderedDataReader.receive(Unknown Source)
at com.microsoft.jdbc.sqlserver.tds.TDSRPCRequest.submitRequest(Unknown
Source)
at com.microsoft.jdbc.sqlserver.tds.TDSCursorRequest.openCursor(Unknown
Source)
at com.microsoft.jdbc.sqlserver.SQLServerImplStatement.execute(Unknown
Source)
at com.microsoft.jdbc.base.BaseStatement.commonExecute(Unknown Source)
at com.microsoft.jdbc.base.BaseStatement.executeQueryInternal(Unknown
Source)
at com.microsoft.jdbc.base.BasePreparedStatement.executeQuery(Unknown
Source)
.............................................. Truncated from here
.................................................

If I run (using same database & JVM) on a computer (W2K prof.) with 512 MB
internal memory and no limmit set for MS-Sqlserver but for JVM 128 MB doing
the same stuff, there is no problem at all.

I suspect it has something to do with a late "kicking in" of the Garbage
Collection causing memory segmentation. So when a big chunk of memory is
required (a large result-set from the database) no contigouis memory block
of memory is availble in the current heap. The JVM claims more memory to
expand the heap, grows and grows until it cosses the limmit. But I also
tought that garbage collection takes care of that too.

I hope some one has seen this behaviour and can point me in the right
direction.

TIA.

Frank.


--
Met vriendelijke groet,
Frank Brouwer

_________________________________________________

Trimergo BV
Project Manufacturing Software

Amersfoortseweg 15 C
7313 AB Apeldoorn, Nederland
Telefoon: 055 - 577 7373
Telefax: 055 - 577 7370

www.trimergo.com


 
James





PostPosted: 2003-10-7 23:17:00 Top

java-programmer >> Memory issue Can you post a complete example of your call to the database. Are you
connection pooling? Are you releasing connections back into the pool or is
the pool growing too large? Are you manipulating arrays - any dangling
references? The data coming back - is it huge? Are any result sets hanging
around? Just a few questions that may spark something.

"Frank Brouwer" <email***@***.com> wrote in message
news:3f819c9c$0$445$email***@***.com...
> Hi All,
>
> I have a problem with memory when I use a connection to a database. I
have
> also posted the question in the database group but as I suspect that it
has
> to do with garbage collection I also post the question here.
>
> I have a problem with memory using SQLserver on a server running WIN2K
> Advanced Server with 2 GB! of internal memory. We limmited the SQLserver
> memory to 1 GB and 512 MB for JVM (version 1.4.2_01). The OutOfMemory
error
> is always at the same place in the jdbc driver as shown below.
>
> java.lang.OutOfMemoryError
> at com.microsoft.util.UtilPagedTempBuffer.compressBlockList(Unknown
Source)
> at com.microsoft.util.UtilPagedTempBuffer.getBlock(Unknown Source)
> at com.microsoft.util.UtilPagedTempBuffer.write(Unknown Source)
> at com.microsoft.util.UtilPagedTempBuffer.write(Unknown Source)
> at com.microsoft.util.UtilByteArrayDataProvider.receive(Unknown Source)
> at com.microsoft.util.UtilByteOrderedDataReader.receive(Unknown Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSRPCRequest.submitRequest(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSCursorRequest.openCursor(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.SQLServerImplStatement.execute(Unknown
> Source)
> at com.microsoft.jdbc.base.BaseStatement.commonExecute(Unknown Source)
> at com.microsoft.jdbc.base.BaseStatement.executeQueryInternal(Unknown
> Source)
> at com.microsoft.jdbc.base.BasePreparedStatement.executeQuery(Unknown
> Source)
> .............................................. Truncated from here
> .................................................
>
> If I run (using same database & JVM) on a computer (W2K prof.) with 512 MB
> internal memory and no limmit set for MS-Sqlserver but for JVM 128 MB
doing
> the same stuff, there is no problem at all.
>
> I suspect it has something to do with a late "kicking in" of the Garbage
> Collection causing memory segmentation. So when a big chunk of memory is
> required (a large result-set from the database) no contigouis memory block
> of memory is availble in the current heap. The JVM claims more memory to
> expand the heap, grows and grows until it cosses the limmit. But I also
> tought that garbage collection takes care of that too.
>
> I hope some one has seen this behaviour and can point me in the right
> direction.
>
> TIA.
>
> Frank.
>
>
> --
> Met vriendelijke groet,
> Frank Brouwer
>
> _________________________________________________
>
> Trimergo BV
> Project Manufacturing Software
>
> Amersfoortseweg 15 C
> 7313 AB Apeldoorn, Nederland
> Telefoon: 055 - 577 7373
> Telefax: 055 - 577 7370
>
> www.trimergo.com
>
>


 
Frank Brouwer





PostPosted: 2003-10-7 23:53:00 Top

java-programmer >> Memory issue Hi James,

"James" <email***@***.com> wrote in message
news:9OAgb.516621$cF.185652@rwcrnsc53...
> Can you post a complete example of your call to the database. Are you
> connection pooling? Are you releasing connections back into the pool or
is
> the pool growing too large? Are you manipulating arrays - any dangling
> references? The data coming back - is it huge? Are any result sets
hanging
> around? Just a few questions that may spark something.
>

I'm not using a pool and use prepared statements only, the close() method's
are invoked when needed. The resultSet is evaluated in a loop and the data
is put into objects which are passed as an Enumeration or Iterator to the
caller. But even if all your suggestions would be the case, it does not
explain why it works on a computer with limited resources (like a note-book)
and not on a machine with plenty resources (big server).

Thanks for thinking with me,

Frank.


 
 
James





PostPosted: 2003-10-8 5:12:00 Top

java-programmer >> Memory issue Well the reason it works on one machine and not the other is that very
little is guaranteed about the JVM and memory amangement, and different
machines and coinfigurations can result in different behavior. I would not
focus on the machine difference, it is in the code. After close, set the
objects to null and make sure the results are cleaned up after they are
sent. Again, no problem, tossing out the things i would check. "Frank
Brouwer" <email***@***.com> wrote in message
news:3f82e142$0$437$email***@***.com...
> Hi James,
>
> "James" <email***@***.com> wrote in message
> news:9OAgb.516621$cF.185652@rwcrnsc53...
> > Can you post a complete example of your call to the database. Are you
> > connection pooling? Are you releasing connections back into the pool or
> is
> > the pool growing too large? Are you manipulating arrays - any dangling
> > references? The data coming back - is it huge? Are any result sets
> hanging
> > around? Just a few questions that may spark something.
> >
>
> I'm not using a pool and use prepared statements only, the close()
method's
> are invoked when needed. The resultSet is evaluated in a loop and the
data
> is put into objects which are passed as an Enumeration or Iterator to the
> caller. But even if all your suggestions would be the case, it does not
> explain why it works on a computer with limited resources (like a
note-book)
> and not on a machine with plenty resources (big server).
>
> Thanks for thinking with me,
>
> Frank.
>
>


 
 
spalding





PostPosted: 2004-8-5 22:42:00 Top

java-programmer >> Memory issue
Koen wrote:
> I also had some OutOfMemoryExceptions while the 2GB limit was stil
> far off.
> I read somewhere about an issue with a 1GB limit but it seems relate
> to
> specific allocation sizes. We had situations where almost eac
> allocation
> done by the website throws OutofMemoryExceptions, even after man
> sessions
> timed-out and so a lot of memory was no longer rooted.
> end [/B]

Hi

I too was getting OutOfMemoryException(s) when users were posting file
to asp.net application via an html form. I set the maxRequestLength i
web.config to about 1GB and tried to upload a 170MB file but afte
about 120MB I would get the above error.

The reason was that asp.net was storing the entire contents of the fil
in memory and at 120MB it would recycle the asp_wp process.

According to the link below, the memoryLimit
attribute in the machine.config file specifies the
percentage of memory the asp_wp process can use before being recycled.
http://tinyurl.com/6lcna

I was able to get around this problem by parsing the posted form
stripping the file contents from the rest of the form and at the sam
time writing it to a file on disk


-
spaldin
-----------------------------------------------------------------------
Posted via http://www.codecomments.co
-----------------------------------------------------------------------

 
 
jamesche





PostPosted: 2004-8-6 2:35:00 Top

java-programmer >> Memory issue Koen,

It's very hard to tell anything from your graphic because several counters
share the same color. In order to tell anything, we'd need the .csv or
.blg file from Perfmon. I can't tell if your virtual memory is the top
yellow line or if that's private bytes. This is important to know because
in order for you to conclude that this may be a native leak, you'd need to
confirm that private bytes is increasing while the # Bytes in all Heaps is
decreasing or stays flat. I can't tell that from this screenshot.

I also don't know if you are running ASP.NET 1.0 or 1.1. The memory model
was changed in 1.1.

If you are interested in an article that gives some detail on how you can
debug memory issues, here is a great one from PAG:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/
DBGch02.asp?frame=true

Jim Cheshire [MSFT]
MCP+I, MCSE, MCSD, MCDBA
Microsoft Developer Support
email***@***.com

This post is provided "AS-IS" with no warranties and confers no rights.

--------------------
>From: "Koen" <email***@***.com>
>Subject: memory issue
>Date: Thu, 5 Aug 2004 14:58:54 +0200
>Lines: 171
>X-Priority: 3
>X-MSMail-Priority: Normal
>X-Newsreader: Microsoft Outlook Express 6.00.2800.1437
>X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441
>Message-ID: <email***@***.com>
>Newsgroups: microsoft.public.dotnet.framework.aspnet
>NNTP-Posting-Host: isa-dgm1.bvdep.com 193.194.158.107
>Path:
cpmsftngxa06.phx.gbl!TK2MSFTNGXA01.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP12
.phx.gbl
>Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:252309
>X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
>
>I've been reading a lot about GC and finding memory leaks etc. However I
>don't understand the memory behavior of our asp.net site.
>
>Here is what happens : when garbage collection combes by, it finds plenty
of
>memory no longer referenced (expired sessions) and this memory is freed
(see
>attached system monitor snapshot). So the #bytes in all heaps goes down by
>300MB. At the same time the total committed bytes on the server goes down
as
>well as the private bytes of the asp_wp process. However the virtual bytes
>of the process stay unchanged.
>Now I know that this happens if your CLR allocations are in the large
object
>heap because this is never compacted, but as the snapshot shows this is not
>the case, the freed memory was in the generation 2 heap
>
>The distance between CLR memory and virtual memory increases and that would
>mean memory leaks in native code called. However I would then expect the
>distance to increase mostly while GC is doing nothing, however I see the
>distance increase when CLR memory is freed while during CLR memory increase
>the Virtual bytes increase along (not immediately after a strong drop but
>shortly after)
>
>Is this the problem described in KB 833610 ? Or is it still something
>different ? Do I have to worry about native memory leaks ?
>
>I also had some OutOfMemoryExceptions while the 2GB limit was still far
off.
>I read somewhere about an issue with a 1GB limit but it seems related to
>specific allocation sizes. We had situations where almost each allocation
>done by the website throws OutofMemoryExceptions, even after many sessions
>timed-out and so a lot of memory was no longer rooted.
>
>Koen.
>
>
>

 
 
Koen





PostPosted: 2004-8-6 15:56:00 Top

java-programmer >> Memory issue Jim,

Thank you for your reply. Actually the same color for different lines didn't
bother me as total VM > committed bytes > private bytes
To be able to compare them I forced all monitors to the same scale (I should
have said that or have included it in my snapshot)
Your explanation of detecting a native leak is what I alos knew - and yes it
is true bytes in heap go down while VM stays flat - but I am surprised that
I don't see VM size go down when the garbage collection frees CLR memory.

I read (before posting) the MSDN article, but before starting up the tools
mentioned I tried to deduce just from the performance monitor if I should
look at the CLR or at native memory. I am quite sure that my problem is not
comming from rooted objects in the CLR.

Koen.

"Jim Cheshire [MSFT]" <email***@***.com> wrote in message
news:email***@***.com...
> Koen,
>
> It's very hard to tell anything from your graphic because several counters
> share the same color. In order to tell anything, we'd need the .csv or
> blg file from Perfmon. I can't tell if your virtual memory is the top
> yellow line or if that's private bytes. This is important to know because
> in order for you to conclude that this may be a native leak, you'd need to
> confirm that private bytes is increasing while the # Bytes in all Heaps is
> decreasing or stays flat. I can't tell that from this screenshot.
>
> I also don't know if you are running ASP.NET 1.0 or 1.1. The memory model
> was changed in 1.1.
>
> If you are interested in an article that gives some detail on how you can
> debug memory issues, here is a great one from PAG:
>
>
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/
> DBGch02.asp?frame=true
>
> Jim Cheshire [MSFT]
> MCP+I, MCSE, MCSD, MCDBA
> Microsoft Developer Support
> email***@***.com
>
> This post is provided "AS-IS" with no warranties and confers no rights.
>
> --------------------
> >Here is what happens : when garbage collection combes by, it finds plenty
> of
> >memory no longer referenced (expired sessions) and this memory is freed
> (see
> >attached system monitor snapshot). So the #bytes in all heaps goes down
by
> >300MB. At the same time the total committed bytes on the server goes down
> as
> >well as the private bytes of the asp_wp process. However the virtual
bytes
> >of the process stay unchanged.


 
 
Koen





PostPosted: 2004-8-6 16:00:00 Top

java-programmer >> Memory issue we are using framework 1.1

"Jim Cheshire [MSFT]" <email***@***.com> wrote in message
news:email***@***.com...

> >
> I also don't know if you are running ASP.NET 1.0 or 1.1. The memory model
> was changed in 1.1.
>


 
 
jamesche





PostPosted: 2004-8-6 23:33:00 Top

java-programmer >> Memory issue Koen,

In order to really tell what's going on for certain, you really need to
examine the memory in a user mode debugger. Using Perfmon, you can tell
that you are using x bytes of memory, but you don't know what sits in x
bytes. You don't know what's rooted and what's not. You don't know what
has been collected and what has not. (When I say "what" here, I don't mean
how much. I mean what objects.)

Perfmon is great for determining if you are really looking at a leak
situation (and what you have described is not a leak), but once you find
the 30,000 details provided by Perfmon, it's time to dig in with a dump and
user mode debugging.

Jim Cheshire [MSFT]
MCP+I, MCSE, MCSD, MCDBA
Microsoft Developer Support
email***@***.com

This post is provided "AS-IS" with no warranties and confers no rights.

--------------------
>From: "Koen" <email***@***.com>
>References: <email***@***.com>
<email***@***.com>
>Subject: Re: memory issue
>Date: Fri, 6 Aug 2004 09:56:42 +0200
>Lines: 60
>X-Priority: 3
>X-MSMail-Priority: Normal
>X-Newsreader: Microsoft Outlook Express 6.00.2800.1437
>X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441
>Message-ID: <email***@***.com>
>Newsgroups: microsoft.public.dotnet.framework.aspnet
>NNTP-Posting-Host: isa-dgm1.bvdep.com 193.194.158.107
>Path:
cpmsftngxa06.phx.gbl!TK2MSFTNGXA01.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP09
.phx.gbl
>Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:252539
>X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
>
>Jim,
>
>Thank you for your reply. Actually the same color for different lines
didn't
>bother me as total VM > committed bytes > private bytes
>To be able to compare them I forced all monitors to the same scale (I
should
>have said that or have included it in my snapshot)
>Your explanation of detecting a native leak is what I alos knew - and yes
it
>is true bytes in heap go down while VM stays flat - but I am surprised
that
>I don't see VM size go down when the garbage collection frees CLR memory.
>
>I read (before posting) the MSDN article, but before starting up the tools
>mentioned I tried to deduce just from the performance monitor if I should
>look at the CLR or at native memory. I am quite sure that my problem is not
>comming from rooted objects in the CLR.
>
>Koen.
>
>"Jim Cheshire [MSFT]" <email***@***.com> wrote in message
>news:email***@***.com...
>> Koen,
>>
>> It's very hard to tell anything from your graphic because several
counters
>> share the same color. In order to tell anything, we'd need the .csv or
>> blg file from Perfmon. I can't tell if your virtual memory is the top
>> yellow line or if that's private bytes. This is important to know
because
>> in order for you to conclude that this may be a native leak, you'd need
to
>> confirm that private bytes is increasing while the # Bytes in all Heaps
is
>> decreasing or stays flat. I can't tell that from this screenshot.
>>
>> I also don't know if you are running ASP.NET 1.0 or 1.1. The memory
model
>> was changed in 1.1.
>>
>> If you are interested in an article that gives some detail on how you can
>> debug memory issues, here is a great one from PAG:
>>
>>
>http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html
/
>> DBGch02.asp?frame=true
>>
>> Jim Cheshire [MSFT]
>> MCP+I, MCSE, MCSD, MCDBA
>> Microsoft Developer Support
>> email***@***.com
>>
>> This post is provided "AS-IS" with no warranties and confers no rights.
>>
>> --------------------
>> >Here is what happens : when garbage collection combes by, it finds
plenty
>> of
>> >memory no longer referenced (expired sessions) and this memory is freed
>> (see
>> >attached system monitor snapshot). So the #bytes in all heaps goes down
>by
>> >300MB. At the same time the total committed bytes on the server goes
down
>> as
>> >well as the private bytes of the asp_wp process. However the virtual
>bytes
>> >of the process stay unchanged.
>
>
>

 
 
jeffvandenberg





PostPosted: 2007-7-3 3:55:00 Top

java-programmer >> Memory issue I just upgraded a small web server to php 5.2.2 and apache to 2.2.4 on
a Win2K Pro box with MySQL 5. The only customization is loading MySQL
and MySQLi modules in PHP. Things had been working well enough under
PHP 4 and Apache 2.0, but I was wanting to upgrade to the latest and
greatest to toy with things and experiment.

However, I have noticed something that I think may be PHP's fault, but
i'm not sure. I'm seeing a memory leak in Apache. I was wondering if
there was something different about how PHP handled MySQLi result sets
or anything? I don't free_result() on all of my select statements
(this is very much an experimental site, though I'm trying to fix it
up). I'm seeing considerably more memory usage under this setup than
before I did the upgrade.

Regards,

Jeff Vandenberg
 
 
Jason.Herald@gmail.com





PostPosted: 2008-2-14 2:49:00 Top

java-programmer >> Memory issue I am having an issue where a struts based application continually
uses more and more ram. I get the available memory on each pass and
manually call gc which has slowed the issue but not fixed it.

I.e:
public ActionForward view (ActionMapping mapping,
ActionForm form,
HttpServletRequest request,
HttpServletResponse response) {

Session ses = new SessionManager().getSession();

Rooms rooms = new Rooms(ses);
Posts posts = new Posts(ses);

HttpSession session = request.getSession();

List room_list = rooms.find_all();
request.setAttribute("rooms", room_list);


List post_list = posts.full_set(new
Long(session.getAttribute("room_id").toString()));
request.setAttribute("posts", post_list);

rooms = null;
posts = null;

/* runtime variable r is declared above */
r.gc();

return mapping.findForward("done");

}


The memory usage follows this pattern:

Pass 1: (before gc) 231.4797 (after gc) 242.2315
Pass 2: (before gc) 232.2050 (after gc) 241.6035
Pass 3: (before gc) 231.8197 (after gc) 241.2478

JVM is started using (although I have tried many different
combinations of the arguments):

-server -XX:+CMSClassUnloadingEnabled -XX:PermSize=128M -
XX:MaxPermSize=128M -Xconcurrentio -XX:+UseConcMarkSweepGC -
XX:NewRatio=30 -XX:+UseParNewGC -XX:NewSize=8m -XX:MaxNewSize=8m -
Xms256m -Xmx256m -XX:MinHeapFreeRatio=70 -XX:MaxHeapFreeRatio=90

Any ideas would be greatly appreciated and if you want to see any of
the other files let me know.

Thanks in advance!

Jason
 
 
Jason.Herald@gmail.com





PostPosted: 2008-2-14 3:45:00 Top

java-programmer >> Memory issue I am having an issue where a struts based application continually
uses more and more ram. I get the available memory on each pass and
manually call gc which has slowed the issue but not fixed it.

I.e:
public ActionForward view (ActionMapping mapping,
ActionForm form,
HttpServletRequest request,
HttpServletResponse response) {

Session ses = new SessionManager().getSession();

Rooms rooms = new Rooms(ses);
Posts posts = new Posts(ses);

HttpSession session = request.getSession();

List room_list = rooms.find_all();
request.setAttribute("rooms", room_list);

List post_list = posts.full_set(new
Long(session.getAttribute("room_id").toString()));
request.setAttribute("posts", post_list);

rooms = null;
posts = null;

/* runtime variable r is declared above */
r.gc();

return mapping.findForward("done");

}

The memory usage follows this pattern:

Pass 1: (before gc) 231.4797 (after gc) 242.2315
Pass 2: (before gc) 232.2050 (after gc) 241.6035
Pass 3: (before gc) 231.8197 (after gc) 241.2478

JVM is started using (although I have tried many different
combinations of the arguments):

-server -XX:+CMSClassUnloadingEnabled -XX:PermSize=128M -
XX:MaxPermSize=128M -Xconcurrentio -XX:+UseConcMarkSweepGC -
XX:NewRatio=30 -XX:+UseParNewGC -XX:NewSize=8m -XX:MaxNewSize=8m -
Xms256m -Xmx256m -XX:MinHeapFreeRatio=70 -XX:MaxHeapFreeRatio=90

Any ideas would be greatly appreciated and if you want to see any of
the other files let me know.

Thanks in advance!

Jason
 
 
none





PostPosted: 2008-2-14 6:16:00 Top

java-programmer >> Memory issue email***@***.com wrote:
> I am having an issue where a struts based application continually
> uses more and more ram. I get the available memory on each pass and
> manually call gc which has slowed the issue but not fixed it.
>
> I.e:
> public ActionForward view (ActionMapping mapping,
> ActionForm form,
> HttpServletRequest request,
> HttpServletResponse response) {
>
> Session ses = new SessionManager().getSession();
>
> Rooms rooms = new Rooms(ses);
> Posts posts = new Posts(ses);
>
> HttpSession session = request.getSession();
>
> List room_list = rooms.find_all();
> request.setAttribute("rooms", room_list);
>
>
> List post_list = posts.full_set(new
> Long(session.getAttribute("room_id").toString()));
> request.setAttribute("posts", post_list);
>
> rooms = null;
> posts = null;
>
> /* runtime variable r is declared above */
> r.gc();
>
> return mapping.findForward("done");
>
> }
>
>
> The memory usage follows this pattern:
>
> Pass 1: (before gc) 231.4797 (after gc) 242.2315
> Pass 2: (before gc) 232.2050 (after gc) 241.6035
> Pass 3: (before gc) 231.8197 (after gc) 241.2478
>
> JVM is started using (although I have tried many different
> combinations of the arguments):
>
> -server -XX:+CMSClassUnloadingEnabled -XX:PermSize=128M -
> XX:MaxPermSize=128M -Xconcurrentio -XX:+UseConcMarkSweepGC -
> XX:NewRatio=30 -XX:+UseParNewGC -XX:NewSize=8m -XX:MaxNewSize=8m -
> Xms256m -Xmx256m -XX:MinHeapFreeRatio=70 -XX:MaxHeapFreeRatio=90
>
> Any ideas would be greatly appreciated and if you want to see any of
> the other files let me know.
>
> Thanks in advance!
>
> Jason

it's hard to say without the list of imports and without the code of the
beans Room and Post.
by the way the before gc value deos not seem to grow...


 
 
Daniel Pitts





PostPosted: 2008-2-14 6:30:00 Top

java-programmer >> Memory issue email***@***.com wrote:
> I am having an issue where a struts based application continually
> uses more and more ram. I get the available memory on each pass and
> manually call gc which has slowed the issue but not fixed it.
>
[snip code]
> Any ideas would be greatly appreciated and if you want to see any of
> the other files let me know.
>
> Thanks in advance!
>
> Jason
Unless you are seeing an Out Of Memory exception, don't worry about
available memory. The Garbage Collector doesn't necessarily pick things
up as soon as they're no longer needed.

You might also make sure that you're not holding on to objects in static
maps/collections or long lived maps/collections.

--
Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
 
 
Mark Space





PostPosted: 2008-2-14 6:55:00 Top

java-programmer >> Memory issue email***@***.com wrote:

> Pass 1: (before gc) 231.4797 (after gc) 242.2315
> Pass 2: (before gc) 232.2050 (after gc) 241.6035
> Pass 3: (before gc) 231.8197 (after gc) 241.2478

First pass, memory went up. First time you use this object more memory
is needed. Second and third pass, memory use went down. So I don't see
that memory is increasing here. I don't think there are any errors in
the program.

Let this app run for a long time. Monitor it. Is it actually
increasing in memory use over the long haul? Do objects eventually time
out and get released and memory reclaimed? (Make sure your session
objects all time out.) Does the GC continue to run with increasing
frequency and less results? Do you eventually get an Out of Memory Error?

Need more info here. Looks ok to me.

Look up some debugging info for Java, especially JMX. Use that. Get
rid of the gratuitous call to the GC, it isn't needed.
 
 
Lew





PostPosted: 2008-2-14 8:04:00 Top

java-programmer >> Memory issue Daniel Pitts wrote:
> email***@***.com wrote:
>> I am having an issue where a struts based application continually
>> uses more and more ram. I get the available memory on each pass and
>> manually call gc which has slowed the issue but not fixed it.
>>
> [snip code]
>> Any ideas would be greatly appreciated and if you want to see any of
>> the other files let me know.
>>
>> Thanks in advance!
>>
>> Jason
> Unless you are seeing an Out Of Memory exception, don't worry about
> available memory. The Garbage Collector doesn't necessarily pick things
> up as soon as they're no longer needed.
>
> You might also make sure that you're not holding on to objects in static
> maps/collections or long lived maps/collections.

For those who answered on clj.programmer, there is more in response to the
clj.help multipost, and vice versa.

To the OP: Please do not multi-post. Read the FAQ post from David Alex Lamb,
"comp.lang.java.{help,programmer} - what they're for (mini-FAQ 2006-03-31)",
and its links for details. The FAQ is posted every five days, so there are no
worries about missing it.

Cross-posted in service of the OP.

--
Lew
 
 
Jason.Herald@gmail.com





PostPosted: 2008-2-14 9:04:00 Top

java-programmer >> Memory issue On Feb 13, 5:16爌m, none <""mrloyal\"@(none)"> wrote:
> email***@***.com wrote:
> > I am 爃aving an issue where a struts based application continually
> > uses more and more ram. 營 get the available memory on each pass and
> > manually call gc which has slowed the issue but not fixed it.
>
> > I.e:
> > ??public ActionForward view (ActionMapping mapping,
> > ????????????????燗ctionForm form,
> > ????????????????燞ttpServletRequest request,
> > ????????????????燞ttpServletResponse response) {
>
> > ????Session ses = new SessionManager().getSession();
>
> > ????Rooms rooms = new Rooms(ses);
> > ????Posts posts = new Posts(ses);
>
> > ????HttpSession session = request.getSession();
>
> > ????List room_list = rooms.find_all();
> > ????request.setAttribute("rooms", room_list);
>
> > ????List post_list = posts.full_set(new
> > Long(session.getAttribute("room_id").toString()));
> > ????request.setAttribute("posts", post_list);
>
> > ????rooms = null;
> > ????posts = null;
>
> > /* runtime variable r is declared above */
> > ????r.gc();
>
> > ????return mapping.findForward("done");
>
> > ??}
>
> > The memory usage follows this pattern:
>
> > Pass 1: (before gc) 231.4797 (after gc) 242.2315
> > Pass 2: (before gc) 232.2050 (after gc) 241.6035
> > Pass 3: (before gc) 231.8197 (after gc) 241.2478
>
> > JVM is started using (although I have tried many different
> > combinations of the arguments):
>
> > -server -XX:+CMSClassUnloadingEnabled -XX:PermSize=128M -
> > XX:MaxPermSize=128M -Xconcurrentio -XX:+UseConcMarkSweepGC -
> > XX:NewRatio=30 -XX:+UseParNewGC -XX:NewSize=8m -XX:MaxNewSize=8m -
> > Xms256m -Xmx256m -XX:MinHeapFreeRatio=70 -XX:MaxHeapFreeRatio=90
>
> > Any ideas would be greatly appreciated and if you want to see any of
> > the other files let me know.
>
> > Thanks in advance!
>
> > Jason
>
> it's hard to say without the list of imports and without the code of the
> beans Room and Post.
> by the way the before gc value deos not seem to grow...- Hide quoted text -
>
> - Show quoted text

The imports are the standard ones for a servlet.
I will post the code in the morning for you to review.
 
 
Jason.Herald@gmail.com





PostPosted: 2008-2-14 9:04:00 Top

java-programmer >> Memory issue On Feb 13, 5:16爌m, none <""mrloyal\"@(none)"> wrote:
> email***@***.com wrote:
> > I am 爃aving an issue where a struts based application continually
> > uses more and more ram. 營 get the available memory on each pass and
> > manually call gc which has slowed the issue but not fixed it.
>
> > I.e:
> > ??public ActionForward view (ActionMapping mapping,
> > ????????????????燗ctionForm form,
> > ????????????????燞ttpServletRequest request,
> > ????????????????燞ttpServletResponse response) {
>
> > ????Session ses = new SessionManager().getSession();
>
> > ????Rooms rooms = new Rooms(ses);
> > ????Posts posts = new Posts(ses);
>
> > ????HttpSession session = request.getSession();
>
> > ????List room_list = rooms.find_all();
> > ????request.setAttribute("rooms", room_list);
>
> > ????List post_list = posts.full_set(new
> > Long(session.getAttribute("room_id").toString()));
> > ????request.setAttribute("posts", post_list);
>
> > ????rooms = null;
> > ????posts = null;
>
> > /* runtime variable r is declared above */
> > ????r.gc();
>
> > ????return mapping.findForward("done");
>
> > ??}
>
> > The memory usage follows this pattern:
>
> > Pass 1: (before gc) 231.4797 (after gc) 242.2315
> > Pass 2: (before gc) 232.2050 (after gc) 241.6035
> > Pass 3: (before gc) 231.8197 (after gc) 241.2478
>
> > JVM is started using (although I have tried many different
> > combinations of the arguments):
>
> > -server -XX:+CMSClassUnloadingEnabled -XX:PermSize=128M -
> > XX:MaxPermSize=128M -Xconcurrentio -XX:+UseConcMarkSweepGC -
> > XX:NewRatio=30 -XX:+UseParNewGC -XX:NewSize=8m -XX:MaxNewSize=8m -
> > Xms256m -Xmx256m -XX:MinHeapFreeRatio=70 -XX:MaxHeapFreeRatio=90
>
> > Any ideas would be greatly appreciated and if you want to see any of
> > the other files let me know.
>
> > Thanks in advance!
>
> > Jason
>
> it's hard to say without the list of imports and without the code of the
> beans Room and Post.
> by the way the before gc value deos not seem to grow...- Hide quoted text -
>
> - Show quoted text -

It does shrink over time.
Before my original changes it took about 3 minutes to get to an out of
memory exception.
 
 
Jason.Herald@gmail.com





PostPosted: 2008-2-14 21:01:00 Top

java-programmer >> Memory issue On Feb 13, 5:54 pm, Mark Space <email***@***.com> wrote:
> email***@***.com wrote:
> > Pass 1: (before gc) 231.4797 (after gc) 242.2315
> > Pass 2: (before gc) 232.2050 (after gc) 241.6035
> > Pass 3: (before gc) 231.8197 (after gc) 241.2478
>
> First pass, memory went up. First time you use this object more memory
> is needed. Second and third pass, memory use went down. So I don't see
> that memory is increasing here. I don't think there are any errors in
> the program.
>
> Let this app run for a long time. Monitor it. Is it actually
> increasing in memory use over the long haul? Do objects eventually time
> out and get released and memory reclaimed? (Make sure your session
> objects all time out.) Does the GC continue to run with increasing
> frequency and less results? Do you eventually get an Out of Memory Error?
>
> Need more info here. Looks ok to me.
>
> Look up some debugging info for Java, especially JMX. Use that. Get
> rid of the gratuitous call to the GC, it isn't needed.


I will post more passes (fyi the number is the current available
memory so it is going to go down).

If I let the app run for 20 minutes it runs out of memory.

And no there are no errors aside from the java.lang.OutofMemory
 
 
Lew





PostPosted: 2008-2-14 21:37:00 Top

java-programmer >> Memory issue email***@***.com wrote:
> I will post more passes (fyi the number is the current available
> memory so it is going to go down).
>
> If I let the app run for 20 minutes it runs out of memory.
>
> And no there are no errors aside from the java.lang.OutofMemory

Usually when one doesn't post an entire example
<http://www.physci.org/codes/sscce.html>
the problem turns out to be in the code not shown. Complete examples allow
for valid answers to your question.

Make sure that you do not multipost when you provide the example.

--
Lew
 
 
Mark Space





PostPosted: 2008-2-15 0:35:00 Top

java-programmer >> Memory issue email***@***.com wrote:

>
> I will post more passes (fyi the number is the current available
> memory so it is going to go down).
>
> If I let the app run for 20 minutes it runs out of memory.
>
> And no there are no errors aside from the java.lang.OutofMemory

Well at least 20 minutes is relatively quickly. Unfortunately after you
attach a debugger 20 minutes might turn into 20,000 minutes. I think
you'll need to try anyway. Get a debugger or other performance tool,
attach it and let the process run. Collect GC info so you can see what
objects are surviving GC. (They'll have their generation number
increasing.) That should at least get you pointed at the right part of
the code.

Here's an older reference, I didn't have time to look extensively for
the latest info:

<http://www.ibm.com/developerworks/library/j-perf06304/>

It will at least get you some key words to search for and get a debugger
running in your environment. I think that's the first step. Until you
have some clue where the error really is, posting code is unlikely to
produce a resolution. And knowing how to attach tools and profile your
code is a useful skill.