Memory Leakage drawing Images from Stream - Minimum example

java-programmer, Memory Leakage drawing Images from Stream - Minimum example Ok folks

I posted this a number of times ago,but never posted a minimum code
example for people to look at.

I was trying to load and draw images from a Mjpeg-Stream, reading from
the stream in a different thread.

This is the thread where the images are read by:

public class Axis2420
{

private HttpURLConnection hurlconnection = null;
private DataInputStream datainstream = null;
private String serialNumber = null;
private boolean connected = false;
private Frame dummy = null;
private Image image = null;
private String MJpegStreamUrl = null;

Runnable decodingThread = null;

public void SetURL(String url)
{
MJpegStreamUrl = url;
}

public boolean Connect()
{

if(MJpegStreamUrl == null)
{
System.out.println("Error: Empty Camera URL String");
return false;
}

try
{

URL streamurl = new URL(MJpegStreamUrl);
hurlconnection = (HttpURLConnection)streamurl.openConnection();
hurlconnection.setReadTimeout(10000);
//hurlconnection.setUseCaches(false);

InputStream instream = hurlconnection.getInputStream();

connected = true;

BufferedInputStream bis = new BufferedInputStream(instream);
datainstream = new DataInputStream(bis);

} catch(IOException e)
{
System.out.println("Fehler");

try
{
hurlconnection.disconnect();
Thread.sleep(60);
}
catch(InterruptedException ie)
{
hurlconnection.disconnect();
Connect();
}
catch(Exception ex)
{return false; }

System.out.println(e.getMessage());

return false;
}

return true;
}

//run method of the reading thread
public void run()
{
//test if still connected
if(!connected)
return;

try
{
//decode stream
if(connected)
{
Image i = null;
//ImageIO.setUseCache(false);
long before,after;

while(!Thread.currentThread().isInterrupted())
{

i = null;

String header;
int k = 0;

do
{
header = datainstream.readLine();
//System.out.println("Zeile: " +header);
}
while(++k < 3);

int j = (new Integer(header.substring("Content-Length:
".length()))).intValue();

datainstream.readByte();
datainstream.readByte();
// datainstream.readLine();
imageJPG = new byte[j];
datainstream.readFully(imageJPG, 0, j);
byte tail[] = new byte[4];
datainstream.readFully(tail,0,4);

i =
Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);

image = i;
i = null;

fireChange(); //calls the stateChanged() method in
the class 'MainProgram' below

image = null;
i = null;

Thread.currentThread().sleep(10);
}
}
}
catch(Exception ex)
{
ex.printStackTrace();
}
finally
{

try {
datainstream.close();
} catch(Exception ex)
{
ex.printStackTrace();
}
}
}

//for the listening objects so they can fetch the latest image
public Image getLatestImage()
{
return image;
}

<....>

}

Once a new image has arrived, the fireChange() method calls this method
on the MainProgram class:

class MainProgram
extends JFrame
implements Runnable,ChangeListener
{
public static String WINDOW_TITLE = "BINO GESTENERKENNUNG";
public static int WINDOW_WIDTH = 640;
public static int WINDOW_HEIGHT = 480;

public static final int CAMERA_LEFT = 0;
public static final int CAMERA_RIGHT = 1;

//public static final String leftCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";

//public static final String rightCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";

public static final String leftCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=70&date=1&fps=5";

public static final String rightCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=100&date=1&fps=5";

Image currentLeftImage;
Image currentRightImage;

Graphics leftImageGraphics;
Graphics rightImageGraphics;
Graphics leftCameraGraphics;
Graphics rightCameraGraphics;
byte[] currentLeftImageJPG;
byte[] currentRightImageJPG;

BufferedImage testImage;

Axis2420 leftCamera = null;
Axis2420 rightCamera = null;

CameraCanvas leftCameraCanvas = null;
CameraCanvas rightCameraCanvas = null;

Runnable drawThread = null;
boolean readyToDraw = false;
boolean newImage = false;

long frameRateLeft = 0,frameRateRight = 0;
long frameTimesLeft[],frameTimesRight[];

//vPointerDetector leftImagePointer;
//vPointerDetector rightImagePointer;
vPointerDetector imagePointer;

boolean updateLeftImage,updateRightImage;

<.....lots of meaningless stuff.......>

public void stateChanged(ChangeEvent e)
{

currentRightImage = rightCamera.getLatestImage();

rightCameraCanvas.SetUpdate(true); //otherwise paintComponent()
of the JCanvas won't draw the image
rightCameraCanvas.SetImage(currentRightImage);
rightCameraCanvas.repaint();
currentRightImage = null;
currentRightImageJPG = null;
}
}

}

} //end of MainProgram class

Basically, all the stateChange method does is getting the image from the
image-reading thread ('rightCamera' of type Axis2420) after
rightCamera called fireChange(). The image is passed to an object of the
class CameraCanvas which derieves from JCanvas. After that, we cast
repaint() on the CameraCanvas (rightCameraCanvas.repaint() ).

This is the CameraCanvas that is responsible for drawing:

public class CameraCanvas extends JPanel
{

private Image image = null;
boolean drawNext = false;
boolean readyForNext;

public void paintComponent(Graphics g)
{
if(drawNext == true)
{

readyForNext =
g.drawImage(image,0,0,getSize().width,getSize().height,this);
//image.flush();
image = null;
//g.dispose();

//imageBuffered.flush();
drawNext = false;
}

}

//indicates that there is a new image
public void SetUpdate(boolean up)
{
drawNext = up;
}

//passes a new image to the Canvas
public void SetImage(Image i)
{
image = i;
}
}

The program draws the stream correctly and is smooth like butter.
However, there is a HUGE memory leak somewhere that raises the memory
usage by about 40MBytes per second. If I remove the call to

readyForNext = g.drawImage(image,0,0,getSize().width,getSize().height,this);

in the CameraCanvas thread, the leak will go down to about 40 kbyte /
second. If I leave it in and post a System.gc() call afterwards, the
leak will also go down.

I think that I'm somewhat polling the AWTEvent thread which lots of
copies of images waiting to be drawn. Also using
Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);

spawns abouth 4 ImageFetcher threads which I think are the reason for
the small memory leak.

Now the question: What can I do about it?

It seems that the garbage collection isn't called after drawing an image
and casting it manually via System.gc() slows everything down a lot.
Using ImageIO.read)() instead of the Toolkit seems to remove the small
leak, but not the big one for drawing. And it tends to be slower since
it's blocking and keeping the Axis2420 thread from reading from the stream.

My thesis is due soon, so pleease have a look at it again.!!
Thanks in advance!!!

java-programmer >> Memory Leakage drawing Images from Stream - Minimum example > I posted this a number of times ago,but never posted a minimum code
> example for people to look at.
>
> I was trying to load and draw images from a Mjpeg-Stream, reading from
> the stream in a different thread.

I would use only _one_ image.
set MemoryImageSource animated.
Read every image to MemoryImageSource's data array and call newpixels();

--
Andrey Kuznetsov
http://uio.imagero.com Unified I/O for Java
http://reader.imagero.com Java image reader
http://jgui.imagero.com Java GUI components and utilities

java-programmer >> Memory Leakage drawing Images from Stream - Minimum example But MemoryImageSource only works with RGB pixel data and not JFIF data,
doesn't it? It would require me to convert it to pixel data first via
BufferedImage via ImageIO and thats what I'm trying to avoid. By the
way: I tried using another thread which does the decoding instead of the
Axis2420 thread with ImageIO and it leaked as well. It seems like
everything depends on the Axis2420 thread not bringing in new images too
fast. If for example I run the program with JProfiler (which drops
performance due to the whole analyzing stuff) the memory-leak doesn't
appear. It is slow, however.

java-programmer >> Memory Leakage drawing Images from Stream - Minimum example
Ok, it seems that I have a semi-stable version running now with what
Andrey mentioned and frames dropping. Thanks for your help!

java-programmer >> Memory Leakage drawing Images from Stream - Minimum example On Wed, 22 Feb 2006 17:56:46 +0100, "E. Naubauer"
<email***@***.com> wrote, quoted or indirectly quoted someone who
said :

>But MemoryImageSource only works with RGB pixel data and not JFIF data,
>doesn't it?

is there document that gives say a 5 page overview of what all the
Java classes for image processing are FOR perhaps with some recipes
for how you do the usual things?

It is all a blur. How do you know which classes to use for what sorts
of problem? How do you know when you glue them together there is not
some much cleaner way?

--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

java-programmer >> Memory Leakage drawing Images from Stream - Minimum example Roedy Green schrieb:
> On Wed, 22 Feb 2006 17:56:46 +0100, "E. Naubauer"
> <email***@***.com> wrote, quoted or indirectly quoted someone who
> said :
>
>> But MemoryImageSource only works with RGB pixel data and not JFIF data,
>> doesn't it?
>
> is there document that gives say a 5 page overview of what all the
> Java classes for image processing are FOR perhaps with some recipes
> for how you do the usual things?
>
> It is all a blur. How do you know which classes to use for what sorts
> of problem? How do you know when you glue them together there is not
> some much cleaner way?
>
>
>

Do you mean something like this?

http://java.sun.com/developer/technicalArticles/Media/imagestrategies/index.html

Although you might already know it.

Ok, let me do a short summary about my experiences of the last weeks
regarding reading images from an (Axis) network camera:

Images arrive at a framerate of ~25 frames per second, meaning every
frame has to be processed within about 40 ms to process them all.
However, the fashion in which Java deals with those is quite different
depending on the classes used under the hood, the threading strategy etc. :

- the best method if it comes only to drawing the images is decoding the
image with Toolkit.createImage and draw it using a double-buffer
strategy. The thread, that fetches and decodes the images (and casts
repaint on the drawing component) should run with the lowest priority,
or you might cause a memory leak since the images don't get garbage
collected correctly.

- ImageIO loads images synchronously and causes less memory problems,
but keeps the fetching/decoding thread busy which might drop the
framerate critically.

- images drawn with drawImage don't seem to be garbage collected
correctly if repaint() is called too often.

java-programmer >> Memory Leakage drawing Images from Stream - Minimum example On Thu, 23 Feb 2006 21:31:36 +0100, "E. Naubauer"
<email***@***.com> wrote, quoted or indirectly quoted someone who
said :

>http://java.sun.com/developer/technicalArticles/Media/imagestrategies/index.html

that's great. I used that article to construct a table I call "let me
count the ways" which enumerates all the ways you can create various
Image related objects.

See http://mindprod.com/jgloss/image.html#CREATING
--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.