Technique for loading user defined modules

java-programmer, Technique for loading user defined modules Dear all,

I am writing a program to do data analysis. I want the user to be able to
extend the functionality of the software by adding their own "analysis
modules" but am unsure how to go about this. From the user perspective, I
would like the application to look in a particular directory (or along some
user defined search path) for modules (classes) with the appropriate
behaviour and make these modules (classes) available through the program's
user interface.

It must be possible to complile these modules without access to the source
code for the entire data analysis program.

Can anyone give me some hints on which direction I should go in to handle
this issue?

Thanks in advance,
Per

java-programmer >> Technique for loading user defined modules
"Per E" wrote...

> I am writing a program to do data analysis.
> I want the user to be able to extend the functionality
> of the software by adding their own "analysis modules" but am unsure how
> to go about this.
> From the user perspective, I would like the
> application to look in a particular directory
> (or along some user defined search path) for modules
> (classes) with the appropriate behaviour and make these
> modules (classes) available through the program's user interface.
>
> It must be possible to complile these modules without
> access to the source code for the entire data analysis
> program.
>
> Can anyone give me some hints on which direction I should
> go in to handle this issue?

This sounds pretty much like some kind of plug-in, or add-in technique.
There are lots of possible approaches to this, but in my experience the
easiest path to do this is by Reflection.

A couple of years ago I made a generic ad-hoc database query application,
which worked against any type of database, as long as there was a type 4
driver along. Then I just dropped the driver into a specific directory, and
the user simply added the driver in the applications configuration (via its
GUI).

This sounds pretty similar.

What you then need to define is a public interface in order to make the
plug-ins "pluggable". A simple example could just be:

------------------------------------

package xyz;

public interface PlugIn
{
Collection run(Collection data);
}

------------------------------------

..i.e. that the application provides some data through some Collection, and
returns the result in another Collection of some type, that your application
then presents in a described manner.

This interface doesn't even need to be in source code form, but just as a
bytecode .class-file.

The part of then getting hold of the "plugged in" class, is then easy or
easier. Just see to that the class is on the classpath when you start the
application, and provide the classname in some way:

Class c = Class.forName("MyPlugIn");
PlugIn p = c.newInstance();
Collection outdata = p.run(indata);

This was just an example, as I mentioned that there are several other
alternatives too.

How complex the plug-in interface has to be is up to you, and what you want
the plugins to be able to do.

There could also be a question of using the ClassLoader depending on how
advanced you want to get, and how you package your application.

But I hope this is has provided a start for you.

/// Bjorn A

Inviato da X-Privat.Org - Registrazione gratuita http://www.x-privat.org/join.php

java-programmer >> Technique for loading user defined modules "Per E" <email***@***.com> wrote in
news:UhIcg.1555$email***@***.com:

> Dear all,
>
> I am writing a program to do data analysis. I want the user to be able
> to extend the functionality of the software by adding their own
> "analysis modules" but am unsure how to go about this. From the user
> perspective, I would like the application to look in a particular
> directory (or along some user defined search path) for modules (classes)
> with the appropriate behaviour and make these modules (classes)
> available through the program's user interface.
>
> It must be possible to complile these modules without access to the
> source code for the entire data analysis program.
>
> Can anyone give me some hints on which direction I should go in to
> handle this issue?
<snip>
I suggest that you pick up a copy of

"Java Reflection in Action" (In Action series) by Ira R. Forman and Nate
Forman

Basically, you are going to need to:
1) Compile the source code. Either the user does this, or your data
analysis program does it either via java.lang.Runtime.exec (the referenced
book has code showing how to do this) or by using the built-in comppiler
features of a recent version of Java (1.5? 1.6?).

2) Use reflection to dynamically load the class file from step 1.

I don't think that you do need to provide source code for any part of your
data analysis program. The user's modules will need to implement some
Interface that you define.

Disclaimer: I have read the book and liked it. I have no other connection
with the book, its publisher, its authors, or a bookstore. I have not yet
implemented any of the ideas presented in the book.

--
Ian Shef 805/F6 * These are my personal opinions
Raytheon Company * and not those of my employer.
PO Box 11337 *
Tucson, AZ 85734-1337 *

java-programmer >> Technique for loading user defined modules "Bjorn Abelli" <email***@***.com> wrote:
> This sounds pretty much like some kind of plug-in, or add-in technique.
> There are lots of possible approaches to this, but in my experience the
> easiest path to do this is by Reflection.
[...]
> What you then need to define is a public interface in order to make the
> plug-ins "pluggable". A simple example could just be:
>
> ------------------------------------
>
> package xyz;
>
> public interface PlugIn
> {
> Collection run(Collection data);
> }
>
> ------------------------------------
>
[...]
> The part of then getting hold of the "plugged in" class, is then easy or
> easier. Just see to that the class is on the classpath when you start the
> application, and provide the classname in some way:
>
> Class c = Class.forName("MyPlugIn");
> PlugIn p = c.newInstance();
> Collection outdata = p.run(indata);
>
[...]
Java offers a generic service-registry. Using it can simplifiy your approach
even more. The following code-snippet is enough for loading all Plugin
implementations found in all jar files of your classpath:

import java.util.Iterator;
import javax.imageio.spi.ServiceRegistry;

Class c = Plugin.class; // your interface class
Iterator iter = ServiceRegistry.lookupProviders(c, c.getClassLoader());
while (iter.hasNext()) {
Plugin plugin = (Plugin) iter.next();
... // add plugin to your application's plugin list
}

In order for this to work each jar-file has to specify which implementations
of the Plugin interface it provides. In your case (interface xyz.Plugin) the
jar has to contain a file
/META-INF/services/xyz.Plugin
This file is a plain text file containing the fully qualified class names
(one per line) of the concrete plugin implementations contained in this jar.

See also <http://java.sun.com/.../j2se/1.4.2/docs/guide/jar/jar.html#Service
Provider> for more info about it.

By the way: Java itself uses this mechanism widely (for image decoders,
image encoders, character encodings, sound decoders, ...).

--
"TFritsch$t-online:de".replace(':','.').replace('$','@')

java-programmer >> Technique for loading user defined modules Thomas Fritsch wrote:
> See also <http://java.sun.com/.../j2se/1.4.2/docs/guide/jar/jar.html#Service
> Provider> for more info about it.
Arrgh, I meant
<http://java.sun.com/j2se/1.4.2/docs/guide/jar/jar.html#Service%20Provider>

--
Thomas

java-programmer >> Technique for loading user defined modules
Per E wrote:
> Dear all,
>
> I am writing a program to do data analysis. I want the user to be able to
> extend the functionality of the software by adding their own "analysis
> modules" but am unsure how to go about this. From the user perspective, I
> would like the application to look in a particular directory (or along some
> user defined search path) for modules (classes) with the appropriate
> behaviour and make these modules (classes) available through the program's
> user interface.
>
> It must be possible to complile these modules without access to the source
> code for the entire data analysis program.
>
> Can anyone give me some hints on which direction I should go in to handle
> this issue?
>
> Thanks in advance,
> Per

Just a note, tangential nonetheless, since these deployed plugin jars
are essentially 'hot-deploys' i.e. would not participate via a build
and get bundled with the mainstream, version management and backups
become the responsibility of the users.

java-programmer >> Technique for loading user defined modules Thomas Fritsch wrote:
> [...]
> Java offers a generic service-registry. Using it can simplifiy your approach
> even more. The following code-snippet is enough for loading all Plugin
> implementations found in all jar files of your classpath:

Allow me to be the voice of security here...

There are security concerns here when you are letting other people's
code run within your appplication. It's not that you shouldn't do it,
but there are security concerns to worry about both for your application
and for your user's privacy and security and you should take the steps
to handle it.

While the topic may have been about allowing users to write their own
plug-ins, the fact is that you should consider the possibility of third
party plug-ins being made available for download. Then you have to
consider the possibility of malicious code in those plug-ins.

You should not have the plug-in jars being on your applications initial
class path. Doing that gives the plug-in the same rights as your
application.

You should have the user specify where the plug-in jars are, either as a
list of jars or a directory where the jars are. You can create your own
class loader pointing to the specific jars using URLClassLoader (they
have a URL format for jars, see the Javadocs for JarUrlConnection).

You can then specify what the plug-in is allowed to do and what classes
of your application it can access. By default it should be sandboxed to
not do much at all.

You should have a mechanism to allow a plug-in to get more access, but
only if it is signed, you present the certificate to the user, and ask
them if they want to allow it.

See the seucrity section of the Java Tutorial.

--
Dale King

java-programmer >> Technique for loading user defined modules <Dale King <"DaleWKing [at]gmail [dot] com">> wrote:
> Allow me to be the voice of security here...
>
> There are security concerns here when you are letting other people's
> code run within your appplication. It's not that you shouldn't do it,
> but there are security concerns to worry about both for your application
> and for your user's privacy and security and you should take the steps
> to handle it.

[...]

Incidentally, I wrote the following quite some time (years) ago. I
didn't look over it to see how valid it is, but it expands on some of
these concepts in detail, including ways to handle security managers for
plugins:

> http://www.javamoderator.org/Java%20White%20Papers/05%20-%20Miscellaneous/Using%20Reflection%2C%20Classloaders%2C%20and%20Plug-ins.doc

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

java-programmer >> Technique for loading user defined modules Thank you all for the bits and pieces of information you have supplied. I
think I have a good over all understanding of the problem now and different
ways to solve it.

Thanks again.
Per

"Chris Smith" <email***@***.com> wrote in message
news:email***@***.com...
> <Dale King <"DaleWKing [at]gmail [dot] com">> wrote:
>> Allow me to be the voice of security here...
>>
>> There are security concerns here when you are letting other people's
>> code run within your appplication. It's not that you shouldn't do it,
>> but there are security concerns to worry about both for your application
>> and for your user's privacy and security and you should take the steps
>> to handle it.
>
> [...]
>
> Incidentally, I wrote the following quite some time (years) ago. I
> didn't look over it to see how valid it is, but it expands on some of
> these concepts in detail, including ways to handle security managers for
> plugins:
>
>> http://www.javamoderator.org/Java%20White%20Papers/05%20-%20Miscellaneous/Using%20Reflection%2C%20Classloaders%2C%20and%20Plug-ins.doc
>
> --
> www.designacourse.com
> The Easiest Way To Train Anyone... Anywhere.
>
> Chris Smith - Lead Software Developer/Technical Trainer
> MindIQ Corporation