Original creation date: 31th May 2020
Note: Originally I posted this on my Wordpress Blog (https://1337codersblog.wordpress.com), but I decided to switch to steemit, so I copied it from there.

Improving the plugin loader

Currently I continue my work on the trading system. While this I found out that the original implementation of the plugin loader had 2 problems:

1. The loader writes the source code to disk and then compiles it. Better would be to compile the source code directly in the main memory, to avoid the additional loading time for the harddisk access
2. When the loader loads the same plugin twice, while the source code of the plugin has been updated, the recently loaded object runs the old source code instead of the updated one

To fix the first problem. I had to implement 2 classes that extend SimpleJavaFileObject: JavaStringObject that will provide the source code to the compiler and JavaByteObject that will get the compiler output and stores it as an array of bytes.

private static class JavaStringObject extends SimpleJavaFileObject{
    private final String code;

    public JavaStringObject(String pluginName, String code) {
        super(URI.create(String.format(
                "string:///%s%s",
                pluginName.replace('.','/'),
                Kind.SOURCE.extension
        )), Kind.SOURCE);
        this.code = code;
    }

    @Override
    public CharSequence getCharContent(boolean ignoreEncodingErrors) {
        return code;
    }
}

private static class JavaByteObject extends SimpleJavaFileObject{
    private final ByteArrayOutputStream outputStream;

    public JavaByteObject(String name) {
        super(URI.create(String.format("bytes:///%s%s", name, name.replaceAll("\\.", "/"))), Kind.CLASS);
        this.outputStream = new ByteArrayOutputStream();
    }

    @Override
    public OutputStream openOutputStream() throws IOException {
        return outputStream;
    }

    public byte[] getBytes() {
        return outputStream.toByteArray();
    }
}

I also had to extend ForwardingJavaFileManager which will write the output of the compiler to the JavaByteObject:

public class PluginFileManager extends ForwardingJavaFileManager<StandardJavaFileManager> {
    private final JavaFileObject javaFileObject;

    public PluginFileManager(StandardJavaFileManager fileManager, JavaFileObject javaFileObject) {
        super(fileManager);
        this.javaFileObject = javaFileObject;
    }

    @Override
    public JavaFileObject getJavaFileForOutput(Location location, String className, JavaFileObject.Kind kind, FileObject sibling) throws IOException {
        return javaFileObject;
    }
}

Now I could call the compiler and run the compilation:

public static byte[] compile(String pluginName, String sourceCode) throws CompilationFailedException {
    JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
    DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<>();
    JavaByteObject javaByteObject = new JavaByteObject(pluginName);
    PluginFileManager pluginFileManager = new PluginFileManager(compiler.getStandardFileManager(diagnostics, null, null), javaByteObject);
    List<String> options = Collections.emptyList();
    JavaCompiler.CompilationTask compilationTask = compiler.getTask(
            null, pluginFileManager, diagnostics,
            options, null, () -> {
                JavaFileObject javaFileObject = new JavaStringObject(pluginName, sourceCode);
                return Collections.singletonList(javaFileObject).iterator();
            });
    boolean compilationSuccessful = compilationTask.call();
    if (!compilationSuccessful){
        String message = diagnostics.getDiagnostics().stream().map(new Function<Diagnostic<? extends JavaFileObject>, String>() {
            @Override
            public String apply(Diagnostic<? extends JavaFileObject> diagnostic) {
                return diagnostic.toString();
            }
        }).collect(Collectors.joining());
        throw new CompilationFailedException(String.format("Failed to compile class '%s':\n%s", pluginName, message), sourceCode);
    }
    return javaByteObject.getBytes();
}

CompilationFailedException is a simple Exception class that I implemented for indicating compilation failures to the caller:

public static class CompilationFailedException extends Exception{
    private final String code;

    public CompilationFailedException(String message, String code) {
        super(message);
        this.code = code;
    }

    public String getCode() {
        return code;
    }
}

Now I solved the second problem. For this I had to implement my own class loader that will be used for loading and finding the compiled classes:

public class PluginClassLoader extends ClassLoader{
    private static final Map<String, byte[]> classes = new HashMap<>();

    public PluginClassLoader() {
        super(PluginClassLoader.class.getClassLoader());
    }

    @Override
    public Class<?> findClass(String fullQualifiedClassName) throws ClassNotFoundException {
        if (!classes.containsKey(fullQualifiedClassName)){
            throw new ClassNotFoundException(String.format("Did not find class '%s'", fullQualifiedClassName));
        }
        Class<?> compiledClass = defineClass(fullQualifiedClassName, classes.get(fullQualifiedClassName), 0, classes.get(fullQualifiedClassName).length);
        return compiledClass;
    }

    @Override
    public Class<?> loadClass(String fullQualifiedClassName) throws ClassNotFoundException {
        if (classes.containsKey(fullQualifiedClassName)){
            return findClass(fullQualifiedClassName);
        }
        ClassLoader defaultLoader = Thread.currentThread().getContextClassLoader();
        Class<?> loadedClass = defaultLoader.loadClass(fullQualifiedClassName);
        return loadedClass;
    }

    public void putClassCode(String fullQualifiedClassName, byte[] compiledClassCode){
        assert fullQualifiedClassName != null;
        assert compiledClassCode != null;
        classes.put(fullQualifiedClassName, compiledClassCode);
    }
}

The class loader will look for classes within its internal Map that contains the full qualified class names as keys and their Java Bytecode as a Byte array. When the class loader does not have a requested class it will delegate to the default class loader of the current Thread:

public class PluginClassLoader extends ClassLoader{
    private static final Map<String, byte[]> classes = new HashMap<>();

    public PluginClassLoader() {
        super(PluginClassLoader.class.getClassLoader());
    }

    @Override
    public Class<?> findClass(String fullQualifiedClassName) throws ClassNotFoundException {
        if (!classes.containsKey(fullQualifiedClassName)){
            throw new ClassNotFoundException(String.format("Did not find class '%s'", fullQualifiedClassName));
        }
        Class<?> compiledClass = defineClass(fullQualifiedClassName, classes.get(fullQualifiedClassName), 0, classes.get(fullQualifiedClassName).length);
        return compiledClass;
    }

    @Override
    public Class<?> loadClass(String fullQualifiedClassName) throws ClassNotFoundException {
        if (classes.containsKey(fullQualifiedClassName)){
            return findClass(fullQualifiedClassName);
        }
        ClassLoader defaultLoader = Thread.currentThread().getContextClassLoader();
        Class<?> loadedClass = defaultLoader.loadClass(fullQualifiedClassName);
        return loadedClass;
    }

    public void putClassCode(String fullQualifiedClassName, byte[] compiledClassCode){
        assert fullQualifiedClassName != null;
        assert compiledClassCode != null;
        classes.put(fullQualifiedClassName, compiledClassCode);
    }
}

The “putClassCode” method will be called by the plugin loader for notifying the class loader about newly compiled classes.

Now the “load” method of the plugin loader can be implemented:

public <OBJECT> OBJECT load(String plugin, Class<OBJECT> pluginInterface) throws LoadingException {
    try {
        ClassInfo classInfo = parseClassInfo(plugin);
        byte[] compile = PluginCompiler.compile(classInfo.getFullQualifiedClassName(), plugin);
        PluginClassLoader pluginClassLoader = new PluginClassLoader();
        pluginClassLoader.putClassCode(classInfo.getFullQualifiedClassName(), compile);

        Class<?> pluginClass = pluginClassLoader.findClass(classInfo.getFullQualifiedClassName());
        if (!pluginInterface.isAssignableFrom(pluginClass)){
            throw new LoadingException(String.format("The compiled class is no implementation or subclass of %s", pluginInterface.getCanonicalName()));
        }
        Object pluginInstance = pluginClass.getConstructor().newInstance();
        //noinspection unchecked
        return (OBJECT) pluginInstance;
    } catch (ClassNotFoundException | NoSuchMethodException | InstantiationException
            | IllegalAccessException | InvocationTargetException | PluginCompiler.CompilationFailedException e) {
        throw new LoadingException(e);
    }
}

Before the plugin can be compiled, its class name and package name have to be parsed from the provided source code:

private ClassInfo parseClassInfo(String plugin) throws LoadingException {
    JavaParser javaParser = new JavaParser();
    ParseResult<CompilationUnit> parseResult = javaParser.parse(plugin);
    if (!parseResult.isSuccessful()){
        StringBuilder message = new StringBuilder();
        message.append("Parsing source file failed:\n");
        parseResult.getProblems().forEach(problem -> message.append(String.format("%s\n", problem.getVerboseMessage())));
        throw new LoadingException(message.toString());
    }
    assert parseResult.getResult().isPresent();
    final Set<TypeDeclaration<?>> publicTopLevelClasses = parseResult.getResult().get().getTypes().stream()
            .filter(TypeDeclaration::isTopLevelType)
            .filter(typeDeclaration -> !typeDeclaration.getModifiers().contains(Modifier.privateModifier()))
            .filter(typeDeclaration -> !typeDeclaration.getModifiers().contains(Modifier.protectedModifier()))
            .collect(Collectors.toSet());
    if (publicTopLevelClasses.isEmpty()){
        throw new LoadingException("Did not found a public top level class in the source code");
    }
    if (publicTopLevelClasses.size() > 1){
        throw new LoadingException("Found multiple public top level classes in the source code");
    }
    String className = publicTopLevelClasses.iterator().next().getName().asString();

    String packageName = null;
    Optional<Node> parentNode = publicTopLevelClasses.iterator().next().getParentNode();
    if (parentNode.isPresent() || (parentNode.get() instanceof CompilationUnit)){
        CompilationUnit compilationUnit = (CompilationUnit) parentNode.get();
        Optional<PackageDeclaration> packageDeclaration = compilationUnit.getPackageDeclaration();
        if (packageDeclaration.isPresent()){
            packageName = packageDeclaration.get().getNameAsString();
        }
    }
    return new ClassInfo(className, packageName);
}

ClassInfo is a simple data holder that contains the package name and the class name which are defined by the given source code:

private static class ClassInfo{
    private final String className;
    private final String packageName;

    private ClassInfo(String className, String packageName) {
        this.className = className;
        this.packageName = packageName;
    }

    public String getFullQualifiedClassName(){
        String packagePath = "";
        if (packageName != null){
            packagePath = String.format("%s.", packageName);
        }
        return String.format("%s%s", packagePath, className);
    }
}

Now the 2 problems I addressed in the beginning have been fixed. You can find the complete source code in my Gitlab repository.