Saturday, August 29, 2009

Mondrian - dynamic schema and datasources xml

A typical catalog entry in datasources.xml file looks like;

<catalog name="FoodMart">
<datasourceinfo>
Provider=mondrian;Jdbc=jdbc:mysql://localhost/mydb;JdbcUser=root;JdbcPassword=root;
JdbcDrivers=com.mysql.jdbc.Driver;
</datasourceinfo>
<definition>/WEB-INF/queries/Foodmart.xml</definition>
</catalog>


Instead of static schema definition file (Foodmart.xml), add a schema processor to define the schema dynamically. In such case, <definition> contains some non-existent file path. Besides this 'dummy' path, all standard and custom parameters (p1, p2) are accessible in the schema processor class.

<catalog name="cat1">
<datasourceinfo>
Provider=mondrian;Jdbc=jdbc:mysql://localhost/mydb;JdbcUser=root;JdbcPassword=root;
JdbcDrivers=com.mysql.jdbc.Driver;DynamicSchemaProcessor=com.xyz.MySchemaProcessor;
p1=value1;p2=value2
</datasourceinfo>
<definition>dummy.xml</definition>
</catalog>


If every department in your organization needs a separate catalog entry, for security and separation, either all those catalog entries can be defined static or generate the datasources.xml dynamically during the system start-up. While generating the content dynamically, make sure to provide unique catalog names and <definition> paths.

<catalog name="cat1">
<datasourceinfo>
Provider=mondrian;Jdbc=jdbc:mysql://localhost/mydb;JdbcUser=root;JdbcPassword=root;
JdbcDrivers=com.mysql.jdbc.Driver;DynamicSchemaProcessor=com.xyz.MySchemaProcessor;
p1=value1;p2=value2
</datasourceinfo>
<definition>dummy1.xml</definition>
</catalog>
<catalog name="cat2">
<datasourceinfo>
Provider=mondrian;Jdbc=jdbc:mysql://localhost/differentdb;JdbcUser=root;
JdbcPassword=root; JdbcDrivers=com.mysql.jdbc.Driver;
DynamicSchemaProcessor=com.xyz.MySchemaProcessor; p1=value111;p2=value222
</datasourceinfo>
<definition>dummy2.xml</definition>
</catalog>

Saturday, March 11, 2006

General Object Caching

http://www.awprofessional.com/articles/article.asp?p=353736&seqNum=5
Second level caching in ORM (Object Relational Mapping) APIs.

http://www.javaworld.com/javaworld/jw-07-2001/jw-0720-cache_p.html
General object caching.

Object caching samples:
----------------------------------------

The two main classes needed are Cacheable.java and CacheManager.java.
ContentLoader.java is a convenience class (helper class) that is used at
the business logic level.

PageFragment.java and GroovyScript.java are objects that implement the Cacheable interface. Example code provided below. AThese objects are self-reloading. i.e when they expire they reload themselves. Reloading could have been left to the CacheManager itself, however, these objects have special semantics associated with
them when they change. In the case of the GroovyScript, if the resource (the script file) has been modified, then the cached object is not the groovy script file, but the GroovyScriptObject (compiled code). Similarly, when the PageFragment is modified, it is not enough that the file be reloaded, the DOM associated with the file has to be cached. Hence the objects are self-loading. In other words, these object never expire in the sense of going stale, they simply reload themselves and
they know how to reload themselves.

--------------------------------------
Cacheable.java
--------------------------------------
package com.radientedge.utils;

public interface Cacheable {
/**
* By requiring all objects to determine their own expirations, the
* algorithm is abstracted from the caching service, thereby providing maximum
* flexibility since each object can adopt a different expiration strategy.
*/
public boolean isExpired();

/**
* This method will ensure that the caching service is not responsible for
* uniquely identifying objects placed in the cache.
*/
public Object getIdentifier();
}

--------------------------------------
CacheManager.java
--------------------------------------
package com.radientedge.utils;


/**
* Title: Caching
* Description:
* Copyright: Copyright (c) 2001
* Company: JavaWorld
* Filename: CacheManager.java
* @author Jonathan Lurie
* @version 1.0
*/

public class CacheManager {

/* This is the HashMap that contains all objects in the cache. */
private static java.util.HashMap cacheHashMap = new java.util.HashMap();

/* This object acts as a semaphore, which protects the HashMap */
/* RESERVED FOR FUTURE USE private static Object lock = new Object(); */

static {
try {
/* Create background thread, which will be responsible for purging expired items. */
Thread cleaner = new Thread(
new Runnable()
{
/* The default time the thread should sleep between scans.
The sleep method takes in a millisecond value so 5000 = 5 Seconds.
*/
//static ;
int milliSecondSleepTime = 5000;

public void run()
{
try
{
/* Sets up an infinite loop. The thread will continue looping forever. */
while (true)
{
//System.out.println("  --> ThreadCleanerUpper Scanning For Expired Objects...");

/* Get the set of all keys that are in cache. These are the unique identifiers */
java.util.Set keySet = cacheHashMap.keySet();

/* An iterator is used to move through the Keyset */
java.util.Iterator keys = keySet.iterator();

/* Sets up a loop that will iterate through each key in the KeySet */
while(keys.hasNext())
{
/* Get the individual key. We need to hold on to this key in case it needs to be removed */
Object key = keys.next();

/* Get the cacheable object associated with the key inside the cache */
Cacheable value = (Cacheable)cacheHashMap.get(key);

/* Is the cacheable object expired? */
if (value.isExpired())
{
/* Yes it's expired! Remove it from the cache */
cacheHashMap.remove(key);
System.out.println(" --> CacheManager removed expired Page '" + key + "'");

//System.out.println("  --> ThreadCleanerUpper Running. Found an Expired Object in the Cache.");
}
}

/*************************************************************************
******* A LRU (least-recently used) or LFU (least-frequently used) *****
******* would best be inserted here                                *****
************************************************************************
*/

/* Puts the thread to sleep. Don't need to check it continuously */
Thread.sleep(this.milliSecondSleepTime);
}
}
catch (Exception e)
{
e.printStackTrace();
}
return;
} /* End run method */
}); /* End class definition and close new thread definition */

// Sets the thread's priority to the minimum value.
cleaner.setPriority(Thread.MIN_PRIORITY);

// Starts the thread.
cleaner.start();
}
catch(Exception e)
{
System.out.println("CacheManager.Static Block: " + e);
}
} /* End static block */

public CacheManager()
{
}

public static void putCache(Cacheable object)
{
// Remember if the HashMap previously contains a mapping for the key, the old value
// will be replaced. This is valid functioning.
cacheHashMap.put(object.getIdentifier(), object);
}

public static Cacheable getCache(Object identifier)
{
//synchronized (lock) // UNCOMMENT LINE XXX
//{ // UNCOMMENT LINE XXX
Cacheable object = (Cacheable)cacheHashMap.get(identifier);

// The code to create the object would be placed here.

//} // UNCOMMENT LINE XXX

if (object == null)
return null;

if (object.isExpired())
{
cacheHashMap.remove(identifier);
return null;
}
else
{
return object;
}
}
// +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
}

--------------------------------------
ContentLoader.java
--------------------------------------
package com.radientedge.utils;

import com.radientedge.busobjects.GroovyScript;
import com.radientedge.busobjects.PageFragment;
import com.radientedge.busobjects.StyleSheet;
import com.radientedge.busobjects.WebPage;

import org.apache.log4j.Category;

import org.jdom.Document;
import org.jdom.Element;
import org.jdom.JDOMException;

import org.jdom.input.SAXBuilder;

import java.io.File;
import java.io.IOException;

import java.util.Iterator;
import java.util.List;


public class ContentLoader {
/** Default SAX Driver class to use */
private static final String DEFAULT_SAX_DRIVER_CLASS = "org.apache.xerces.parsers.SAXParser";

protected Category log = Category.getInstance(this.getClass().getName());

private String contentDescFileName;
private String basePath;

public ContentLoader(String bp, String cdfn) {
this.contentDescFileName = cdfn;
this.basePath = bp;
}

public void setContentDescFileName(String cdfn) {
this.contentDescFileName = cdfn;
}

public String getContentDescFileName() {
return this.contentDescFileName;
}

public void loadPageFragment(String key) throws IOException {
String inputFilePath = basePath + "templates/" + key + ".xml";
PageFragment pf = new PageFragment(inputFilePath, key);
CacheManager.putCache(pf);     
}

public PageFragment getPageFragment(String key) throws IOException {
String inputFilePath = basePath + "templates/" + key + ".xml";
PageFragment pf = new PageFragment(inputFilePath, key);
return pf;
}

public GroovyScript getGroovyScript(String key) throws IOException {
String inputFilePath = basePath + "scripts/" + key + ".groovy";
GroovyScript gs = new GroovyScript(inputFilePath, key);
return gs;
}

public void createCache() throws IOException, JDOMException {
// Build the JDOM Document
String saxDriverClass = DEFAULT_SAX_DRIVER_CLASS;

SAXBuilder builder = new SAXBuilder(saxDriverClass);
File inputFile = new File(this.basePath + this.contentDescFileName);

Document doc = builder.build(inputFile);

// Get the root element
Element root = doc.getRootElement();

// Find all Content Elements
List clist = root.getChildren("Content");
Iterator i = clist.iterator();

while (i.hasNext()) {
Element content = (Element) i.next();
String key = content.getChild("Key").getTextTrim();
String filename = content.getChild("File").getTextTrim();
String type = content.getChild("Type").getTextTrim();

log.info(" --> Caching [" + type + "] " + filename + " with key :'" + key + "'");

String inputFilePath = basePath + filename;

if ("WebPage".equals(type)) {
WebPage WP = new WebPage(inputFilePath, key);
CacheManager.putCache(WP);
} else if ("PageFragment".equals(type)) {
PageFragment PF = new PageFragment(inputFilePath, key);
CacheManager.putCache(PF);
} else if ("StyleSheet".equals(type)) {
StyleSheet SS = new StyleSheet(inputFilePath, key);
CacheManager.putCache(SS);
} else if ("GroovyScript".equals(type)) {
GroovyScript GS = new GroovyScript(inputFilePath, key);
CacheManager.putCache(GS);                
} else {
System.out.println("CacheLoader error recognizing file type " + type);
}
}
}
}

--------------------------------------
GroovyScript.java
--------------------------------------
package com.radientedge.busobjects;

import groovy.lang.GroovyClassLoader;
import groovy.lang.GroovyObject;

import java.io.File;
import java.io.IOException;

import org.codehaus.groovy.syntax.SyntaxException;

import com.radientedge.utils.Cacheable;

public class GroovyScript implements Cacheable {

private String       identifier;
private String       sourceFileName;
private long         lastModifiedTime;
private GroovyObject gsObject;
private boolean      acquired;

public GroovyScript(String id) {
identifier = id;
}

public GroovyScript(String inputFileName, String id)
throws IOException {
this.identifier = id;
this.sourceFileName = inputFileName;
this.load();
}

public void load() {
File inputFile = new File(this.sourceFileName);

try {
if (inputFile != null) {
this.lastModifiedTime = inputFile.lastModified();

GroovyClassLoader loader = new GroovyClassLoader(GroovyScript.class.getClassLoader());
Class groovyClass = loader.parseClass(new File(this.sourceFileName));
this.gsObject = (GroovyObject) groovyClass.newInstance();

this.acquired = false;                
}
} catch (IOException e) {
System.out.println("Exception occured while reloading " + this.sourceFileName);
System.out.println("  exception : " + e);            
} catch (IllegalAccessException e) {
System.out.println("Exception occured while reloading " + this.sourceFileName);
System.out.println("  exception : " + e);
} catch (InstantiationException e) {
System.out.println("Exception occured while reloading " + this.sourceFileName);
System.out.println("  exception : " + e);    
} catch (SyntaxException e) {
System.out.println("Exception occured while reloading " + this.sourceFileName);
System.out.println("  exception : " + e);            
}
}

public boolean isExpired() {
boolean expired = false;
File sfile = new File(this.sourceFileName);
long lmt = sfile.lastModified();

if (lmt > this.lastModifiedTime) {
// reload the file. the script has been touched...
this.gsObject = null;
this.load();
expired = true;
}

// we return false as the CacheManager will unload this object if we return true
return false;
}

public boolean isAcquired() {
return this.acquired;
}

public void setIdentifier(String id) {
this.identifier = id;
}

public Object getIdentifier() {
return identifier;
}

public GroovyObject getGsObject() {
this.acquired = true;
return this.gsObject;
}

}

--------------------------------------
PageFragment.java
--------------------------------------
package com.radientedge.busobjects;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import org.dom4j.Document;
import org.dom4j.DocumentException;
import org.dom4j.io.SAXReader;

import com.radientedge.utils.Cacheable;

public class PageFragment implements Cacheable {
private String   identifier;
private String   sourceFileName;
private long     lastModifiedTime;
private Document dom;

public PageFragment(String id) {
identifier = id;
}

public PageFragment(String inputFileName, String id) throws IOException {
this.identifier = id;
this.sourceFileName = inputFileName;
this.load();
}

public void load() {
File inputFile = new File(this.sourceFileName);


try {
if (inputFile != null) {
FileReader fr = new FileReader(inputFile);
this.lastModifiedTime = inputFile.lastModified();
SAXReader reader = new SAXReader();
//[rr]this.dom = reader.read(inputFile); // resin-3.0.6 does not like this. Strangeness!
this.dom = reader.read(fr);
}
} catch (FileNotFoundException e) { //MalformedURLException e) {
System.out.println("Exception occured while reloading " + this.sourceFileName);
System.out.println("  exception : " + e);   
} catch (DocumentException e) {
System.out.println("Exception occured while reloading " + this.sourceFileName);
System.out.println("  exception : " + e);   
} 
}

public boolean isExpired() {
File sfile = new File(this.sourceFileName);
long lmt = sfile.lastModified();

if (lmt > this.lastModifiedTime) {
// reload the file. the page has been touched...
this.dom = null;
this.load();
}

return false;
}

public void setIdentifier(String id) {
this.identifier = id;
}

public Object getIdentifier() {
return identifier;
}

public Document getDom() {
return this.dom;
}
}

--------------------------------------