注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

和申的个人主页

专注于java开发,1985wanggang

 
 
 

日志

 
 

使用自定义的metaclasses来优化Groovy提高性能-Using custom Groovy metaclasses to boost performance  

2013-07-03 15:39:09|  分类: groovy |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |
groovy的方法查找策略,默认先从this.getClass的metaClass里找,再从上下文里找,这里修改策略,反过来找。
当把一个java method放入groovy后,会自动被包装为org.codehaus.groovy.runtime.MethodClosure,如果本身就是java类,则无包装替换metaClass见这里:http://www.jroller.com/melix/entry/using_custom_groovy_metaclasses_to,具体代码见:groovy.lang.
 MetaClassRegistry<br />

Groovy is a dynamic language by nature. This means, in Groovy, that method dispatches are controlled by metaclasses. This is really powerful, and allows nice tricks like the "method missing" or "property missing" calls that are used in many DSLs. The metaclass dictates what behaviour is expected whenever a call to a property or a method is made.

The "problem" with this is that method execution paths may be really long. Though Groovy uses call site caching to improve performance, the chain of invocation to call a method in Groovy is much, much longer than in plain Java. The only reason why this chain is long is to allow that dynamic behaviour.

Now, when performance is critical and that you still want to use Groovy as a DSL, there are not so many things you can do to improve performance :

  • coding critical parts in Java and using mixed compilation to call Java classes
  • use profilers to hunt down slow parts

Today, I'll add a nice one that I've just experienced : using custom metaclasses.

The explanation

To simplify, almost every method call you make in Groovy will end up beeing dispatched by the class called MetaClassImpl. This class manages a wide variety of method calls or property access. It deals with closures as well as "regular objects". We'll take, as an example, the body of the MetaClassImpl#invokeMethod(Class sender, Object object, String methodName, Object[] originalArguments, boolean isCallToSuper, boolean fromInsideClass) method :
checkInitalised();
       
if (object == null) {
           
throw new NullPointerException("Cannot invoke method: " + methodName + " on null object");
       
}

       
final Object[] arguments = originalArguments == null ? EMPTY_ARGUMENTS : originalArguments;
//        final Class[] argClasses = MetaClassHelper.convertToTypeArray(arguments);
//
//        unwrap(arguments);

       
MetaMethod method = getMethodWithCaching(sender, methodName, arguments, isCallToSuper);
       
MetaClassHelper.unwrap(arguments);

       
if (method == null)
            method
= tryListParamMetaMethod(sender, methodName, isCallToSuper, arguments);

       
final boolean isClosure = object instanceof Closure;
       
if (isClosure) {
           
final Closure closure = (Closure) object;

           
final Object owner = closure.getOwner();

           
if (CLOSURE_CALL_METHOD.equals(methodName) || CLOSURE_DO_CALL_METHOD.equals(methodName)) {
               
final Class objectClass = object.getClass();
               
if (objectClass == MethodClosure.class) {
                   
final MethodClosure mc = (MethodClosure) object;
                    methodName
= mc.getMethod();
                   
final Class ownerClass = owner instanceof Class ? (Class) owner : owner.getClass();
                   
final MetaClass ownerMetaClass = registry.getMetaClass(ownerClass);
                   
return ownerMetaClass.invokeMethod(ownerClass, owner, methodName, arguments, false, false);
               
} else if (objectClass == CurriedClosure.class) {
                   
final CurriedClosure cc = (CurriedClosure) object;
                   
// change the arguments for an uncurried call
                   
final Object[] curriedArguments = cc.getUncurriedArguments(arguments);
                   
final Class ownerClass = owner instanceof Class ? (Class) owner : owner.getClass();
                   
final MetaClass ownerMetaClass = registry.getMetaClass(ownerClass);
                   
return ownerMetaClass.invokeMethod(owner, methodName, curriedArguments);
               
}
               
if (method==null) invokeMissingMethod(object,methodName,arguments);
           
} else if (CLOSURE_CURRY_METHOD.equals(methodName)) {
               
return closure.curry(arguments);
           
}

           
final Object delegate = closure.getDelegate();
           
final boolean isClosureNotOwner = owner != closure;
           
final int resolveStrategy = closure.getResolveStrategy();

           
final Class[] argClasses = MetaClassHelper.convertToTypeArray(arguments);

           
switch (resolveStrategy) {
               
case Closure.TO_SELF:
                    method
= closure.getMetaClass().pickMethod(methodName, argClasses);
                   
if (method != null) return method.invoke(closure, arguments);
                   
break;
               
case Closure.DELEGATE_ONLY:
                   
if (method == null && delegate != closure && delegate != null) {
                       
MetaClass delegateMetaClass = lookupObjectMetaClass(delegate);
                        method
= delegateMetaClass.pickMethod(methodName, argClasses);
                       
if (method != null)
                           
return delegateMetaClass.invokeMethod(delegate, methodName, originalArguments);
                       
else if (delegate != closure && (delegate instanceof GroovyObject)) {
                           
return invokeMethodOnGroovyObject(methodName, originalArguments, delegate);
                       
}
                   
}
                   
break;
               
case Closure.OWNER_ONLY:
                   
if (method == null && owner != closure) {
                       
MetaClass ownerMetaClass = lookupObjectMetaClass(owner);
                       
return ownerMetaClass.invokeMethod(owner, methodName, originalArguments);
                   
}
                   
break;
               
case Closure.DELEGATE_FIRST:
                   
if (method == null && delegate != closure && delegate != null) {
                       
MetaClass delegateMetaClass = lookupObjectMetaClass(delegate);
                        method
= delegateMetaClass.pickMethod(methodName, argClasses);
                       
if (method != null)
                           
return delegateMetaClass.invokeMethod(delegate, methodName, originalArguments);
                   
}
                   
if (method == null && owner != closure) {
                       
MetaClass ownerMetaClass = lookupObjectMetaClass(owner);
                        method
= ownerMetaClass.pickMethod(methodName, argClasses);
                       
if (method != null) return ownerMetaClass.invokeMethod(owner, methodName, originalArguments);
                   
}
                   
if (method == null && resolveStrategy != Closure.TO_SELF) {
                       
// still no methods found, test if delegate or owner are GroovyObjects
                       
// and invoke the method on them if so.
                       
MissingMethodException last = null;
                       
if (delegate != closure && (delegate instanceof GroovyObject)) {
                           
try {
                               
return invokeMethodOnGroovyObject(methodName, originalArguments, delegate);
                           
} catch (MissingMethodException mme) {
                               
if (last == null) last = mme;
                           
}
                       
}
                       
if (isClosureNotOwner && (owner instanceof GroovyObject)) {
                           
try {
                               
return invokeMethodOnGroovyObject(methodName, originalArguments, owner);
                           
} catch (MissingMethodException mme) {
                                last
= mme;
                           
}
                       
}
                       
if (last != null) return invokeMissingMethod(object, methodName, originalArguments, last, isCallToSuper);
                   
}

                   
break;
               
default:
                   
if (method == null && owner != closure) {
                       
MetaClass ownerMetaClass = lookupObjectMetaClass(owner);
                        method
= ownerMetaClass.pickMethod(methodName, argClasses);
                       
if (method != null) return ownerMetaClass.invokeMethod(owner, methodName, originalArguments);
                   
}
                   
if (method == null && delegate != closure && delegate != null) {
                       
MetaClass delegateMetaClass = lookupObjectMetaClass(delegate);
                        method
= delegateMetaClass.pickMethod(methodName, argClasses);
                       
if (method != null)
                           
return delegateMetaClass.invokeMethod(delegate, methodName, originalArguments);
                   
}
                   
if (method == null && resolveStrategy != Closure.TO_SELF) {
                       
// still no methods found, test if delegate or owner are GroovyObjects
                       
// and invoke the method on them if so.
                       
MissingMethodException last = null;
                       
if (isClosureNotOwner && (owner instanceof GroovyObject)) {
                           
try {
                               
return invokeMethodOnGroovyObject(methodName, originalArguments, owner);
                           
} catch (MissingMethodException mme) {
                               
if (methodName.equals(mme.getMethod())) {
                                   
if (last == null) last = mme;
                               
} else {
                                   
throw mme;
                               
}
                           
}
                           
catch (InvokerInvocationException iie) {
                               
if (iie.getCause() instanceof MissingMethodException) {
                                   
MissingMethodException mme = (MissingMethodException) iie.getCause();
                                   
if (methodName.equals(mme.getMethod())) {
                                       
if (last == null) last = mme;
                                   
} else {
                                       
throw iie;
                                   
}
                               
}
                               
else
                                 
throw iie;
                           
}
                       
}
                       
if (delegate != closure && (delegate instanceof GroovyObject)) {
                           
try {
                               
return invokeMethodOnGroovyObject(methodName, originalArguments, delegate);
                           
} catch (MissingMethodException mme) {
                                last
= mme;
                           
}
                           
catch (InvokerInvocationException iie) {
                               
if (iie.getCause() instanceof MissingMethodException) {
                                    last
= (MissingMethodException) iie.getCause();
                               
}
                               
else
                                 
throw iie;
                           
}
                       
}
                       
if (last != null) return invokeMissingMethod(object, methodName, originalArguments, last, isCallToSuper);
                   
}
           
}
       
}

       
if (method != null) {
           
return method.doMethodInvoke(object, arguments);
       
} else {
           
return invokePropertyOrMissing(object, methodName, originalArguments, fromInsideClass, isCallToSuper);
       
}

Ok, before you get a headache, you'll just focus on one thing : most of this code deals with the Closure case. What if your class is not a closure ? I guess that's true for about 90% of method calls. For example, in the following code :

def myObject = new MySuperFastJavaObject()
myObject
.mySuperFastMethod()
Here, you just want the method call to behave exactly as if it were made from pure Java. If you don't write a custom metaclass, the metaclass that will be used by Groovy will dispatch your method call through the upper algorithm. You have understood that there's no need to deal with the closure case here. So, you could write your own metaclass that removes everything from the closure case in the invokeMethod method :
package groovy.runtime.metaclass.com.mypackage;

public class MySuperFastJavaObjectMetaClass extends MetaClassImpl {
 
...
       
@Override
       
public Object invokeMethod(Class sender, Object object, String methodName, Object[] originalArguments, boolean isCallToSuper, boolean fromInsideClass) {
                checkInitalised
();
               
if (object == null) {
                       
throw new NullPointerException("Cannot invoke method: " + methodName + " on null object");
               
}

               
final Object[] arguments = originalArguments == null ? EMPTY_ARGUMENTS : originalArguments;
               
MetaMethod method = getMethodWithCaching(sender, methodName, arguments, isCallToSuper);
               
MetaClassHelper.unwrap(arguments);

               
if (method != null) {
                       
return method.doMethodInvoke(object, arguments);
               
} else {
                       
return invokeMissingMethod(object, methodName, arguments);
               
}
       
}
}

Several things to notice :

  • usage of the groovy.runtime.metaclass prefix in your package will guarantee that Groovy will automatically load your metaclass and assign it to your class
  • the name of your metaclass is the name of your class plus the MetaClass suffix
  • overriding some methods only can lead to huge performance improvements

Now, say your object doesn't require dynamic behaviour like methodMissing. Why would you end up calling theinvokeMissingMethod method ? If you take a closer look at this method, you'll notice that the default implementation does many things, and, in your case, will always fail since you don't have defined any methodMissing method. That's really a waste of time. So you can directly replace it with a missing method exception :

package groovy.runtime.metaclass.com.mypackage;

public class MySuperFastJavaObjectMetaClass extends MetaClassImpl {
 
...
       
@Override
       
public Object invokeMethod(Class sender, Object object, String methodName, Object[] originalArguments, boolean isCallToSuper, boolean fromInsideClass) {
                checkInitalised
();
               
if (object == null) {
                       
throw new NullPointerException("Cannot invoke method: " + methodName + " on null object");
               
}

               
final Object[] arguments = originalArguments == null ? EMPTY_ARGUMENTS : originalArguments;
               
MetaMethod method = getMethodWithCaching(sender, methodName, arguments, isCallToSuper);
               
MetaClassHelper.unwrap(arguments);

               
if (method != null) {
                       
return method.doMethodInvoke(object, arguments);
               
} else {
                       
throw new new MissingMethodException(methodName, sender, arguments);
               
}
       
}
}

Note that I also had the opposite case : if I were on a missing method, the target method in my Java class was always the same. So I could hijack the meta-object protocol so that it doesn't try a tons of things before dispatching to always the same method missing implementation :

@Override
       
public Object invokeMissingProperty(final Object instance, final String propertyName, final Object optionalValue, final boolean isGetter) {
               
Layer layer = (Layer) instance; // Layer metaclass, it's ALWAYS a Layer
               
return layer.java_propertyMissing(propertyName);
       
}

Another case was dealing with the following bug (?) : http://jira.codehaus.org/browse/GROOVY-4495. In a specific case, static method invocation always lead to the longest execution path possible. However, in my case, the class used is an utility class which inherits another, the two written in pure Java, and for which every method is static. In another words, it's a toolbox. Why would I want dynamic method dispatching on this class ? There's no reason. So, I implemented my own metaclass which directly delegated calls to the appropriate methods AND worked around the bug by adding the method defined by the parent class in the method cache.

Expected improvements

So, doing this, how much improvement can you expect ? Well, I'll talk about my case. As I've already said, the critical parts of code are all written in pure Java. So most of Groovy is used as a DSL. After having added a single metaclass, I reached a 10% improvement in execution time. I was so surprised by the result that I added a few metaclasses, and I reached up to 25% execution time improvement depending on the application. The only thing I can say is that it is really important to take this in consideration when you want to make the best of Groovy. Try it yourself !

Permalink Comments[3]

Comments:

Overall, great post. I know that Guillaume Laforge is watching: http://twitter.com/#!/glaforge/status/2475741996392448. I hope he notes this and uses this information to speed up groovy. Perhaps some sort of cache of "object instanceof Closure" is possible. 

You missed something interesting: 

if (method == null) 
method = tryListParamMetaMethod(sender, methodName, isCallToSuper, arguments); 

That takes a list and converts an args[0] List to an array of args. 

1-2 instanceofs per method call can really add up. Guillaume, a fix sounds like a potential big win. Do you think that a fix is possible?

Posted by Solomon Duskis on November 10, 2010 at 11:28 PM CET #

Why not use groovy++ then? It will give you the same performance benefit - I assume - with much less hassle.

Posted by Johannes Link on November 11, 2010 at 09:03 AM CET #

@Solomon : Yes, there are tons of conversions like this in the process. I'm sure there are ways of improving it, but with the new meta-object protocol planned for Groovy 2.0, I'm not sure it's worth working on it. 

@Johannes : there are multiple reasons why not using Groovy++. Though it's an interesting project, there are a few drawbacks. First, Alex Tkachman himself says it's not ready for production (it's still not in 1.0). Moreover, there are differences in the semantics of Groovy and Groovy++ code that will be hard to explain/understand. People already deal with the differences between Java and Groovy, explaining them the differences between Groovy and Groovy++ would be complicated. Eventually, Groovy++ requires that you update your code to add explicit types whenever possible, and add @Typed annotations. While this is acceptable for "code", this is not that acceptable for DSLs. 

BTW, there's an interesting discussion here regarding Groovy++ : http://groovy.329449.n5.nabble.com/Relationship-between-Groovy-and-Groovy-xpost-to-groovy-user-list-td2801779.html

摘自: http://www.jroller.com/melix/entry/using_custom_groovy_metaclasses_to
统计信息错误解决cms near TYPE=MyISAM DEFAULT CHARACTER SET gbk - 和申 - 和申的个人主页
  评论这张
 
阅读(1241)| 评论(0)
推荐 转载

历史上的今天

在LOFTER的更多文章

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2016