Tuna Toksoz's weblog

Disclaimer: This will be another post on events and extensibility points of a framework, but I can’t resist!

Recently, on one of my pet projects, I wanted to use db4o, and the problem was I should have explicit POIDs on my objects. Db4o has two mechanisms for Id stuff, one is GetId function, which is the physical address of the object on the disk and the second was the UUID that is used for replication purposes. The first one is fast but you can’t rely on it because the physical address may change after defragmentation, and the other is not very readable. This is the time I started to think of Id Generators for db4o. My friend Dario Quintana from NHibernate definitely had same needs(who doesn’t?) and as one having the right spirit, he implemented similar feature for db4o but this time with touching the codebase. My feature will use db4o Extensibility Point and will allow you to have several different Id Generators. I will use ideas from NHibernate POID generators specifically HiLo algorithm.

What is HiLo algorithm?

It is a nice algorithm that allows you to have ids without putting locks, thus slowing down the things. The algorithm works as follows.

Each Generator requests a Hi value from database and with every request, this value is incremented. The generator then uses a magic formula (Hi-1)*Capacity+(++Lo) value. Lo values are generated by the generator and incremented with every new object saved into database. Hi value is stored in database while Lo value is stored in a variable in the client. When Lo becomes larger than Capacity (that is Lo values are exhausted), the generator requests a new Hi value from the database. As you see, we don’t issue many requests for Hi values thus there is practically no overhead on the database side. The larger the capacity, the less there is need to request for new id thus the less overhead. However, Hi Capacity means there will be ranges that will never ever be used, and people dislike it (why does it bother them is a question for me, too). Practically Int32 can handle values up to 2,147,483,647 while Int64 can handle up to 9,223,372,036,854,775,807 (no I could not say this number in spoken English but learnt that is is nine quantillion). Having Capacity=32767 will allow you to have 281483566907400 Hi values and assuming you have 100 servers… wait it is obvious that you cannot consume all those in several million years?

How can we integrate it into db4o?

Well, this is easy. Db4o provides us an event mechanism that can be used for such stuff. It has several events namely Creating, Created, Updating, Updated etc. All we have to do is to catch this event on the server side, and increment Hi values there. There is one thing to be careful about: Db4o events may fool you. In case of Client/Server mode you should call the below code on ServerContainer, otherwise you may have to use another ObjectClient as each object container runs in its own transaction. If you are using it as embedded database, then you won’t have any problem with it.

Now, I define a contract called IIdGenerator

public interface IIdGenerator
{
    object Generate();
}
public interface IIdGenerator<T>:IIdGenerator
{
    new T Generate();
}

Pretty self explanatory, isnt’ it? Now it comes to persistent id generator, which is for now the increment generator

public class IncrementGenerator:BaseIdGenerator<long>
{
    private readonly Type type;
    private readonly IObjectContainer container;
    private readonly string semaphoreName;
    public IncrementGenerator(Type type,IObjectContainer container)
    {
        this.type = type;
        this.container = container;
        this.semaphoreName = string.Format("id_gen_{0}", this.type.Name);
    }

    public override long Generate()
    {
        long valueToBeReturned;
        while (!container.Ext().SetSemaphore(semaphoreName, 1000)) ;//Do some busy wait
        IObjectSet set=this.container.QueryByExample(new IncrementTypeValuePair{Type=type});
        IncrementTypeValuePair pair;

        if (set.Count == 0)
            pair = new IncrementTypeValuePair {Type = type};
        else
            pair = set[0] as IncrementTypeValuePair;
        valueToBeReturned = ++pair.Value;
        container.Store(pair);
        container.Ext().ReleaseSemaphore(semaphoreName);
        return valueToBeReturned;
    }
}

There we used Semaphore in order not to have concurrency issue, otherwise we could give very same id to different objects. This id generator will go to db everytime an id is requested, and it will create some bottleneck in case several hundreds of entities are inserted in a second. As an enhancement to this generator, we’ll inherit from this generator and create HiLoGenerator.

[MethodImpl(MethodImplOptions.Synchronized)]
public override long Generate()
{
    if(currentLo>=capacity)
    {
        currentHi=base.Generate();
        currentLo = 0;
    }

    return (currentHi - 1)*capacity + (++currentLo);
}

Exactly what I told in oral.

As the last step, I designed an ugly fluent interface for this, which looks like the following

var serverContainer = this.server.Ext().ObjectContainer();
serverContainer.IdMap(Map<Person>.On(x => x.Id)
         .SetGenerator(new HiLoGenerator(3, typeof(Person),serverContainer)));
var person = new Person();
container.Store(person);

The event wiring stuff is done a bit ugly too.

void registry_Creating(object sender, CancellableObjectEventArgs args)
{
    object item = args.Object;
    foreach (var map in idMaps)
    {
        if(map.EntityType.IsAssignableFrom(item.GetType()))
        {
            MemberExpression member = ((LambdaExpression)map.Expression).Body as MemberExpression;
            ((PropertyInfo) member.Member).SetValue(item, map.Generator.Generate(),null);
            break;
        }
    }
}

Whole code can be found if you follow this link

db4o, algorithms

UPDATE 3: Exception throwing for business rules is not a good practice, making this post almost useless.
UPDATE 2: Did you notice the duplication in event handling?
UPDATE 1: Jeremy Skinner was kind enough to respond to my feature request, please look at the first block. Thanks Jeremy!

I have been trying to implement validation for BlogSharp. There are several validation frameworks including Castle Validator but it didn’t feel good to put attributes on the entity itself. I wondered if there is a 3.5 style of doing it and thankfully Jeremy Skinner provided a good framework that uses Lambda Expressions which is called Fluent Validation. Its syntax is very similar to the that of Fluent NHibernate.

public class PostCommentValidator : ValidatorBase<PostComment>
{
    public PostCommentValidator()
    {
        RuleFor(x => x.Comment).NotEmpty();

        RuleFor(x => x.Email).NotEmpty();
        RuleFor(x => x.Email).EmailAddress();        RuleFor(x => x.Email).NotEmpty().And().EmailAddress();
        RuleFor(x => x.Web).Url().When(x=>!string.IsNullOrEmpty(x.Web));
    }
}

It is also easy to extend its validation capacities. All you need to do is to implement IPropertyValidator<T>. For my case, I needed to have Url validation and it was enough to inherit from RegularExpressionValidator.

public class UrlValidationRule<T>:RegularExpressionValidator<T>
{
    public UrlValidationRule():base(@"URLRegularExpressionHere")
    {
        
    }
}

You may also want to create an extension method in order to have a uniform syntax.

public static class UrlValidationExtension
{
    public static IRuleBuilderOptions<T, string> Url<T>(this IRuleBuilder<T, string> ruleBuilder)
    {
        return ruleBuilder.SetValidator(new UrlValidationRule<T>());
    }
}

There are some additions that i made in ValidatorBase<T>. Currently it returns ValidationResult, which has all the information about the validation process. I made it throw Exception in order not to deal with layering.

Nothing very special up to this point.

What I wanted to tell in detail today is how to implement validation using Db4o, Castle and Fluent Validation stack. Even though I was to do the validation at Service level, I decided that it may be better to have it at persistence level.

Db4o has events (did I mention that I like events?) that take place after and before various operations including Activate, Create, Update, Delete, Commit. The best place to implement this validation is to use the events Creating and Updating which are raised_before_ the updated/added object is stored back in Db4o. I needed a way to integrate them with my home made Db4o Facility that was inspired by Castle NHibernate Integration. I created an interface like the one below

public interface IDb4oInitializationHandler
{
    void HandleObjectContainerCreated(IExtObjectContainer extObjectContainer);
}

Implementors will be called just after the IObjectContainer so that we will have the opportunity to wire up our events. Currently you can’t specify a specific InitializationHandler for a specific container, but I plan to implement it soon.

public void HandleObjectContainerCreated(IExtObjectContainer extObjectContainer)
{
    var factory = EventRegistryFactory.ForObjectContainer(extObjectContainer);
    factory.Creating += ValidationHandler;
    factory.Updating += ValidationHandler;
}protected void ValidationHandler(object sender, CancellableObjectEventArgs args)
{
    try
    {
        ValidateObject(args.Object);
    }
    catch(ValidationException ex)
    {
        args.Cancel();
        throw ex;
    }
}

and in the handlers of the events, I check if the stored object has validator associated with it and then validate.

protected virtual void ValidateObject<T>(T obj)
{
    var type = obj.GetType();
    var validatorType=typeof (IValidatorBase<>).MakeGenericType(type);
    if(container.HasComponent(validatorType))
    {
        var validator = container.Resolve(validatorType) as IValidatorBase;
        validator.ValidateAndThrowException(obj);                
    }
}

The exception is caught at the Controller, then is passed to the ModelState.

try
{
    postService.AddComment(comment);
}
catch(ValidationException vex)
{
    this.ModelState.AddValidationExceptionToModel("comment",vex);
}
            

AddValidationExceptionToModel is an extension method

public static void AddValidationExceptionToModel(this ModelStateDictionary model, string prefix,ValidationException exception)
{
    var errors=exception.Errors;
    foreach (var error in errors)
    {
        model.AddModelError(string.Format("{0}.{1}",prefix,error.PropertyName), error.Message);
    }
}

 

The result is the following:

It is as easy as this, thanks to the extensibility of Db4o and Fluent Validation frameworks.

If you like it, don't forget to kick and/or shout it

db4o, mvc, validation