Home >

Id Generation for db4o

19. May 2009

Disclaimer: This will be another post on events and extensibility points of a framework, but I can’t resist!

Recently, on one of my pet projects, I wanted to use db4o, and the problem was I should have explicit POIDs on my objects. Db4o has two mechanisms for Id stuff, one is GetId function, which is the physical address of the object on the disk and the second was the UUID that is used for replication purposes. The first one is fast but you can’t rely on it because the physical address may change after defragmentation, and the other is not very readable. This is the time I started to think of Id Generators for db4o. My friend Dario Quintana from NHibernate definitely had same needs(who doesn’t?) and as one having the right spirit, he implemented similar feature for db4o but this time with touching the codebase. My feature will use db4o Extensibility Point and will allow you to have several different Id Generators. I will use ideas from NHibernate POID generators specifically HiLo algorithm.

What is HiLo algorithm?

It is a nice algorithm that allows you to have ids without putting locks, thus slowing down the things. The algorithm works as follows.

Each Generator requests a Hi value from database and with every request, this value is incremented. The generator then uses a magic formula (Hi-1)*Capacity+(++Lo) value. Lo values are generated by the generator and incremented with every new object saved into database. Hi value is stored in database while Lo value is stored in a variable in the client. When Lo becomes larger than Capacity (that is Lo values are exhausted), the generator requests a new Hi value from the database. As you see, we don’t issue many requests for Hi values thus there is practically no overhead on the database side. The larger the capacity, the less there is need to request for new id thus the less overhead. However, Hi Capacity means there will be ranges that will never ever be used, and people dislike it (why does it bother them is a question for me, too). Practically Int32 can handle values up to 2,147,483,647 while Int64 can handle up to 9,223,372,036,854,775,807 (no I could not say this number in spoken English but learnt that is is nine quantillion). Having Capacity=32767 will allow you to have 281483566907400 Hi values and assuming you have 100 servers… wait it is obvious that you cannot consume all those in several million years?

How can we integrate it into db4o?

Well, this is easy. Db4o provides us an event mechanism that can be used for such stuff. It has several events namely Creating, Created, Updating, Updated etc. All we have to do is to catch this event on the server side, and increment Hi values there. There is one thing to be careful about: Db4o events may fool you. In case of Client/Server mode you should call the below code on ServerContainer, otherwise you may have to use another ObjectClient as each object container runs in its own transaction. If you are using it as embedded database, then you won’t have any problem with it.

Now, I define a contract called IIdGenerator

public interface IIdGenerator
{
    object Generate();
}
public interface IIdGenerator<T>:IIdGenerator
{
    new T Generate();
}

Pretty self explanatory, isnt’ it? Now it comes to persistent id generator, which is for now the increment generator

public class IncrementGenerator:BaseIdGenerator<long>
{
    private readonly Type type;
    private readonly IObjectContainer container;
    private readonly string semaphoreName;
    public IncrementGenerator(Type type,IObjectContainer container)
    {
        this.type = type;
        this.container = container;
        this.semaphoreName = string.Format("id_gen_{0}", this.type.Name);
    }

    public override long Generate()
    {
        long valueToBeReturned;
        while (!container.Ext().SetSemaphore(semaphoreName, 1000)) ;//Do some busy wait
        IObjectSet set=this.container.QueryByExample(new IncrementTypeValuePair{Type=type});
        IncrementTypeValuePair pair;

        if (set.Count == 0)
            pair = new IncrementTypeValuePair {Type = type};
        else
            pair = set[0] as IncrementTypeValuePair;
        valueToBeReturned = ++pair.Value;
        container.Store(pair);
        container.Ext().ReleaseSemaphore(semaphoreName);
        return valueToBeReturned;
    }
}

There we used Semaphore in order not to have concurrency issue, otherwise we could give very same id to different objects. This id generator will go to db everytime an id is requested, and it will create some bottleneck in case several hundreds of entities are inserted in a second. As an enhancement to this generator, we’ll inherit from this generator and create HiLoGenerator.

[MethodImpl(MethodImplOptions.Synchronized)]
public override long Generate()
{
    if(currentLo>=capacity)
    {
        currentHi=base.Generate();
        currentLo = 0;
    }

    return (currentHi - 1)*capacity + (++currentLo);
}

Exactly what I told in oral.

As the last step, I designed an ugly fluent interface for this, which looks like the following

var serverContainer = this.server.Ext().ObjectContainer();
serverContainer.IdMap(Map<Person>.On(x => x.Id)
         .SetGenerator(new HiLoGenerator(3, typeof(Person),serverContainer)));
var person = new Person();
container.Store(person);

The event wiring stuff is done a bit ugly too.

void registry_Creating(object sender, CancellableObjectEventArgs args)
{
    object item = args.Object;
    foreach (var map in idMaps)
    {
        if(map.EntityType.IsAssignableFrom(item.GetType()))
        {
            MemberExpression member = ((LambdaExpression)map.Expression).Body as MemberExpression;
            ((PropertyInfo) member.Member).SetValue(item, map.Generator.Generate(),null);
            break;
        }
    }
}

Whole code can be found if you follow this link

,

Comments

5/19/2009 5:40:13 PM #
Why not use a system that does not force you to have an ID in your entity ?
5/19/2009 5:47:06 PM #
Do you mean why to use or why not to use? Db4o doesn't force me to have an ID.
A system that allows me to use its own mechanism, such as NH, is good but sometimes I don't need performance of RDBMS and full features of NH, and db4o is just a quick start, that is the only reason
5/20/2009 5:07:52 AM #
Trackback from Community News

Id Generation for db4o
5/20/2009 12:05:05 PM #
Trackback from DotNetKicks.com

Id Generation for db4o
10/9/2010 4:18:58 AM #
Shreenathji - Merged Form of Krishnradha
12/9/2010 9:51:51 PM #
Pingback from softwareobjects.net

IDb4oConvention.cs · Software Objects Source Code
12/13/2010 4:12:16 PM #
We are also very happy to now offer web hosting services, regardless of whether or not you want us to build you a website;
There’s no need to go scouring the net for a great web hosting service to compliment your new site…
GFsoul does web hosting and takes care of all the important details like domain name registration.
Our design and hosting services perfectly compliment each other; we are offering customers true “one-stop shopping” experience, and at extremely affordable prices. You won’t find customized web design and hosting at these amazingly low prices anywhere buy here.
Our web hosting services features all of the same appointments that other (better known) web hosting services can offer you, but with lots of great tech support, customization and a lower overall cost.
12/26/2010 4:38:02 PM #
Very informative post and really help me a lot and I am going to bookmark this blog for not to miss anything.
1/2/2011 4:39:04 AM #
Thanks for taking the time to share this, I feel strongly about it and love reading more on this topic. If possible, as you gain knowledge, would you mind updating your blog with extra information? It is extremely helpful for me.
1/10/2011 4:00:25 AM #
amazing post! thanks for sharing!bru
jrnail23
jrnail23
1/11/2011 1:43:57 PM #
Unfortunately this code no longer works with latest version of Db4o.
The event model has become more restrictive, and when you attempt to update the generator state in the event handler, you'll receive a nasty exception: "Db4objects.Db4o.Ext.Db4oIllegalStateException : Objects must not be updated in callback"

Lovely, isn't it?
1/12/2011 11:53:29 AM #
I wanted to thank you for this great read!! I definitely enjoying every little bit of it.I have you bookmarked to check out new stuff you post.
3/22/2011 6:09:10 PM #
Very interesting site you have. I come here often to see what new articles you have written.
3/22/2011 9:08:42 PM #
I actually adore the facts this specific online site constantly provides. Top-notch!
3/23/2011 4:19:41 AM #
I am often to blogging and i actually appreciate your content. The article has actually peaks my interest. I am going to bookmark your website and hold checking for brand new information.
3/23/2011 4:38:50 AM #
I think this is among the most important information for me. And i'm glad reading your article. But should remark on some general things, The web site style is wonderful, the articles is really excellent : D. Good job, cheers
3/23/2011 9:30:52 AM #
I like the helpful information you provide in your articles. I will bookmark your blog and check again here regularly. I am quite certain I will learn plenty of new stuff right here! Best of luck for the next!
11/10/2011 9:49:08 AM #
Pingback from xsourcecode.com

IDb4oConvention.cs – 1320911319960 - Source Code
Comments are closed