Disclaimer: This will be another post on events and extensibility points of a framework, but I can’t resist!
Recently, on one of my pet projects, I wanted to use db4o, and the problem was I should have explicit POIDs on my objects. Db4o has two mechanisms for Id stuff, one is GetId function, which is the physical address of the object on the disk and the second was the UUID that is used for replication purposes. The first one is fast but you can’t rely on it because the physical address may change after defragmentation, and the other is not very readable. This is the time I started to think of Id Generators for db4o. My friend Dario Quintana from NHibernate definitely had same needs(who doesn’t?) and as one having the right spirit, he implemented similar feature for db4o but this time with touching the codebase. My feature will use db4o Extensibility Point and will allow you to have several different Id Generators. I will use ideas from NHibernate POID generators specifically HiLo algorithm.
What is HiLo algorithm?
It is a nice algorithm that allows you to have ids without putting locks, thus slowing down the things. The algorithm works as follows.
Each Generator requests a Hi value from database and with every request, this value is incremented. The generator then uses a magic formula (Hi-1)*Capacity+(++Lo) value. Lo values are generated by the generator and incremented with every new object saved into database. Hi value is stored in database while Lo value is stored in a variable in the client. When Lo becomes larger than Capacity (that is Lo values are exhausted), the generator requests a new Hi value from the database. As you see, we don’t issue many requests for Hi values thus there is practically no overhead on the database side. The larger the capacity, the less there is need to request for new id thus the less overhead. However, Hi Capacity means there will be ranges that will never ever be used, and people dislike it (why does it bother them is a question for me, too). Practically Int32 can handle values up to 2,147,483,647 while Int64 can handle up to 9,223,372,036,854,775,807 (no I could not say this number in spoken English but learnt that is is nine quantillion). Having Capacity=32767 will allow you to have 281483566907400 Hi values and assuming you have 100 servers… wait it is obvious that you cannot consume all those in several million years?
How can we integrate it into db4o?
Well, this is easy. Db4o provides us an event mechanism that can be used for such stuff. It has several events namely Creating, Created, Updating, Updated etc. All we have to do is to catch this event on the server side, and increment Hi values there. There is one thing to be careful about: Db4o events may fool you. In case of Client/Server mode you should call the below code on ServerContainer, otherwise you may have to use another ObjectClient as each object container runs in its own transaction. If you are using it as embedded database, then you won’t have any problem with it.
Now, I define a contract called IIdGenerator
public interface IIdGenerator
{
object Generate();
}
public interface IIdGenerator<T>:IIdGenerator
{
new T Generate();
}
Pretty self explanatory, isnt’ it? Now it comes to persistent id generator, which is for now the increment generator
public class IncrementGenerator:BaseIdGenerator<long>
{
private readonly Type type;
private readonly IObjectContainer container;
private readonly string semaphoreName;
public IncrementGenerator(Type type,IObjectContainer container)
{
this.type = type;
this.container = container;
this.semaphoreName = string.Format("id_gen_{0}", this.type.Name);
}
public override long Generate()
{
long valueToBeReturned;
while (!container.Ext().SetSemaphore(semaphoreName, 1000)) ;//Do some busy wait
IObjectSet set=this.container.QueryByExample(new IncrementTypeValuePair{Type=type});
IncrementTypeValuePair pair;
if (set.Count == 0)
pair = new IncrementTypeValuePair {Type = type};
else
pair = set[0] as IncrementTypeValuePair;
valueToBeReturned = ++pair.Value;
container.Store(pair);
container.Ext().ReleaseSemaphore(semaphoreName);
return valueToBeReturned;
}
}
There we used Semaphore in order not to have concurrency issue, otherwise we could give very same id to different objects. This id generator will go to db everytime an id is requested, and it will create some bottleneck in case several hundreds of entities are inserted in a second. As an enhancement to this generator, we’ll inherit from this generator and create HiLoGenerator.
[MethodImpl(MethodImplOptions.Synchronized)]
public override long Generate()
{
if(currentLo>=capacity)
{
currentHi=base.Generate();
currentLo = 0;
}
return (currentHi - 1)*capacity + (++currentLo);
}
Exactly what I told in oral.
As the last step, I designed an ugly fluent interface for this, which looks like the following
var serverContainer = this.server.Ext().ObjectContainer();
serverContainer.IdMap(Map<Person>.On(x => x.Id).SetGenerator(new HiLoGenerator(3, typeof(Person),serverContainer)));
var person = new Person();
container.Store(person);
The event wiring stuff is done a bit ugly too.
void registry_Creating(object sender, CancellableObjectEventArgs args)
{
object item = args.Object;
foreach (var map in idMaps)
{
if(map.EntityType.IsAssignableFrom(item.GetType()))
{
MemberExpression member = ((LambdaExpression)map.Expression).Body as MemberExpression;
((PropertyInfo) member.Member).SetValue(item, map.Generator.Generate(),null);
break;
}
}
}
Whole code can be found if you follow this link
db4o, algorithms