Threading and concurrency

Metakit 1.8.5 does not support concurrent access to its datafile. It does support threading in a limited way. Multi-user/concurrent access is by far the most requested feature (second on the list are the requests to make Metakit even faster and/or to let it handle much larger files - see the Performance page).

Some facts about Metakit 1.8.5

• You can the Tcl interface (Mk4tcl) recently, to enforce the above rules. That's the easy part. The main issue which must be solved is cache coherence. Right now, there's some information which gets loaded on file open, and then cached. This caching increases as the file is being modified. To solve the problem, either some form of IPC, or some sort of change propagation is needed.

So that's it? No concurrent use at all?

The suggested way around this, is to introduce a client/server mechanism (this can also be used inside a single multi-threaded app, btw). For C++, this might be a lot of work - but the concept is definitely not new or complex. In Tcl, a server and client each take no more than about a dozen lines of code. Such an approach will not be applicable in all cases, I am looking at several ways to provide more solutions.

August 1999, good news: a multi-threaded version of MK has been designed, which supports multiple readers and one writer - this is expected to be ready and fully tested well before the end of 1999.

April 2000, bad news: it hasn't happened. But as the New MK transaction modes page shows, there is tangible progress. It is slightly different from the original design, and it's working code.

August 2003 : Metakit Locking Class

Jeffrey Kay (mailto:jeff@k2.com) has very kindly donated the source and an example for a c++ Metakit locking class. JCW Has kindly uploaded it to http://www.equi4.com/pub/etc/mklock.zip

From The discussion forum :- " I've solved the problem by creating a couple of "locking" classes to ensure that I don't accidentally access a store from multiple threads. The first class opens up a db file. I've designed my code so that I have only one class managing a db file, so that class is derived from shDBManager. The second class retrieves a reference to a given store and sets a critical section to ensure that only one thread is working on the store. To protect myself, I usually make shStore a stack variable. "

A further note from Jeff "I only compiled the debug version - dependencies are to ../../metakit-2.4.7. The code works fine with 2.9.2 or whatever the most recent MK version is - I just didn't happen to have it on the system that I'm sitting at right now."

Many thanks Jeff for being such a good sport.

If people wish to see another way of doing it, please also see John Taylor's work 'openrepo' (repository), from www.shift-right.com .... Ex John : "If anyone is interested. In the course of using Metakit in our products at work, we have created a Multi-threaded C++ wrapper for Metakit. It is a client/server model using ITC (inter-thread-communications) to isolate all database actions for a given a database to a single thread. The wrapper is part of my company's open source C++ library."

I hope putting such articles together here in one place helps.

cheers, Garth