Table of Contents

Data Persistence Abstraction

Persistence Definition

In the context of the Tlabs Library data persistence means

  • Accessing large data sets being stored in databases by the means of querying and filtering

  • Making data (probably being processed with complex algorithms) persistent (to free up precious memory and make the data survive the inevitable next system reboot)

This is a long standing task for computer systems and many software applications spend a substantial part of their code base on this problem.
After having spent quite some work on data persistence the general finding is to keep high focus on following strategic goals:

Goals

  • Substantially mitigate the object–relational impedance mismatch by usage of an ORM framework
    (i.e. there are such fundamental differences between OO programing and relational data storage in how data is represented and being related, that it requires a quite sophisticated sort of adapter to make them better collaborate)

  • Treat data persistence with being agnostic of the

    1. Database type
      (Application developers should concentrate on the business domain problem and not be required to concern with the specialties of database systems of different type and vendor)

    2. Specific ORM framework
      (Since there are several options available and it is expected to see even more even better to come, it is a good idea to stay open for the future and be prepared to choose and change.)

  • For being future proof stay true with an agile design
    (Even if we are talking about persistence and we assume the longevity of data, that does not necessarily require any third party tools used by the application to access the information represented with the data to stay unchanged as well.)

Solution

To meet with these strategic goals the Tlabs Library provides some tactical tools and implementations of best practices. Together these are forming a thin abstraction layer on top of an effective ORM framework.

ORM Abstraction

From an application programmers view those concepts like a Hibernate Session or an Entity Framework DbContext are replaced with an uniform IDataStore.

The IDataStore is responsible to synchronize a subset of the persistent state of a database with the corresponding transient set of entity objects in application memory (typically represented with the result of one ore more queries). It is exemplary for a service with a scoped lifecycle that keeps its state in the scope (or duration) of a logical data transaction. An application developer would decide whether to commit any changes applied to the in-memory entities explicitly or implicitly at the end of the service scope (or transaction).

The persistence abstraction of the Tlabs Library is encouraging the application of the data repository pattern. The IRepo<> interface and its base implementations provide a strongly typed view on persistent data which in combination with LINQ allows to handle data in a way that is approximating that of familiar collections.
Also it recommends to encapsulate any serious data queries as dedicated methods of a repository class to centralize queries that potentially might require special database index related optimizations.

Even if many ORM frameworks are presenting their own specific data query languages the IRepo<>/IDataStore abstraction mandates for the usage of the excellent .NET/C# integrated LINQ technology. Supplemented with a lightweight FilterParam/QueryFilter abstraction to handle closely UI related cases, where the requirements demand the incremental incorporation of additional filters and sorters to an existing query.

Data Entity Modelling and Relational Mapping

With the Tlabs Library, persistent data is in generell modelled with Entity classes. These are plain .NET classes (with no specific inheritance required) with the only requisite that Entities must keep some intrinsic identity value suitable to be directly mapped to a database primary key. (Regardless of any other properties, two Entities are considered different if their identity values are different.)

Any ORM framework requires to have the mapping of

  • Entities to database tables

  • Entity properties to table columns

  • the Entity relations

expressed in its specific form.
With the Tlabs Library these mappings are getting packaged into one or several (service) objects of type IDStoreConfigModel. It is explicitly supported to have these data model configurations spread across several application modules (or bound contexts). A dedicated implementation service of the IDStoreCtxConfigurator is then responsible to assemble all these mappings into the final ORM session context.

If the database and its schema are already existing or must be managed exclusively with a dedicated database tool (e.g. by the corporate database department) - up to here we are all set.

In case we also need to generate the entire database schema and want to be able to create a database from scratch on application startup if it does not exist already, the Tlabs Library can support this as well with utilities, documentation and examples. (This is also known as the code first approach.)
Additionally supported are the cases where

  • the database must not only initialized with a schema but also with initial data
    (data seeding)

  • the schema of the database needs to be changed (according to new and changed Entities and their mappings), which might also require to update exiting data as well
    (data migration)

Database Server Configuration

The Tlabs Library aims to allow for almost everything being configurable. Same with persistence.

This is the purpose of special abstract template classes like DbServerConfigurator<> that can be utilized to configure the underlying ORM framework in order

  • to use the application’s data model

  • a specific database type (provider)

  • a desired database connection

and all this in a very concise and vastly database type agnostic way.

Further Reading

Tlabs Library Persistence Guide