The new database model of Eratosthenes
This is a detailed discussion for Eratosthenes’s Issue #21.
The primary feature we are trying to implement on Eratosthenes is to include library version in its various libraries. Currently, Eratosthenes will always hold the latest release of external library, and make sure the archive of each library is up-to-date. However, this is not a desired feature since Arduino now has so many different configurations, and it is not always a good idea to provide the latest library due to the compatibility issue. So the best approach is we provide different versions in our system, and let user to choose what version of library they want (just like what we do in npm’s package.json or composer’s composer.json). If they do not specify a version, then by default the latest version will be provided. This is a good practice as we can enhance user’s usability and system’s functionality, and meanwhile most likely user’s current experience will not be affected as well.
So how do we do that? Currently there are several consideration in our design:
- There may be multiple versions for each library
- For each version, the supporting architectures may also be different
- The version may not always be perfect semantic versioning (weird name such as '@first-version.1.0‘ are expected)
- In future, each user of the system (currently only codebender) may update their default supported version for each library
First, in each library, there may be more than one version, so we need to find a easy way to maintain them. The solution we proposed is to create a new table called Version. In this table, it holds a foreign key
library_id that refer to the library id, and we use a ManyToOne relationship with the Library table to describe it. In this way, we can easily find which library a version belongs to, and also from a given library, find all the versions it has.
Secondly, as Arduino is an open source platform, people can create their own Arduino controller with different hardware architecture. Accordingly, there may be different library for different architecture as well. The solution we proposed is firstly create an Architecture table to store the architectures used by various Arduino. Then we use a relationship to connect architecture with the version of the library. So for a library given, we can find out all versions it has; by using this table, we know what is the supported architectures for this version.
Next, since there is no compulsory standard to name the version (although the semantic versioning is recommended), we need to find a way to unify the naming. Also, it the name uses some special characters, the file system may not be able to display them correctly.
One possible solution is we hash all the version name, so we will always get unique and fixed-length name. But on the other hand, the system admin will have a hard time to do the maintaining work because it is not human readable.
Another approach is we will make use the version id in the database, plus the adjusted version number. e.g. 130-1.1 (the id in the Version table). In this way, the filename is always compatible with the file system, and it is also easier to maintain things as the name is human readable. However, that would means you will always need to insert the library into the database first, then generate the name. In future, if the database has some problem, then basically we need to rebuild the whole file system!
Finally, our decided approach is we will manually adjust the version name when we are creating this version for the system. As we have discussed, we discovered that only a small amount of libraries are using special characters in their version names. Therefore, when we are generating the version to be used on file system, we will escape those characters. e.g. A version named ‘1.3.?’ will be escaped into things like ‘1.3.-‘. In such way, we can decouple the database from the versioning, at the same time the readability is guaranteed as well.
So the folder structure for each library is as follows:
Finally, in order to store the default version used by each user, we need to create another two tables: Partner and Preference, where the authentication information for different partners(users) as well as their default version preference will be hold.
The Database diagram is as following:
We are creating Doctrine entities according to this design now, and next step we will start to write fixture as well as migration for our models.